WO2022116725A1 - 数据处理方法、装置、设备及存储介质 - Google Patents

数据处理方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2022116725A1
WO2022116725A1 PCT/CN2021/125214 CN2021125214W WO2022116725A1 WO 2022116725 A1 WO2022116725 A1 WO 2022116725A1 CN 2021125214 W CN2021125214 W CN 2021125214W WO 2022116725 A1 WO2022116725 A1 WO 2022116725A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
parameter
fusion
segment
target
Prior art date
Application number
PCT/CN2021/125214
Other languages
English (en)
French (fr)
Inventor
程勇
陶阳宇
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to JP2023513949A priority Critical patent/JP7512518B2/ja
Priority to EP21899759.1A priority patent/EP4203398A4/en
Publication of WO2022116725A1 publication Critical patent/WO2022116725A1/zh
Priority to US17/969,159 priority patent/US20230040570A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/42—Loop networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/256—Integrating or interfacing systems involving database management systems in federated or virtual databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/12—Avoiding congestion; Recovering from congestion

Definitions

  • the present application relates to the technical field of artificial intelligence, and in particular, to a data processing method, apparatus, device and storage medium.
  • Federated Learning Federated Learning
  • Federated Learning can complete the training tasks of machine learning models through multi-party cooperation.
  • the multi-party collaborative federated learning adopts the customer service-server architecture, and the federated learning system includes a central federated server. shape topology.
  • Embodiments of the present application provide a data processing method, apparatus, device, and storage medium, and the technical solutions are as follows.
  • a data processing method executed by a first device in a data processing system, the data processing system includes at least three devices, and the devices in the data processing system are connected in a ring structure, the method includes :
  • the second fusion parameters include model parameters determined by the at least three devices in the data processing system
  • the target model parameter of the first service processing model is determined.
  • a data processing method executed by a second device in a data processing system, the data processing system includes at least three devices, the devices in the data processing system are connected in a ring structure, the method includes :
  • a target model parameter of the second business processing model is determined.
  • a data processing system includes at least three devices, the at least three devices include a first device and at least two second devices, the first devices are the data processing Any one of the systems; the devices in the data processing system are connected in a ring structure;
  • the first device is used for acquiring first sample data of the target service; training a first service processing model based on the first sample data to obtain first model parameters of the first service processing model;
  • the second device connected to the first device sends the first model parameter, the second device determines the first fusion parameter, and sends the first fusion parameter to other second devices, where the first fusion parameter is all
  • the second device is obtained based on the received first model parameters and the second model parameters determined by itself, and the second device is a device other than the first device in the data processing system; obtain the first Two fusion model parameters, the second fusion model parameters include model parameters determined by the at least three devices in the data processing system; according to the second fusion model parameters, determine the target model of the first business processing model parameter;
  • Any second device is used to obtain the second sample data of the target service; the second service processing model is trained based on the second sample data to obtain the second model parameters of the second service processing model; the first model parameters determined by the first device; fuse the first model parameters with the second model parameters determined by itself to obtain first fusion parameters; send the first fusion parameters to the data processing system other equipment; determine the target model parameters of the second service processing model.
  • a data processing apparatus comprising:
  • an acquisition module used to acquire the first sample data of the target business
  • a training module configured to train a first business processing model based on the first sample data to obtain first model parameters of the first business processing model
  • a sending module configured to send the first model parameter to a second device connected to the first device, the second device determines the first fusion parameter, and sends the first fusion parameter to other second devices , the first fusion parameter is obtained by the second device based on the received first model parameter and the second model parameter determined by itself, and the second device is obtained by dividing the first model parameter in the data processing system a device other than a device;
  • the obtaining module is further configured to obtain second fusion parameters, where the second fusion parameters include model parameters determined by the at least three devices in the data processing system;
  • a determination module configured to determine target model parameters of the first service processing model according to the second fusion parameters.
  • a data processing apparatus comprising:
  • an acquisition module for acquiring the second sample data of the target business
  • a training module configured to train a second business processing model based on the second sample data to obtain second model parameters of the second business processing model
  • a receiving module configured to receive a first model parameter determined by a first device, where the at least three devices include the first device
  • a fusion module configured to fuse the first model parameter with the second model parameter determined by itself to obtain the first fusion parameter
  • a sending module configured to send the first fusion parameter to other devices in the data processing system, where the devices in the data processing system are connected in a ring structure;
  • the receiving module is further configured to determine target model parameters of the second service processing model.
  • an electronic device comprising one or more processors and one or more memories, the one or more memories storing at least one computer program, the at least one computer program consisting of The one or more processors are loaded and executed to implement various optional implementations of the above-described data processing methods.
  • a computer-readable storage medium in which at least one computer program is stored, the at least one computer program is loaded and executed by a processor to realize various optional implementation manners of the above data processing method .
  • a computer program product or computer program comprising one or more pieces of computer programs stored in a computer-readable storage medium.
  • the one or more processors of the electronic device can read the one or more computer programs from a computer-readable storage medium, the one or more processors execute the one or more computer programs to enable the electronic device to The data processing method of any one of the above possible implementations is performed.
  • FIG. 1 is a schematic diagram of a sample space and a feature space provided by an embodiment of the present application
  • FIG. 2 is a flowchart of a data processing system provided by an embodiment of the present application.
  • FIG. 3 is a flowchart of a data processing system provided by an embodiment of the present application.
  • FIG. 5 is a flowchart of a data processing method provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a data processing system provided by an embodiment of the present application.
  • FIG. 7 is a flowchart of a data processing method provided by an embodiment of the present application.
  • FIG. 8 is a flowchart of a data processing method provided by an embodiment of the present application.
  • FIG. 9 is a flowchart of a data processing method provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a distributed system 100 provided by an embodiment of the present application applied to a blockchain system;
  • Block Structure is a schematic diagram of a block structure (Block Structure) provided by an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • 15 is a structural block diagram of a terminal provided by an embodiment of the present application.
  • FIG. 16 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • first, second, etc. are only used to distinguish the devices.
  • first sample data, the second sample data, the third sample data, etc. are only used to distinguish the sample data.
  • a first device could be termed a second device, and, similarly, a second device could be termed a first device, without departing from the scope of the various examples. Both the first device and the second device are devices, and in some cases, can be separate and distinct devices.
  • At least one in this application means one or more, and the term “plurality” in this application means two or more, for example, a plurality of devices means two or more equipment.
  • the meaning of "at least two” means two or more.
  • the meaning of "at least three” means three or more.
  • the size of the sequence number of each process does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not be used in the embodiment of the present application. Implementation constitutes any limitation.
  • determining B based on A does not mean determining B based only on A, but can also determine B based on A and/or other information.
  • the term “if” may be interpreted to mean “when” or “upon” or “in response to determining” or “in response to detecting.”
  • the phrases “if it is determined" or “if a [statement or event] is detected” can be interpreted to mean “when determining" or “in response to determining... ” or “on detection of [recited condition or event]” or “in response to detection of [recited condition or event]”.
  • the data processing methods provided in the embodiments of the present application are applied to artificial intelligence technology.
  • the following is a brief introduction to artificial intelligence.
  • AI Artificial Intelligence
  • Federated learning has gradually become an emerging artificial intelligence basic technology.
  • the design goal of federated learning is to ensure information security during big data exchange, protect the privacy of terminal data and personal data, and ensure legal compliance.
  • Efficient machine learning is carried out between multiple computing nodes.
  • the machine learning algorithms that can be used in federated learning are not limited to neural networks, but also include important algorithms such as random forests. Federated learning is expected to become the basis for the next generation of artificial intelligence collaborative algorithms and collaborative networks.
  • Federated learning also known as federated learning, can make data "available and invisible" under the premise of protecting user privacy and data security, that is, to complete the training task of machine learning models through multi-party cooperation. In addition, it can also provide machine learning models. inference service.
  • Federated learning can use multi-party data sources from multiple parties to train machine learning models, and improve the performance of machine learning models by using multi-party and complementary data sources, for example, to improve the accuracy of advertising recommendation models.
  • Federated learning does not require centralized data centers in one data center to jointly train machine learning models and deep learning models using multiple decentralized data sources.
  • Federated learning can not only meet the requirements of data protection laws and regulations, but also realize cross-department, cross-organization, and even cross-industry data cooperation. It provides new solutions for breaking "data silos", and begins to move from theoretical research to large-scale research. Commercial application landing.
  • federated learning is divided into horizontal federated learning (HFL), vertical federated learning (VFL) and federated transfer learning (FTL).
  • HFL horizontal federated learning
  • VFL vertical federated learning
  • FTL federated transfer learning
  • horizontal federated learning also known as sample-based federated learning
  • vertical federated learning also known as feature-based federated learning
  • federated transfer learning is suitable for the case where the dataset is not only different in the sample space but also in the feature space.
  • the data processing method provided by the embodiment of the present application is a horizontal federated learning method, and the horizontal federated learning will be described in detail below.
  • participant A and participant B share the same feature space and different sample space, that is, when two participants perform feature extraction on different sample data, they can use the same feature representation method to represent the features.
  • participant A obtains sample data A
  • participant B obtains sample data B. They perform feature extraction on the sample data respectively, and can obtain feature vectors in the form of (X1, X2, X3), the dimension of the feature vector and each Dimensions have the same meaning.
  • X1, X2, X3 feature vectors in the form of (X1, X2, X3), the dimension of the feature vector and each Dimensions have the same meaning.
  • the scenario where horizontal federated learning can be applied is that the datasets owned by the participants of federated learning have the same feature space and different sample spaces.
  • a scenario includes three parties, party 1 owns samples U1-U4, party 2 owns samples U5-U10, and party 3 owns samples U11-U15.
  • Each sample has the same feature space [F1, F2, ..., F L ], where L is an integer greater than 3.
  • L is an integer greater than 3.
  • this is only an example, and the embodiment of the present application does not limit the number of features in the feature space.
  • horizontal federated learning The core idea of horizontal federated learning is to let each participant train a model locally using its own sample data, and then obtain a better global model through security model fusion (such as security model parameter averaging).
  • security model fusion such as security model parameter averaging
  • each participant when the federated learning system performs data processing, each participant usually trains the business processing model according to the sample data.
  • the model parameters determined by the party are fused.
  • This kind of client-server architecture is prone to single-point communication congestion.
  • the central federated server is prone to overload and communication congestion, and thus cannot receive Model parameters sent by the participants, causing the system to crash.
  • this phenomenon is more serious.
  • the star topology there is only one communication link between a participant and the central federation server. It is easy to cause a participant to be disconnected due to a temporary interruption of the communication link, temporarily leaving the federated learning system and interrupting the federated learning process. . Therefore, the stability and robustness of the above data processing methods are poor.
  • FIG. 2 and FIG. 3 are schematic diagrams of an implementation environment of a data processing system provided by an embodiment of the present application.
  • the data processing method provided in the embodiment of the present application is applied to the data processing system 200, which may also be referred to as the federated learning system 200.
  • the system architecture of the data processing system 200 will be introduced below.
  • the data processing system includes at least three devices, the at least three devices include a first device and at least two second devices, and the first device is any one of the data processing systems; the devices in the data processing system use a ring Structural connection, in which, in a ring structure, each device is connected in series, and finally a closed loop is formed, and the information sent by the entire network is transmitted in this loop.
  • a device in the data processing system can be connected with two other devices to form a ring or multiple rings.
  • each device can be referred to as a participant.
  • Each participant can be directly or indirectly connected through wired or wireless communication, which is not limited in this embodiment of the present application.
  • the device may include at least one of a computing device, multiple computing devices, a cloud computing platform, or a virtualization center.
  • a certain device undertakes the main computing work in the federated learning, and other devices undertake the secondary computing work in the federated learning; or, a distributed computing architecture is used among the various devices to perform collaborative computing.
  • the data processing system 200 includes at least three devices 201, and the at least three devices 201 only take participant 1, participant 2, participant 3, . . . , participant K as examples in FIG. 2, wherein, K is an integer greater than 3.
  • K is an integer greater than 3.
  • Multiple participants adopt a Ring Architecture each participant has two communication links with other participants, and each participant only connects with its own neighbors, forming a ring topology.
  • each participant of the horizontal federated learning only needs to communicate with its immediate neighbors, and there are two communication links between each two participants, so that one of the communication links is temporarily interrupted, and it can also Data is transmitted through another communication link, which has better link fault tolerance.
  • the data processing system 200 includes at least three devices, and the data processing system includes at least two device groups. That is, the at least three devices can be divided into a plurality of device groups, each device group includes at least three devices, and each device group can adopt a ring structure, so that the data processing system forms a multi-layer ring structure.
  • Each device group includes a management device and a follower device, and the devices in each device group are connected in a ring structure, that is, the devices in each device group can form a ring, and each device is connected to at least two devices in the group; the The management devices of at least two device groups are connected in a ring structure. This grouping can be deployed when there are many devices in the data processing system, thereby improving the parallelism and orderliness of data processing by multiple groups of devices, and improving data processing efficiency.
  • participant 1 when the number of participants is large, the participants can be grouped to form such a multi-layered ring topology.
  • participant 1, participant 2, ..., participant 10 are used as examples for illustration.
  • Participant 1, Participant 2 and Participant 3 are the first device group
  • Participant 4, Participant 5 and Participant 6 are the second device group
  • Participant 7, Participant 8, Participant 9, Participant Square 10 is the third device group.
  • participant 3, participant 6, and participant 10 are management devices (leaders) in three device groups (each group of participants), and participant 1 and participant 2 are follower devices (followers) of participant 3.
  • the first layer has 3 rings composed of 3 groups
  • the second layer has a ring composed of leaders of 3 groups.
  • the computation of each participant in each ring can be performed in parallel, and the computation of each ring can also be performed in parallel.
  • the computation of each participant in each ring can be performed in parallel.
  • the calculation process in each loop can be as described in steps 1 to 4 above, especially the process of calculating the weighted average of model parameters in parallel and segmented as described in step 3 above.
  • FIG. 2 and FIG. 3 are only an example, and does not limit the number of devices included in the data processing system.
  • the federated learning system further includes a third-party collaborator server, which is used to provide a federated learning platform or a federated data analysis platform, and the collaborator server can be deployed in a public cloud or a private cloud.
  • Provides cloud services such as federated learning, federated data analysis services, and federated business intelligence services.
  • the collaborator server is configured with a functional module of the federal data analysis task, and the data processing method provided by the embodiment of the present application is implemented through the functional module, so as to improve the security and usability of the federal data analysis task.
  • the above-mentioned collaborator server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, and network services. , cloud communication, middleware service, domain name service, security service, content delivery network (CDN), and cloud server for basic cloud computing services such as big data and artificial intelligence platforms.
  • cloud communication middleware service, domain name service, security service, content delivery network (CDN), and cloud server for basic cloud computing services such as big data and artificial intelligence platforms.
  • CDN content delivery network
  • the device in the above data processing system may be a terminal, and the device type of the terminal includes, but is not limited to, at least one of a smart phone, a tablet computer, a laptop computer, or a desktop computer.
  • the device type of the terminal includes, but is not limited to, at least one of a smart phone, a tablet computer, a laptop computer, or a desktop computer.
  • a terminal including a smart phone is used as an example for illustration.
  • the number of the above-mentioned devices can be more or less.
  • the above-mentioned device is only one, or the above-mentioned device is dozens or hundreds, or more.
  • the embodiments of the present application do not limit the number and types of devices.
  • each device may include a message management and communication module, a participant management module, a process scheduling module, a model fusion module, an encryption and decryption module, and a model training module.
  • Data communication is carried out between multiple devices through the message management and communication module to transmit the model parameters determined by them.
  • the model training module can perform model training based on local data and user-defined models.
  • checkpoint model checkpoint
  • checkpoint is a database event
  • the checkpoint event is generated by the checkpoint process ( LGWR/CKPT process)
  • DBWR Data Base Writer, data writing process
  • dirty block refers to the block of dirty data
  • dirty data Dirty Read
  • dirty data Dirty Read
  • the device After the device obtains its own model parameters through training, it can be encrypted by the encryption and decryption module, and then the model parameters determined by itself can be fused with the model parameters determined by other devices through the model fusion module.
  • Model parameters are passed through the message management and communication modules.
  • the participants In the model parameter fusion, the participants can also be managed through the participant management module, and the model parameter fusion process can be scheduled through the process scheduling module. For example, determine which device initiates the weighted merging process of which model parameter and is responsible for the weighted merging of model parameters, and so on.
  • the method is executed by a first device in a data processing system, where the data processing system includes at least three devices, and the devices in the data processing system adopt a ring-shaped
  • the first device may be a terminal or a server.
  • the method includes the following steps.
  • the first device acquires first sample data of the target service.
  • the data processing method provided by the embodiment of the present application is used to train a business processing model based on sample data, and obtain target model parameters of the business processing model, and the first sample data is the sample data stored on the first device.
  • the data processing method is implemented by a data processing system, the data processing system includes a plurality of devices, the first device belongs to the data processing system, and the data processing method is completed by the cooperation of the plurality of devices. Therefore, the data processing method is a federation study method.
  • the plurality of devices may also be referred to as a plurality of participants.
  • the first device trains a first service processing model based on the first sample data, and obtains first model parameters of the first service processing model.
  • the first device obtains the first sample data, performs model training on the first service processing model locally on the first device, and obtains the first model parameters.
  • the first model parameters are obtained by local training of the first device, and may also be referred to as local model parameters of the first device, or local model parameters determined by the first device, or the like.
  • other devices that is, the second device
  • each device in the data processing system obtains model parameters based on its own sample data training.
  • the target model parameters of the business processing model can be obtained by fusing the model parameters determined by multiple devices in the data processing system.
  • the devices in the data processing system are connected in a ring structure, which can form one or more rings.
  • One device in the data processing system is connected to at least two devices. Connection, the device in each ring is connected to two devices in the ring, and each device can communicate with its neighbors, so that one device has at least two communication links with other devices, even if one of the communication links is temporarily interrupted, The device can also communicate with other devices through other communication links, so the data processing system will not interrupt the data processing flow due to the temporary interruption of a certain communication link, and has good stability and robustness.
  • the first device sends the first model parameter to a second device connected to the first device, the second device determines the first fusion parameter, and sends the first fusion parameter to other second devices, the first The fusion parameter is obtained by the second device based on the received first model parameter and the second model parameter determined by itself, and the second device is a device other than the first device in the data processing system.
  • the first device initiates and sends the first model parameters to the connected second device, and the second device sends the first model parameters to the connected second device.
  • the first device After receiving the first model parameter, it fuses the first model parameter with the second model parameter determined by itself, and then continues to send it to the next second device.
  • each device compares the received model parameter with the The model parameters determined by itself are fused and then transmitted, so that the amount of data transmitted between devices is small, which can effectively avoid the problems of overload and communication congestion, effectively improve the speed and efficiency of data processing, and ensure the stability of data processing.
  • the first device acquires second fusion parameters, where the second fusion parameters include model parameters determined by the at least three devices in the data processing system.
  • a device connected to the first device in the data processing system obtains second fusion parameters after merging the model parameters determined by itself with the received fusion model parameters, and the second fusion parameters include all the parameters in the data processing system. Device-determined model parameters. Therefore, the second fusion parameter can be transmitted back to the first device, and the first device further determines the target model parameters of the first service processing model.
  • the second fusion parameter includes model parameters determined by the at least three devices in the data processing system means that the second fusion parameter fuses the model parameters determined by the at least three devices in the data processing system .
  • the first device determines, according to the second fusion parameter, target model parameters of the first service processing model.
  • the devices in the data processing system are connected in a ring structure, and each device can communicate with neighboring devices.
  • a device has two communication links with other devices, even if one of the communication links is temporarily interrupted.
  • the device can also communicate with other devices through other communication links, so the data processing system will not interrupt the data processing flow due to the temporary interruption of a certain communication link, and has good stability and robustness.
  • the model parameters determined by each device are sequentially transmitted according to the above communication link, and during the transmission process, each device fuses the received model parameters with the model parameters determined by itself.
  • FIG. 5 is a flowchart of a data processing method provided by an embodiment of the present application.
  • the method is executed by a second device in a data processing system.
  • the data processing system includes at least three devices.
  • the devices in the data processing system use a ring
  • the second device may be a terminal or a server.
  • the method includes the following steps.
  • the second device acquires second sample data of the target service.
  • the second device trains a second service processing model based on the second sample data, and obtains second model parameters of the second service processing model.
  • the steps 501 and 502 are the same as the above-mentioned steps 401 and 402.
  • the second device may also obtain its own second sample data to perform model training on the second service processing model to obtain the second model parameters.
  • the second device receives the first model parameter determined by the first device.
  • the at least three devices include the first device.
  • This step 503 corresponds to the foregoing step 403, and the second device receives the first model parameters sent by the first device in the foregoing step 403.
  • the second device fuses the first model parameter with the second model parameter determined by itself to obtain the first fusion parameter.
  • the second device receives the first model parameters determined by the first device, and also trains the second model parameters locally. Therefore, the two can be merged and then sent to other devices instead of sending the two model parameters directly.
  • the amount of data transmitted between the second device and other devices can be reduced, the problems of overload and communication congestion can be effectively avoided, and the data processing speed and efficiency can be effectively improved.
  • the second device sends the first fusion parameter to other devices in the data processing system.
  • the second device when other devices receive the first fusion parameters sent by the second device, they can also obtain new fusion model parameters based on the first fusion parameters and the model parameters determined by themselves, and then continue to transmit them. After the second fusion parameters including the model parameters of all devices in the data processing system are subsequently obtained, the parameters can be transmitted back to the first device to determine the target model parameters of the first service processing model.
  • the second device determines target model parameters of the second service processing model.
  • the target model parameters can be sent to any device in the data processing system.
  • the second device may then receive target model parameters of the second service processing model.
  • the devices in the data processing system are connected in a ring structure, and each device can communicate with neighboring devices.
  • a device has two communication links with other devices, even if one of the communication links is temporarily interrupted.
  • the device can also communicate with other devices through other communication links, so the data processing system will not interrupt the data processing flow due to the temporary interruption of a certain communication link, and has good stability and robustness.
  • the model parameters determined by each device are sequentially transmitted according to the above communication link, and during the transmission process, each device fuses the received model parameters with the model parameters determined by itself.
  • FIG. 6 is a schematic diagram of a data processing system provided by an embodiment of the present application.
  • the data processing system includes at least three devices, and the at least three devices include a first device 601 and at least two second devices 602 .
  • the first device is any one of the data processing system; the second device is a device other than the first device in the data processing system.
  • the devices in the data processing system are connected in a ring structure, and one device is connected with the other two devices.
  • the first device is configured to execute the data processing method executed by the first device in each method embodiment.
  • the second device is configured to execute the data processing method executed by the second device in each method embodiment.
  • FIG. 4 the method embodiments shown in FIG. 4 , FIG. 5 , and FIG. 7 above.
  • FIG. 7 is a flow chart of a data processing method provided by an embodiment of the present application.
  • the method is applied to a data processing system.
  • the following is a method of completing data processing through the cooperation of multiple devices included in the data processing system.
  • the data processing method will be explained.
  • the data processing system includes four devices as an example for description, the device connected to the first device is referred to as the second device, and the second device other than the second device is referred to as the third device and the fourth device.
  • the data processing system includes four devices as an example for description.
  • the data processing system may also include more devices or less devices, and the method steps performed by the data processing system are the same as those performed by the devices in the following embodiments.
  • the method steps are the same. Referring to Figure 7, the method includes the following steps.
  • the first device acquires first sample data of the target service.
  • each device in the data processing system can acquire sample data of the target service to train a corresponding service processing model.
  • each device can obtain its own sample data, and the sample features obtained by processing the sample data are located in the same feature space.
  • the target service can be any service, for example, image classification, speech recognition, and the like.
  • the service processing model can be any model, for example, an image classification model, a speech recognition model, and the like.
  • the business processing model is a deep learning model, and the embodiment of the present application does not limit the model structure of the deep learning model.
  • the sample data may also be different.
  • the sample data is a sample image.
  • the sample data is sample speech.
  • the first sample data is stored in the first device, and the first device can retrieve the first sample data from local storage.
  • the first sample data may be stored in a database, and the first device may extract the first sample data from the database.
  • the first sample data refers to a plurality of pieces of sample data corresponding to the first device, or is referred to as a sample data set corresponding to the first device.
  • the first device trains a first service processing model based on the first sample data, and obtains first model parameters of the first service processing model.
  • the first service processing model is also a service processing model corresponding to the first device, or a service processing model deployed on the first device.
  • the first device After acquiring the first sample data, the first device can locally train the first service processing model to obtain first model parameters of the first service processing model.
  • the model parameters obtained by the model training of each device in the data processing system are called model parameters, so as to indicate that the model parameters are the results obtained from the training of a device model.
  • the parameters are synthesized to obtain final model parameters, which are referred to as target model parameters in the embodiments of the present application.
  • the model training process is that the first device performs feature extraction on the first sample data through the first business processing model to obtain the features of the first sample data, and then further performs the target on the features.
  • the prediction result of the first sample data is obtained, then the loss value is determined based on the target result in the first sample data and the prediction result, and the model parameters of the first business processing model are adjusted based on the loss value, Stop until the training end condition is met, where the target result is also the label carried by the first sample data.
  • the training end condition may be that the loss value converges, or the number of iterations reaches the target number of times, etc., which is not limited in this embodiment of the present application.
  • the first device may locally perform model training for one or several epochs on the first service processing model. Among them, 1 epoch is equivalent to training the model once using all the samples in the sample data. In layman's terms, the value of epoch is the number of times the entire sample data is used.
  • the first device can use any training method and training framework, such as PyTorch, TensorFlow, Keras, Horovod, or PyTorch-on-Angel, etc.
  • PyTorch is an open source Python machine learning library.
  • TensorFlow is a symbolic math system based on dataflow programming, which is widely used in the programming implementation of various machine learning algorithms.
  • Tensor refers to tensors and Flow refers to streams.
  • Keras is an open source artificial neural network library written in Python.
  • Horovod is a deep learning tool open sourced by Uber.
  • PyTorch-on-Angel is the PyTorch project running on the Angel platform.
  • Angel is a computing platform developed in Java and Scala, a high-performance distributed computing framework for machine learning. It should be noted that the first device, the second device, the third device and the fourth device can select different training methods and training frameworks, that is, each device in the data processing system can select its own training method and training framework. The specific selection of the training framework can be determined by the relevant technical personnel according to the requirements or the hardware and software conditions of the device, which is not limited in this embodiment of the present application.
  • the second device acquires second sample data of the target service.
  • the second device trains the second service processing model based on the second sample data, and obtains second model parameters of the second service processing model.
  • Steps 703 and 704 are the same as the above steps 701 and 702.
  • the second device can also perform model training based on its own second sample data to obtain second model parameters.
  • the second service processing model is also the second device.
  • the corresponding business processing model, the second business processing model is a business processing model deployed on the second device, and the second business processing model can be either a business processing model with the same structure as the first business processing model, or a business processing model with the same structure as the first business processing model.
  • the first service processing model is a service processing model with different structures, which is not limited in this embodiment of the present application.
  • the third device acquires third sample data of the target service.
  • the third device trains the third service processing model based on the third sample data, and obtains third model parameters of the third service processing model.
  • Steps 705 and 706 are the same as the above-mentioned steps 701 and 702.
  • the third device can also perform model training based on its own third sample data to obtain third model parameters.
  • the third service processing model is also the third device.
  • the corresponding business processing model, the third business processing model is a business processing model deployed on a third device, and the third business processing model can be either a business processing model with the same structure as the first business processing model, or a business processing model with the same structure as the first business processing model.
  • the first service processing model is a service processing model with different structures, which is not limited in this embodiment of the present application.
  • the fourth device acquires four sample data of the target service.
  • the fourth device trains the fourth service processing model based on the fourth sample data, and obtains fourth model parameters of the fourth service processing model.
  • Steps 707 and 708 are the same as the above-mentioned steps 701 and 702.
  • the fourth device can also perform model training based on its own fourth sample data to obtain fourth model parameters.
  • the fourth service processing model is also the fourth device.
  • the corresponding business processing model, the fourth business processing model is a business processing model deployed on the fourth device, and the fourth business processing model can be either a business processing model with the same structure as the first business processing model, or a business processing model with the same structure as the first business processing model.
  • the first service processing model is a service processing model with different structures, which is not limited in this embodiment of the present application.
  • step sequence numbers do not limit the execution order of steps 701 and 702 , steps 703 and 704 , steps 705 and 706 , and steps 707 and 708 .
  • the process of model training based on sample data by each device in the data processing system is independent of each other and can be executed in parallel. Compared with collecting all sample data for training by one device, the steps of model training are divided into various devices and processed in parallel by each device, which can greatly improve the speed and efficiency of model training, and can also reduce the burden on the device during model training. , to avoid the situation that the equipment is overloaded and the system crashes.
  • the first device sends the first model parameter to a second device connected to the first device.
  • the first device is any device in the data processing system.
  • the first device is an initiator of model parameter fusion and a device that processes the fusion model parameters to obtain target model parameters as an example for description.
  • other devices may also be selected as the initiators of model parameter fusion, and other devices may also be selected as devices that process the fusion model parameters to obtain target model parameters, which are not limited in this embodiment of the present application.
  • the connection manner of the four devices may be as shown in FIG. 8 .
  • the data processing system includes a first device 801 , a second device 802 , a third device 803 and a fourth device 804 .
  • a ring structure is adopted between the four devices, the first device 801 is connected to the second device 802 and the fourth device 804 respectively, the second device 802 is connected to the third device 803 and the first device 801 respectively, the third device 803 is respectively connected to The second device 802 is connected to the fourth device 804, and the fourth device 804 is connected to the third device 803 and the first device 801, respectively.
  • the four devices may also be referred to as four participants, for example, the four devices may be referred to as participant 1, participant 2, participant 3, and participant 4, respectively.
  • the first device After the first device determines the first model parameters, it can send the first model parameters to other devices through a communication connection.
  • a second device and a fourth device are connected to the first device.
  • only sending the first model parameters to the second device is taken as an example for description.
  • the first device may also The first model parameter is sent to the fourth device, and the second fusion parameter determined by the second device to the third device is subsequently transmitted back to the first device.
  • the data when data is transmitted between various devices in the data processing system, the data is encrypted before being sent.
  • each device can use a method based on secret sharing to protect the model parameters obtained by local training.
  • the method of secret sharing can be implemented by adding a random mask.
  • the first device may obtain a random mask, fuse the random mask with the first model parameters to obtain encrypted model parameters, and then send the encrypted model to a second device connected to the first device parameter.
  • the model parameters determined by each device may be floating-point numbers.
  • each device may convert the determined model parameters from floating-point numbers to integers, and then fetch the rounded integers. The remainder is sent to other devices, so that the security of data transmission can be improved, the amount of transmitted data can be effectively reduced, and the data processing efficiency can be improved.
  • the above random mask acquisition process is as follows: the first device acquires a candidate random mask, and then rounds the candidate random mask to obtain the random mask.
  • the first device may further perform the above-mentioned rounding on the first model parameters, and then fuse the random mask and the rounded first model parameters.
  • the first device rounds the first model parameter, sums the random mask and the rounded first model parameter, and takes the remainder of the summation result to obtain the encrypted model parameter .
  • the acquisition process of the random mask can be obtained by accomplish. in, is a candidate random mask, INT(x) means rounding x.
  • P is a very large integer, which is used to convert the candidate random mask into an integer.
  • the selection of the integer P determines the degree of precision of floating-point numbers. The larger P is, the higher the precision of floating-point numbers is preserved. The value is not limited.
  • r 1 is a random mask.
  • a weight may be assigned to the first model parameter, so that each device corresponds to a weight, and subsequently based on the weight of each device, the data can be
  • the model parameters determined by multiple devices in the processing system are weighted and combined to obtain encrypted model parameters.
  • the first device may fuse the random mask with the rounded first model parameters through (r 1 +d 1 w 1 )modQ.
  • d 1 is the weight of the first device.
  • Q is a very large prime number
  • mod is the modulo function.
  • 2d 1 w 1 ⁇ Q where d 1 is an integer.
  • d 1 is a floating point number
  • the same method can be used to convert d 1 to an integer as well.
  • the encrypted model parameters can be sent to the second device, and the second device fuses the second model parameters determined by itself on this basis.
  • the encryption model parameter is the remainder obtained by the above-mentioned remainder processing.
  • the second device receives the first model parameter determined by the first device.
  • the second device performs fusion based on the first model parameter and the second model parameter determined by itself to obtain the first fusion parameter.
  • the second device may fuse it with the second model parameter determined by itself.
  • the second device determines the first fusion parameter, and sends the first fusion parameter to other second devices, and the first fusion parameter Obtained for the second device based on the received first model parameter and the second model parameter determined by itself.
  • the data transmitted between subsequent devices all contain the random mask, which effectively ensures the security of data transmission between all devices in the data processing system.
  • the fusion process is the same as the fusion process in step 709 above.
  • the second device performs a sum based on the received encryption model parameters and the second model parameters determined by itself and rounded up, and takes the remainder of the summation result to obtain a first fusion parameter, and the first The fusion parameters are sent to other second devices.
  • the second device can sum the received (r 1 +d 1 w 1 )modQ with d 2 w 2 , and then take the remainder to obtain (r 1 +d 1 w 1 +d 2 w 2 )modQ.
  • (r 1 +d 1 w 1 +d 2 w 2 )modQ is the first fusion parameter.
  • d 2 is the weight of the second device
  • w 2 is the second model parameter determined by the second device.
  • the second device sends the first fusion parameter to the third device.
  • the third device receives the first fusion parameter sent by the second device.
  • the third device performs fusion based on the first fusion parameter and the third model parameter determined by itself to obtain a third fusion parameter.
  • the steps 713 and 714 are the same as the above-mentioned steps 710 and 711, and are not repeated here.
  • the third device may perform the same fusion step to obtain the third fusion parameter (r 1 +d 1 w 1 +d 2 w 2 +d 3 w 3 )modQ.
  • the third device sends the third fusion parameter to the fourth device.
  • the fourth device receives the third fusion parameter sent by the third device.
  • the fourth device performs fusion based on the third fusion parameter and the fourth model parameter determined by itself to obtain the second fusion parameter.
  • Steps 716 and 717 are the same as the above-mentioned steps 710 and 711, and are not repeated here.
  • the fourth device may perform the same fusion step to obtain the second fusion parameter (r 1 +d 1 w 1 +d 2 w 2 +d 3 w 3 +d 1 w 4 )modQ.
  • the fourth device has fused the model parameters of all devices in the data processing system, and therefore, the second fusion parameter obtained by fusion can be returned to the first device.
  • the fourth device sends the second fusion parameter to the first device.
  • the first device receives the second fusion parameter.
  • Step 719 is a process for the first device to acquire the second fusion parameter.
  • the fourth device fuses may not be the second fusion parameter, but a candidate fusion parameter, which includes a random mask, and the first device also needs to use the random mask.
  • the second fusion parameter is obtained.
  • the second fusion parameter includes the model parameters determined by the at least three devices in the data processing system, that is, including the model parameters determined by the first device, the second device, the third device, and the fourth device, in some cases
  • the second fusion parameter combines the model parameters determined by the first device, the second device, the third device, and the fourth device.
  • the first device receives candidate fusion parameters including the random mask and model parameters determined by the at least three devices in the data processing system, and then the first device removes the candidate fusion parameters from the candidate fusion parameters
  • the random mask obtains the second fusion parameter.
  • the first device may compute (r 1 +d 1 w 1 +d 2 w 2 +d 3 w 3 +d 4 w 4 -r 1 )modQ.
  • r 1 is the random mask.
  • the first device determines target model parameters of the first service processing model according to the second fusion parameter.
  • the first device may process it to obtain target model parameters of the first service processing model.
  • the second fusion parameter is a fusion result. If the target model parameter is to be obtained, it needs to be converted into the model parameter determined by a single device.
  • the first device may obtain a quotient of the second fusion parameter and a total weight, and use the quotient as a target model parameter of the first business processing model, where the total weight is the at least three weights in the data processing system The sum of the weights of the devices.
  • each model parameter is multiplied by P, so it needs to be removed when averaging, so that the weighted combined result of the model parameters of the four devices can be obtained.
  • the weight of any device may be the quantity of sample data, or may be other parameters, such as data quality indicators, contribution degree indicators, reward indicators, etc., which are not limited in this embodiment of the present application.
  • the target model parameters are obtained by performing a weighted average of the model parameters above.
  • the first device computes (w 1 +w 2 +w 3 +w 4 )/4 as a model parameter.
  • the above-mentioned model parameters and target model parameters may include N segments; where N is an integer greater than 1.
  • each device is responsible for the calculation of a segment of a target model parameter, and then multiple devices in the data processing system can calculate the segment of the target model parameter in parallel, which can further divide the calculation steps into multiple devices, and parallel processing can Improve data processing speed and efficiency.
  • the participants of the horizontal federated learning can divide the model parameters into N segments, and elect N participants to calculate the weighted average of the N model segments separately, which can be calculated in parallel. That is, the number of segments can be the same as the number of devices, or it can be different from the number of devices.
  • 1 ⁇ N ⁇ K, K is the number of horizontal federated learning participants.
  • the number of segments can be less than the number of devices. Some devices in the data processing system perform segment calculation, and some devices do not need to perform segment calculation.
  • the number of model parameter segments N>K can be greater than the number of devices, so that each device can be responsible for the calculation of multiple segments, and then calculate in multiple times, each time L is calculated Weighted average of model parameter segments, L ⁇ K.
  • the above step 709 may be: the first device sends the first segment in the first model parameter to the second device connected to the first device, and then, in step 710, the second The device receives the first segment in the first model parameter. Then from steps 711 to 712, the second device obtains the first fused segment based on the first segment in the first model parameter and the second model parameter determined by itself, and sends the first fused segment to the third equipment. The steps performed by the third device and the fourth device are the same. After the fourth device determines the second fusion segment, it can send the second fusion segment to the first device, and then the first device obtains the second fusion parameter, where the second fusion segment includes the at least one in the data processing system.
  • the first segment of three device-determined model parameters may determine the target first segment in the target model parameters of the service processing model according to the second fusion segment, and obtain the target model of the service processing model based on the target first segment parameter.
  • the target second segment when acquiring the target model parameters, in addition to the target first segment, the target second segment also needs to be determined.
  • the target second segment is determined by a device other than the first device in the data processing system, so that a plurality of devices can determine the model segment in parallel.
  • the first device may also receive the second segment or the first fused second segment in the model parameters sent by the second device, and combine the received second segment or the first fused second segment with the first segment.
  • the second segment in the first model parameter determined by a device is fused to obtain a second fused second segment, and the second fused second segment obtained by the first device is sent to other second devices.
  • the step of fusing multiple segments may be performed by the first device, or may be performed by other devices in the data processing system. Therefore, based on the target first segment, the step of acquiring the target model parameters of the business processing model may include the following two ways.
  • Mode 1 The first device receives the target second segment in the target model parameter, the target second segment is the segment in the target model parameter except the target first segment, and the first device receives the target second segment.
  • the target first segment and the target second segment in the model parameters are spliced to obtain the target model parameters of the first service processing model.
  • the first device is responsible for obtaining target model parameters by fusing all target segments.
  • Mode 2 The first device sends the target first segment, the first device receives the target model parameters of the first service processing model, and the target model parameters are determined by the second device for the target first segment and the target second segment. After splicing, the second segment of the target is a segment other than the first segment of the target in the parameters of the target model. In the second mode, other devices other than the first device are responsible for the fusion of all target segments to obtain target model parameters, and then synchronize the target model parameters to the first device.
  • participant 2 is responsible for calculating the weighted average process of model parameter segment 2
  • participant 3 is responsible for calculating the weighted average of model parameter segment 3.
  • Participant 4 is responsible for calculating the weighted average process of model parameter segment 4. Only the weighted average process of Participant 1 is described here, and the same is true for other participants. This process can be divided into the following five steps.
  • Step 1 The encryption and decryption module of Participant 1 generates a random mask 1, denoted as For other parties, its own random mask is generated, for example, party 2 generates random mask 2, which is r 2,2 .
  • the model fusion module of Participant 1 calculates the sum of its own parameter segment 1 (that is, the first segment of the first model parameter of Participant 1) and the random mask, that is, it calculates (r 1,1 +d 1 w 1,1 ) modQ and send the result to party 2.
  • (r 1,1 +d 1 w 1,1 ) modQ is the encrypted segment 1 obtained by the participant 1.
  • w 1,1 refers to the parameter segment 1 of participant 1
  • d 1 can be selected as the number of training samples owned by participant 1 or other parameters (such as data quality indicators, contribution indicators, reward indicators, etc.)
  • Q is a very large prime, such as 2d 1 w 1,1 ⁇ Q.
  • Step 2 Participant 2 sums the encrypted segment 1 received from participant 1 and its own parameter segment 1, that is, calculates (r 1,1 +d 1 w 1,1 +d 2 w 2, 1 ) modQ and send the result to party 3.
  • (r 1,1 +d 1 w 1,1 +d 2 w 2,1 ) modQ is the encrypted segment 1 obtained by the participant 2.
  • d 2 can be selected as the number of training samples owned by participant 2 or other parameters.
  • Step 3 Participant 3 sums the encrypted segment 1 received from participant 2 and its own parameter segment 1, that is, calculates (r 1,1 +d 1 w 1,1 +d 2 w 2, 1 +d 3 w 3,1 )modQ and send the result to party 4.
  • (r 1,1 +d 1 w 1,1 +d 2 w 2,1 +d 3 w 3,1 ) modQ is the encrypted segment 1 obtained by the participant 3.
  • d3 can be selected as the number of training samples owned by participant 3 or other parameters.
  • Step 4 Participant 4 sums the encrypted segment 1 received from participant 3 and its own parameter segment 1, that is, calculates (r 1,1 +d 1 w 1,1 +d 2 w 2, 1 +d 3 w 3,1 +d 1 w 4,1 )modQ and send the result to Party 1.
  • (r 1,1 +d 1 w 1,1 +d 2 w 2,1 +d 3 w 3,1 +d 1 w 4,1 ) modQ is the encrypted segment 1 obtained by the participant 4.
  • d 4 can be selected as the number of training samples or other parameters owned by the participant 4 .
  • Step 5 Participant 1 subtracts the random mask 1 generated by itself from the encrypted segment 1 received from Participant 4, that is, calculates (r 1,1 +d 1 w 1,1 +d 2 w 2,1 +d 3 w 3,1 +d 4 w 4,1 -r 1,1 )modQ, the weighted sum of parameter segment 1 can be obtained, and d 1 w 1,1 +d 2 w 2,1 + d 3 w 3,1 +d 4 w 4,1 .
  • participant 1 may calculate a weighted average of model parameter segment 1 (d 1 w 1,1 +d 2 w 2,1 +d 3 w 3,1 +d 4 w 4,1 )/(Pd).
  • the average segment 1 It may be called the average segment 1, and may also be called the parameter segment 1 in the following, but the parameter segment 1 obtained at this time is the weighted average result of the parameter segment 1 of the four devices.
  • d (d 1 +d 2 +d 3 +d 4 ) may be the total number of training samples of all participants or the sum of other parameters.
  • the addition and subtraction operations are operations in the integer domain, which need to include a modulo operation, that is, the model parameters and the random mask are rounded, and the division operation in the final step 5 is an operation of dividing an integer by an integer, The result is a floating point number. It should be noted that, as can be seen from the above description, since each participant is responsible for calculating the weighted average of one model parameter segment, the participants can execute the weighted average step of the model parameter segment in parallel.
  • the first device is a management device in the target device group; the second device is a follower device in the target device group.
  • participant 3 is the management device, and each participant in the first device group They are respectively responsible for the weighted merging of parameter segment 1 to parameter segment 3, and participant 3 is responsible for splicing the parameter segments obtained by each participant in the first group of participants to obtain the model parameter score obtained by the first group of participants.
  • segment 1 to parameter segment 3 i.e.
  • participant 6 is responsible for merging parameter segment 4 to parameter segment 6 weighted and combined by each participant in the second equipment group, that is,
  • participant 10 is responsible for merging the parameter segment 7 to the parameter segment 10 weighted and combined by each participant in the third device group, that is, Further, participants 3, 6, and 10 can then combine the obtained parameters by weighting and combine them in sections to obtain in,
  • the devices in the data processing system are connected in a ring structure, and each device can communicate with neighboring devices.
  • a device has two communication links with other devices, even if one of the communication links is temporarily interrupted.
  • the device can also communicate with other devices through other communication links, so the data processing system will not interrupt the data processing flow due to the temporary interruption of a certain communication link, and has good stability and robustness.
  • the model parameters determined by each device are sequentially transmitted according to the above communication link, and during the transmission process, each device fuses the received model parameters with the model parameters determined by itself.
  • the data processing system in the embodiments shown in FIG. 4 , FIG. 5 , and FIG. 7 may be a blockchain system, and the devices in the data processing system may be node devices in the blockchain system.
  • Blockchain is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm.
  • Blockchain essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block.
  • the blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.
  • the system involved in the embodiments of the present application may be a distributed system formed by connecting a client and multiple nodes (any form of computing device in an access network, such as a server and a user terminal) through network communication.
  • the distributed system may be a blockchain system.
  • FIG. 10 is a schematic structural diagram of the distributed system 100 provided in the embodiment of the present application applied to the blockchain system.
  • a peer-to-peer (P2P, Peer To Peer) network is formed between nodes, and the P2P protocol is a protocol that runs on the Transmission Control Protocol (Transmission Control Protocol, TCP) protocol. application layer protocol.
  • TCP Transmission Control Protocol
  • application layer protocol In a distributed system, any machine such as a server and a terminal can join to become a node, and a node includes a hardware layer, a middle layer, an operating system layer and an application layer.
  • the involved functions include the following.
  • Routing a basic function that a node has to support communication between nodes.
  • a node can also have the following functions.
  • Application used to deploy in the blockchain, implement specific business according to actual business needs, record data related to the realization of functions to form record data, carry a digital signature in the record data to indicate the source of the task data, and send the record data To other nodes in the blockchain system, for other nodes to add the record data to the temporary block when verifying the source and integrity of the record data successfully.
  • Blockchain including a series of blocks (Blocks) that follow each other in chronological order. Once a new block is added to the blockchain, it will not be removed. The block records the blockchain system. The record data submitted by the middle node.
  • FIG. 11 is a schematic diagram of a block structure (Block Structure) provided by an embodiment of the present application, and each block includes a hash value (the hash value of the The hash value of a block, each block is connected by the hash value to form a blockchain.
  • the block may also include information such as a timestamp when the block was generated.
  • Blockchain (Blockchain) is essentially a decentralized database, which is a series of data blocks associated with cryptographic methods. Each data block contains relevant information to verify the validity of its information. (anti-counterfeiting) and generate the next block.
  • the above-mentioned first device, second device, third device and fourth device may all be node devices in the blockchain system, and multiple node devices are connected in a ring structure, and each node device can obtain The sample data of the target business is used to train the business processing model based on the sample data, and the model parameters of the business processing model are obtained. After obtaining the model parameters, the node device can send the model parameters determined by itself to the next connected node device, and the next node device can fuse the received model parameters with the model parameters determined by itself, and then send it to the next connected node device. Node devices until the last node device returns the first node device that started sending. Based on the fused model parameters, the node device can determine the final target model parameters.
  • the node device After the node device determines the final target model parameters, it can generate a block based on the target model parameters, broadcast the block to other node devices in the blockchain system, and the blockchain system will reach a consensus on the block , when the consensus is passed, each node device can add the block to the blockchain, so that each node device synchronizes the target model parameters.
  • each node device after each node device obtains the target segment it is responsible for, it can be spliced by a node device to obtain the target model parameters, which are then uploaded to the chain by generating blocks. Each node device can also generate blocks and put them on the chain based on the target segment they are responsible for.
  • the node device can also generate a block based on the model parameters determined by itself after obtaining the model parameters determined by itself, and upload the block to the chain through consensus.
  • the finalized target model parameters or intermediate data can be uploaded to the chain, which can take advantage of the fact that the data on the chain in the blockchain system cannot be tampered with to ensure data security, accuracy and integrity.
  • the chain consensus mechanism can avoid the calculation error of a single node device and ensure the accuracy of the data.
  • FIG. 12 is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application. Referring to FIG. 12 , the apparatus includes the following functional modules:
  • the obtaining module 1001 is used for obtaining the first sample data of the target service.
  • the training module 1002 is configured to train a first service processing model based on the first sample data to obtain first model parameters of the first service processing model.
  • the sending module 1003 is configured to send the first model parameter to a second device connected to the first device, the second device determines the first fusion parameter, and sends the first fusion parameter to other second devices, the first A fusion parameter is obtained by the second device based on the received first model parameter and the second model parameter determined by itself, the second device is a device other than the first device in the data processing system, the data The equipment in the processing system is connected in a ring structure.
  • the obtaining module 1001 is further configured to obtain second fusion parameters, where the second fusion parameters include model parameters determined by the at least three devices in the data processing system.
  • the determining module 1004 is configured to determine the target model parameter of the first service processing model according to the second fusion parameter.
  • the first model parameter includes N segments. Wherein, the N is an integer greater than 1.
  • the sending module 1003 is configured to send the first segment in the first model parameter to the second device connected to the first device, the second device determines the first fusion segment, and the first fusion segment It is sent to other second devices, where the first fusion segment is obtained by the second device based on the received first segment and the first segment in the second model parameter determined by itself.
  • the obtaining module 1001 is configured to obtain the second fusion parameter based on the second fusion segment, where the second fusion segment includes the first segment in the model parameters determined by the at least three devices in the data processing system.
  • the determining module 1004 is used to:
  • the target first segment in the target model parameters of the first service processing model is determined.
  • target model parameters of the first service processing model are acquired.
  • the obtaining module 1001 is configured to perform any of the following:
  • a target second segment in the target model parameter is received, where the target second segment is a segment other than the target first segment in the target model parameter.
  • the target first segment and the target second segment are spliced to obtain target model parameters of the first service processing model.
  • a target model parameter of the first service processing model is determined, and the target model parameter is obtained by splicing the target first segment and the target second segment by the second device.
  • the obtaining module 1001 is further configured to receive the second segment or the first fused second segment in the model parameters sent by the second device.
  • the device also includes:
  • a fusion module configured to fuse the second segment or the first fused second segment with the second segment in the model parameter determined by the first device to obtain a second fused second segment.
  • the sending module 1003 is further configured to send the second merged second segment to other second devices.
  • the data processing system includes at least two device groups, each device group includes a management device and a follower device, and the devices in each device group are connected in a ring structure.
  • the management devices of the at least two device groups are connected in a ring structure.
  • the first device is a management device in the target device group.
  • the second device is a follower device in the target device group, and the target device group is any one of the at least two device groups.
  • the sending module 1003 is configured to send the first model parameter to a follower device in the target device group, the follower device determines the first fusion parameter, and sends the first fusion parameter to other follower devices,
  • the first fusion parameter is obtained by the following device based on the received first model parameter and the second model parameter determined by itself.
  • the acquisition module 1001 is used to:
  • third fusion parameters include model parameters determined by the target device group.
  • the management device in the other device group determines the fourth fusion parameter, and send the fourth fusion parameter to the follower device in the other device group, the fourth fusion parameter
  • the fusion parameter is obtained by the management device in the other device group based on the received third fusion parameter and the fusion parameter determined by the device in its own group.
  • the second fusion parameter includes fusion parameters determined by at least two device groups in the data processing system.
  • the sending module 1003 is used to:
  • the random mask and the first model parameters are fused to obtain encrypted model parameters.
  • the acquisition module 1001 is used to:
  • Receive candidate fusion parameters where the candidate fusion parameters include the random mask and model parameters determined by the at least three devices in the data processing system.
  • the random mask is removed from the candidate fusion parameters to obtain the second fusion parameter.
  • the fusion module is used to:
  • This first model parameter is rounded.
  • the random mask and the rounded first model parameter are summed.
  • the second device determines the first fusion parameter and sends the first fusion parameter to other second devices, including:
  • the second device performs a summation based on the received encryption model parameters and the second model parameters determined by itself and rounded up, the first fusion parameter is obtained by taking the remainder of the summation result, and the first fusion parameter is sent to other second devices.
  • the fusion module is used to:
  • This first model parameter is rounded.
  • the random mask and the first product are summed.
  • the second device performs summation based on the received encryption model parameters and the second product, takes the remainder of the summation result, obtains the first fusion parameter, and sends the first fusion parameter to other second devices.
  • the square product is the product of the second model parameter determined and rounded by the second device itself and the weight of the second device.
  • the sending module 1003 is configured to: obtain a candidate random mask.
  • the candidate random mask is rounded to obtain the random mask.
  • the determining module 1004 is configured to obtain the quotient of the second fusion model parameter and the total weight, and use the quotient as the target model parameter of the first business processing model, and the total weight is the data processing system The sum of the weights of the at least three devices in .
  • the devices in the data processing system are connected in a ring structure, and each device can communicate with neighboring devices.
  • a device has two communication links with other devices, even if one of the communication links is temporarily interrupted.
  • the device can also communicate with other devices through other communication links, so the data processing system will not interrupt the data processing flow due to the temporary interruption of a certain communication link, and has good stability and robustness.
  • the model parameters determined by each device are sequentially transmitted according to the above communication link, and during the transmission process, each device fuses the received model parameters with the model parameters determined by itself.
  • the data processing apparatus provided by the above-mentioned embodiments only uses the division of the above-mentioned functional modules as an example for data processing. In practical applications, the above-mentioned functions can be allocated to different functional modules according to needs. The internal structure of the data processing apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the data processing apparatus and the data processing method embodiments provided by the above embodiments belong to the same concept, and the specific implementation process thereof is detailed in the method embodiments, which will not be repeated here.
  • FIG. 13 is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application. Referring to FIG. 13 , the apparatus includes:
  • the obtaining module 1101 is configured to obtain second sample data of the target service.
  • the training module 1102 is configured to train the second service processing model based on the second sample data to obtain second model parameters of the second service processing model.
  • the receiving module 1103 is configured to receive a first model parameter determined by a first device, where the at least three devices include the first device.
  • the fusion module 1104 is configured to fuse the first model parameter with the second model parameter determined by itself to obtain the first fusion parameter.
  • the sending module 1105 is configured to send the first fusion parameter to other devices in the data processing system, and the devices in the data processing system are connected in a ring structure.
  • the receiving module 1103 is further configured to determine target model parameters of the second service processing model.
  • the first model parameter includes N segments. Wherein, the N is an integer greater than 1.
  • the receiving module 1103 is used for:
  • a first segment in the first model parameter determined by the first device is received.
  • the receiving module 1103 is further configured to fuse the first segment determined by the first device with the first segment in the second model parameter determined by itself to obtain the first fused segment.
  • the first fused segment is sent to other devices in the data processing system.
  • a second fused segment is obtained, and the second fused segment includes the second segment in the model parameters determined by the at least three devices in the data processing system.
  • a target second segment in the target model parameters of the second service processing model is determined.
  • target model parameters of the second service processing model are acquired.
  • the devices in the data processing system are connected in a ring structure, and each device can communicate with neighboring devices.
  • a device has two communication links with other devices, even if one of the communication links is temporarily interrupted.
  • the device can also communicate with other devices through other communication links, so the data processing system will not interrupt the data processing flow due to the temporary interruption of a certain communication link, and has good stability and robustness.
  • the model parameters determined by each device are sequentially transmitted according to the above communication link, and during the transmission process, each device fuses the received model parameters with the model parameters determined by itself.
  • the electronic device 1200 may vary greatly due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1201 and one or more memories 1202, wherein at least one computer program is stored in the memory 1202, and the at least one computer program is loaded and executed by the processor 1201 to implement the data processing methods provided by the above method embodiments.
  • the electronic device can also include other components for realizing the functions of the device.
  • the electronic device can also have components such as a wired or wireless network interface, an input and output interface, etc., so as to perform input and output. This embodiment of the present application will not be repeated here.
  • FIG. 15 is a structural block diagram of a terminal provided by an embodiment of the present application.
  • the terminal 1300 may be a portable mobile terminal, such as a smart phone, a tablet computer, a notebook computer or a desktop computer.
  • Terminal 1300 may also be called user equipment, portable terminal, laptop terminal, desktop terminal, and the like by other names.
  • the terminal 1300 includes: a processor 1301 and a memory 1302 .
  • the processor 1301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like.
  • the processor 1301 can use at least one hardware form among DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array, programmable logic array) accomplish.
  • the processor 1301 may further include an AI (Artificial Intelligence, artificial intelligence) processor, where the AI processor is used to process computing operations related to machine learning.
  • AI Artificial Intelligence, artificial intelligence
  • Memory 1302 may include one or more computer-readable storage media, which may be non-transitory. Memory 1302 may also include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices, flash storage devices. In some embodiments, a non-transitory computer-readable storage medium in the memory 1302 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 1301 to implement the data processing provided by the method embodiments in this application. method.
  • the terminal 1300 may optionally further include: a peripheral device interface 1303 and at least one peripheral device.
  • the processor 1301, the memory 1302 and the peripheral device interface 1303 can be connected through a bus or a signal line.
  • Each peripheral device can be connected to the peripheral device interface 1303 through a bus, a signal line or a circuit board.
  • the peripheral device includes at least one of: a radio frequency circuit 1304 , a display screen 1305 , and a power supply 1306 .
  • the peripheral device interface 1303 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 1301 and the memory 1302 .
  • the radio frequency circuit 1304 is used for receiving and transmitting RF (Radio Frequency, radio frequency) signals, also called electromagnetic signals.
  • the radio frequency circuit 1304 communicates with communication networks and other communication devices via electromagnetic signals.
  • the radio frequency circuit 1304 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals.
  • the radio frequency circuit 1304 may communicate with other terminals through at least one wireless communication protocol.
  • the wireless communication protocol includes but is not limited to: World Wide Web, Metropolitan Area Network, Intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), wireless local area network and/or WiFi (Wireless Fidelity, Wireless Fidelity) network.
  • the radio frequency circuit 1304 may further include a circuit related to NFC (Near Field Communication, short-range wireless communication), which is not limited in this application.
  • NFC Near Field Communication, short-range wireless communication
  • the display screen 1305 is used to display UI (User Interface, user interface).
  • the UI can include graphics, text, icons, video, and any combination thereof.
  • the display screen 1305 also has the ability to acquire touch signals on or above the surface of the display screen 1305 .
  • the touch signal may be input to the processor 1301 as a control signal for processing.
  • Power supply 1306 is used to power various components in terminal 1300 .
  • the power source 1306 may be alternating current, direct current, disposable batteries, or rechargeable batteries.
  • the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. Wired rechargeable batteries are batteries that are charged through wired lines, and wireless rechargeable batteries are batteries that are charged through wireless coils.
  • the rechargeable battery can also be used to support fast charging technology.
  • terminal 1300 also includes one or more sensors 1307 .
  • the one or more sensors 1307 include, but are not limited to, a pressure sensor 1308 and an optical sensor 1309 .
  • the pressure sensor 1308 may be disposed on the side frame of the terminal 1300 and/or the lower layer of the display screen 1305 .
  • the processor 1301 can perform left and right hand identification or shortcut operations according to the holding signal collected by the pressure sensor 1308.
  • the processor 1301 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 1305.
  • the operability controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.
  • Optical sensor 1309 is used to collect ambient light intensity.
  • the processor 1301 can control the display brightness of the display screen 1305 according to the ambient light intensity collected by the optical sensor 1309 . In some embodiments, when the ambient light intensity is high, the display brightness of the display screen 1305 is increased; when the ambient light intensity is low, the display brightness of the display screen 1305 is decreased.
  • FIG. 15 does not constitute a limitation on the terminal 1300, and may include more or less components than the one shown, or combine some components, or adopt different component arrangements.
  • FIG. 16 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • the server 1400 may vary greatly due to different configurations or performance, and may include one or more processors (Central Processing Units, CPU) 1401 and one or more memories 1402, wherein at least one computer program is stored in the memory 1402, and the at least one computer program is loaded and executed by the processor 1401 to implement the data processing methods provided by the above method embodiments.
  • the server can also have components such as wired or wireless network interfaces and input and output interfaces for input and output, and the server can also include other components for implementing device functions, which are not described here.
  • a computer-readable storage medium such as a memory including at least one computer program, the at least one computer program being executable by a processor to complete the data processing method in the above embodiment.
  • the computer-readable storage medium can be a read-only memory (Read-Only Memory, referred to as: ROM), a random access memory (Random Access Memory, referred to as: RAM), a read-only optical disk (Compact Disc Read-Only Memory, referred to as: CD-ROM), magnetic tapes, floppy disks, and optical data storage devices, etc.
  • a computer program product or computer program comprising one or more pieces of program code stored in a computer-readable storage medium .
  • One or more processors of the electronic device can read the one or more program codes from the computer-readable storage medium, and the one or more processors execute the one or more program codes, so that the electronic device can execute the above data Approach.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Medical Informatics (AREA)
  • Evolutionary Biology (AREA)
  • Information Transfer Between Computers (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种数据处理方法、装置、设备及存储介质，由数据处理***中的第一设备执行，方法包括：获取目标业务的第一样本数据(401)；基于第一样本数据对第一业务处理模型进行训练，得到第一业务处理模型的第一模型参数(402)；向与第一设备连接的第二设备发送第一模型参数，由第二设备确定第一融合参数，将第一融合参数发送给其他第二设备，第一融合参数为第二设备基于接收到的第一模型参数与自身确定的第二模型参数得到的，第二设备为数据处理***中除第一设备之外的设备(403)；获取第二融合参数，第二融合参数包括数据处理***中至少三个设备确定的模型参数(404)；根据第二融合参数，确定第一业务处理模型的目标模型参数(405)。

Description

数据处理方法、装置、设备及存储介质
本申请要求于2020年12月2日提交的申请号为202011385627.5、发明名称为“数据处理方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能技术领域，特别涉及一种数据处理方法、装置、设备及存储介质。
背景技术
随着计算机技术的发展和人工智能(Artificial Intelligence，AI)技术的进步，联邦学习(Federated Learning)逐渐成为AI领域的一个热门课题，联邦学习能够通过多方协作完成机器学习模型的训练任务。
目前，多方协作的联邦学习采用客服-服务器架构，在联邦学习***包括中心联邦服务器，该联邦学习***中的其它客服设备(在此称参与方)均与该中心联邦服务器连接以通信，形成星形拓扑。
发明内容
本申请实施例提供了一种数据处理方法、装置、设备及存储介质，技术方案如下。
一方面，提供了一种数据处理方法，由数据处理***中的第一设备执行，所述数据处理***包括至少三个设备，所述数据处理***中的设备采用环形结构连接，所述方法包括：
获取目标业务的第一样本数据；
基于所述第一样本数据对第一业务处理模型进行训练，得到所述第一业务处理模型的第一模型参数；
向与所述第一设备连接的第二设备发送所述第一模型参数，由所述第二设备确定第一融合参数，将所述第一融合参数发送给其他第二设备，所述第一融合参数为所述第二设备基于接收到的所述第一模型参数与自身确定的第二模型参数得到的，所述第二设备为所述数据处理***中除所述第一设备之外的设备；
获取第二融合参数，所述第二融合参数包括所述数据处理***中所述至少三个设备确定的模型参数；
根据所述第二融合参数，确定所述第一业务处理模型的目标模型参数。
一方面，提供了一种数据处理方法，由数据处理***中的第二设备执行，所述数据处理***包括至少三个设备，所述数据处理***中的设备采用环形结构连接，所述方法包括：
获取目标业务的第二样本数据；
基于所述第二样本数据对第二业务处理模型进行训练，得到所述第二业务处理模型的第二模型参数；
接收第一设备确定的第一模型参数，所述至少三个设备包括所述第一设备；
将所述第一模型参数与自身确定的第二模型参数进行融合，得到第一融合参数；
将所述第一融合参数发送给所述数据处理***中的其他设备；
确定所述第二业务处理模型的目标模型参数。
一方面，提供了一种数据处理***，所述数据处理***包括至少三个设备，所述至少三个设备包括第一设备和至少两个第二设备，所述第一设备为所述数据处理***中的任一个；所述数据处理***中的设备采用环形结构连接；
第一设备用于获取目标业务的第一样本数据；基于所述第一样本数据对第一业务处理模型进行训练，得到所述第一业务处理模型的第一模型参数；向与所述第一设备连接的第二设备发送所述第一模型参数，由所述第二设备确定第一融合参数，将所述第一融合参数发送给其他第二设备，所述第一融合参数为所述第二设备基于接收到的所述第一模型参数与自身确定的第二模型参数得到的，所述第二设备为所述数据处理***中除所述第一设备之外的设备；获取第二融合模型参数，所述第二融合模型参数包括所述数据处理***中所述至少三个设备确定的模型参数；根据所述第二融合模型参数，确定所述第一业务处理模型的目标模型参数；
任一第二设备用于获取所述目标业务的第二样本数据；基于所述第二样本数据对第二业务处理模型进行训练，得到所述第二业务处理模型的第二模型参数；接收所述第一设备确定的第一模型参数；将所述第一模型参数与自身确定的第二模型参数进行融合，得到第一融合参数；将所述第一融合参数发送给所述数据处理***中的其他设备；确定所述第二业务处理模型的目标模型参数。
一方面，提供了一种数据处理装置，所述装置包括：
获取模块，用于获取目标业务的第一样本数据；
训练模块，用于基于所述第一样本数据对第一业务处理模型进行训练，得到所述第一业务处理模型的第一模型参数；
发送模块，用于向与所述第一设备连接的第二设备发送所述第一模型参数，由所述第二设备确定第一融合参数，将所述第一融合参数发送给其他第二设备，所述第一融合参数为所述第二设备基于接收到的所述第一模型参数与自身确定的第二模型参数得到的，所述第二设备为所述数据处理***中除所述第一设备之外的设备；
所述获取模块，还用于获取第二融合参数，所述第二融合参数包括所述数据处理***中所述至少三个设备确定的模型参数；
确定模块，用于根据所述第二融合参数，确定所述第一业务处理模型的目标模型参数。
一方面，提供了一种数据处理装置，所述装置包括：
获取模块，用于获取目标业务的第二样本数据；
训练模块，用于基于所述第二样本数据对第二业务处理模型进行训练，得到所述第二业务处理模型的第二模型参数；
接收模块，用于接收第一设备确定的第一模型参数，所述至少三个设备包括所述第一设备；
融合模块，用于将所述第一模型参数与自身确定的第二模型参数进行融合，得到第一融合参数；
发送模块，用于将所述第一融合参数发送给所述数据处理***中的其他设备，所述数据处理***中的设备采用环形结构连接；
所述接收模块还用于确定所述第二业务处理模型的目标模型参数。
一方面，提供了一种电子设备，所述电子设备包括一个或多个处理器和一个或多个存储器，所述一个或多个存储器中存储有至少一条计算机程序，所述至少一条计算机程序由所述一个或多个处理器加载并执行以实现上述数据处理方法的各种可选实现方式。
一方面，提供了一种计算机可读存储介质，所述存储介质中存储有至少一条计算机程序，所述至少一条计算机程序由处理器加载并执行以实现上述数据处理方法的各种可选实现方式。
一方面，提供了一种计算机程序产品或计算机程序，所述计算机程序产品或所述计算机程序包括一条或多条计算机程序，所述一条或多条计算机程序存储在计算机可读存储介质中。电子设备的一个或多个处理器能够从计算机可读存储介质中读取所述一条或多条计算机程序，所述一个或多个处理器执行所述一条或多条计算机程序，使得电子设备能够执行上述任一种可能实施方式的数据处理方法。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还能够根据这些附图获得其他的附图。
图1是本申请实施例提供的一种样本空间和特征空间的示意图；
图2是本申请实施例提供的一种数据处理***的流程图；
图3是本申请实施例提供的一种数据处理***的流程图；
图4是本申请实施例提供的一种数据处理方法的流程图；
图5是本申请实施例提供的一种数据处理方法的流程图；
图6是本申请实施例提供的一种数据处理***的示意图；
图7是本申请实施例提供的一种数据处理方法的流程图；
图8是本申请实施例提供的一种数据处理方法的流程图；
图9是本申请实施例提供的一种数据处理方法的流程图；
图10是本申请实施例提供的分布式***100应用于区块链***的结构示意图；
图11是本申请实施例提供的区块结构(Block Structure)的一个示意图；
图12是本申请实施例提供的一种数据处理装置的结构示意图；
图13是本申请实施例提供的一种数据处理装置的结构示意图；
图14是本申请实施例提供的一种电子设备的结构示意图；
图15是本申请实施例提供的一种终端的结构框图；
图16是本申请实施例提供的一种服务器的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
本申请中术语“第一”“第二”“第三”等字样用于对作用和功能基本相同的相同项或相似项进行区分，应理解，“第一”、“第二”、“第n”之间不具有逻辑或时序上的依赖关系，也不对数量和执行顺序进行限定。还应理解，尽管以下描述使用术语第一、第二等来描述各种元素，但这些元素不应受术语的限制。这些术语只是用于将一元素与另一元素区别分开。例如，第一设备、第二设备、第三设备等，仅用于对设备进行区分。又例如，第一样本数据、第二样本数据和第三样本数据等仅用于对样本数据进行区分。在不脱离各种示例的范围的情况下，第一设备能够被称为第二设备，并且类似地，第二设备能够被称为第一设备。第一设备和第二设备都是设备，并且在某些情况下，能够是单独且不同的设备。
本申请中术语“至少一个”的含义是指一个或多个，本申请中术语“多个”的含义是指两个或两个以上，例如，多个设备是指两个或两个以上的设备。“至少两个”的含义是指两个或两个以上。“至少三个”的含义是指三个或三个以上。
应理解，在本文中对各种示例的描述中所使用的术语只是为了描述特定示例，而并非旨在进行限制。如在对各种示例的描述和所附权利要求书中所使用的那样，单数形式“一个(‘a’/‘an’)”和“该”旨在也包括复数形式，除非上下文另外明确地指示。
还应理解，本文中所使用的术语“和/或”是指并且涵盖相关联的所列出的项目中的一个或多个项目的任何和全部可能的组合。术语“和/或”，是一种描述关联对象的关联关系，表示能 够存在三种关系，例如，A和/或B，能够表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本申请中的字符“/”，一般表示前后关联对象是一种“或”的关系。
还应理解，在本申请的各个实施例中，各个过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
还应理解，根据A确定B并不意味着仅仅根据A确定B，还能够根据A和/或其它信息确定B。
还应理解，术语“包括”(也称“includes”、“including”、“comprises”和/或“comprising”)当在本说明书中使用时指定存在所陈述的特征、整数、步骤、操作、元素、和/或部件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元素、部件、和/或其分组。
还应理解，术语“如果”可被解释为意指“当...时”(“when”或“upon”)或“响应于确定”或“响应于检测到”。类似地，根据上下文，短语“如果确定...”或“如果检测到[所陈述的条件或事件]”可被解释为意指“在确定...时”或“响应于确定...”或“在检测到[所陈述的条件或事件]时”或“响应于检测到[所陈述的条件或事件]”。
本申请实施例提供的数据处理方法应用到了人工智能技术。下面对人工智能进行简单介绍。
人工智能(Artificial Intelligence，AI)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用***。
联邦学习逐渐成为一种新兴的人工智能基础技术，联邦学习的设计目标是在保障大数据交换时的信息安全、保护终端数据和个人数据隐私、保证合法合规的前提下，在多参与方或多计算节点之间开展高效率的机器学习。可选地，联邦学习可使用的机器学习算法并不局限于神经网络，还包括随机森林等重要算法，联邦学习有望成为下一代人工智能协同算法和协作网络的基础。
出于用户隐私保护、数据使用合规、商业利益竞争等考虑，数据合作面临很多困难，使得理想中的大数据变成了“数据孤岛”，阻碍数据价值创造和人工智能的发展。为破解数据合作困境，联邦学习(Federated Learning)应运而生。下面对联邦学习进行解释说明。
联邦学习又称为联合学习，能够在保护用户隐私和数据安全的前提下实现数据的“可用而不可见”，也即通过多方协作完成机器学习模型的训练任务，此外，还能够提供机器学习模型的推理服务。
联邦学习可以利用来自于多个参与方的多方数据源训练机器学习模型，通过使用多方、互补的数据源来提升机器学习模型的性能，例如，提升广告推荐模型的准确率。
与传统的机器学习方法不同，联邦学习不要求将分散的数据集中到一个数据中心就能够利用多个分散的数据源联合训练机器学习模型和深度学习模型。联邦学习既能满足数据保护法律和法规的要求，又能实现跨部门、跨组织、甚至跨行业的数据合作，打破“数据孤岛”提供了新的解决思路，并开始从理论研究迈向大规模商业应用落地。
根据数据在不同参与方之间的分布情况，联邦学习分为横向联邦学习(Horizontal Federated Learning，HFL)、纵向联邦学习(Vertical Federated Learning，VFL)和联邦迁移学习(Federated Transfer Learning，FTL)。
其中，横向联邦学习又称为基于样本的联邦学习，适用于数据集共享相同特征空间但样本空间不同的情况；纵向联邦学习又称为基于特征的联邦学习，适用于数据集共享相同样本空间但特征空间不同的情况；联邦迁移学习则适用于数据集不仅在样本空间上不同而且在特征空间上也不同的情况。
本申请实施例提供的数据处理方法为一种横向联邦学习方法，下面对横向联邦学习进行详细介绍。
以两个参与方进行横向联邦学习为例，将两个参与方分别称为参与方A和参与方B，两方联合训练一个或多个机器学习模型。其中，参与方A和参与方B共享相同的特征空间和不同的样本空间，也即两个参与方对不同的样本数据进行特征提取时，能够采用同样的特征表征方式对特征进行表征。例如，参与方A获取样本数据A，参与方B获取样本数据B，他们分别对样本数据进行特征提取，均能够得到形式为(X1，X2，X3)的特征向量，特征向量的维度以及每个维度的含义也相同。这样对于庞大的样本数据，能够将其分给多个参与方进行训练，最终将多个参与方的训练情况综合，得到基于庞大的样本数据训练得到的模型参数。
参见图1，横向联邦学习可以应用的场景是联邦学习的参与方拥有的数据集有相同的特征空间和不同的样本空间。例如，场景包括三个参与方，参与方1拥有样本U1-U4，参与方2拥有样本U5-U10，参与方3拥有样本U11-U15。每个样本均具有相同的特征空间[F1、F2、…、F L]，其中，L为大于3的整数。当然，在此仅为一个示例，本申请实施例对该特征空间中特征数量不作限定。
横向联邦学***均)得到一个更好的全局模型。通过安全设计横向联邦学习的模型训练过程，可以保证横向联邦学习的任何参与方都不能获得其他参与方的训练数据，并且联邦训练的模型的性能近似于将所有数据集中在一起训练出来的模型的性能。
相关技术中，联邦学习***进行数据处理时，通常是由各个参与方根据样本数据对业务处理模型进行训练，得到模型参数后，将模型参数均发送至中心联邦服务器，由中心联邦服务器对各个参与方确定的模型参数进行融合。
这种客服-服务器架构(即星形拓扑)很容易出现单点通信拥塞，例如，当参与方都向中心联邦服务器发送他们模型参数时，中心联邦服务器很容易出现过载和通信拥塞，进而无法接收参与方发送的模型参数，导致***崩溃。尤其是业务处理模型规模很大时，这种现象更严重。且星形拓扑里，一个参与方与中心联邦服务器之间只有一条通信链路，很容易出现因为暂时的通信链路中断而导致一个参与方掉线，暂时离开联邦学习***，使得联邦学习流程中断。因而，上述数据处理方法的稳定性和鲁棒性很差。
图2和图3是本申请实施例提供的一种数据处理***的实施环境示意图。本申请实施例提供的数据处理方法应用于数据处理***200，也可以称之为联邦学习***200，以下对数据处理***200的***架构进行介绍。
该数据处理***包括至少三个设备，该至少三个设备包括第一设备和至少两个第二设备，该第一设备为该数据处理***中的任一个；该数据处理***中的设备采用环形结构连接，其中，在环形结构中，各设备串接，最后形成一个闭环，整个网络发送的信息就是在这个环中传递。该数据处理***中的设备可以与另外两个设备连接，构成一个环或多个环。
该数据处理***中的设备均参与到联邦学习过程，因而，可以将每个设备称为参与方。各个参与方之间能够通过有线或无线通信方式进行直接或间接地连接，本申请实施例在此不做限制。
对于每个设备，该设备可以包括一台计算设备、多台计算设备、云计算平台或者虚拟化中心中的至少一项。可选地，某一个设备在联邦学习中承担主要计算工作，其他设备在联邦学习中承担次要计算工作；或者，各个设备之间采用分布式计算架构进行协同计算。
参见图2，该数据处理***200包括至少三个设备201，该至少三个设备201在图2中仅以参与方1、参与方2、参与方3、…、参与方K为例，其中，K为大于3的整数。多个参与方采用环形结构(Ring Architecture)，每个参与方与其他参与方均有两条通信链路，每个参与方只与自己的邻居连接，组成一个环状拓扑。在环状拓扑里，横向联邦学习的每个参与方只需要与其直接的邻居进行通信，且每两个参与方之间都有两条通信链路，这样其中一条通 信链路暂时中断，也能够通过另一条通信链路传输数据，具有较好的链路故障容错功能。
参见图3，该数据处理***200包括至少三个设备，该数据处理***包括至少两个设备组。也即是，该至少三个设备能够被分为多个设备组，每个设备组包括至少三个设备，每个设备组可以采用环形结构，进而该数据处理***形成多层的环形结构。各设备组包括管理设备和跟随设备，各设备组中的设备采用环形结构连接，也即是，各设备组中的设备能够构成一个环，每个设备与组内的至少两个设备连接；该至少两个设备组的管理设备采用环形结构连接。该分组的情况可以在数据处理***中设备较多时部署，提高多组设备对数据处理的并行性和有序性，提高数据处理效率。
例如，当参与方个数较多时，可以对参与方进行分组，组成这样多层的环状拓扑结构，在图3中仅以参与方1、参与方2、…、参与方10为例进行说明，参与方1、参与方2和参与方3为第一个设备组，参与方4、参与方5和参与方6为第二个设备组，参与方7、参与方8、参与方9、参与方10为第三个设备组。其中，参与方3、参与方6、参与方10为三个设备组(每组参与方)中的管理设备(leader)，参与方1和参与方2为参与方3的跟随设备(follower)。如图3所示例，有10个横向联邦学***均的流程。
需要说明的是，上述图2和图3中设备数量仅为一种示例，并不对该数据处理***包括的设备数量造成限定。
在一个示例性场景中，联邦学***台或联邦数据分析平台，该协作者服务器可以以公有云或私有云的部署方式对外提供联邦学习、联邦数据分析服务、联邦商业智能服务等云服务。例如，该协作者服务器配置有联邦数据分析任务的一个功能模块，通过该功能模块实现本申请实施例提供的数据处理方法，以提升联邦数据分析任务的安全性和可用性。
可选地，上述协作者服务器可以是独立的物理服务器，或者是多个物理服务器构成的服务器集群或者分布式***，或者是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(Content Delivery Network，CDN)、以及大数据和人工智能平台等基础云计算服务的云服务器。
可选地，上述数据处理***中的设备可以为终端，该终端的设备类型包括但不限于：智能手机、平板电脑、膝上型便携计算机或者台式计算机中的至少一种。以下实施例，以终端包括智能手机来进行举例说明。
本领域技术人员可以知晓，上述设备的数量能够更多或更少。比如上述设备仅为一个，或者上述设备为几十个或几百个，或者更多数量。本申请实施例对设备的数量和设备类型不加以限定。
如图2所示，每个设备可以包括消息管理和通信模块、参与方管理模块、流程调度模块、模型融合模块、加解密模块、模型训练模块。多个设备之间通过消息管理和通信模块进行数据通信，以传输各自确定的模型参数。在每个设备中，模型训练模块能够根据本地数据以及用户定义模型后，进行模型训练，模型训练后，能够执行模型checkpoint(检查点)，其中，checkpoint为一个数据库事件，checkpoint事件由checkpoint进程(LGWR/CKPT进程)发出，当checkpoint事件发生时，DBWR(Data Base Writer，数据写进程)会将脏块写入到磁盘中，同时数据文件和控制文件的文件头也会被更新以记录checkpoint信息。其中，脏块是指脏数据的块，脏数据(Dirty Read)是指源***中的数据不在给定的范围内或对于实际业务毫无意义，或是数据格式非法，以及在源***中存在不规范的编码和含糊的业务逻辑。
该设备在训练得到自身的模型参数后，可以由加解密模块进行加密后，通过模型融合模 块将自己确定的模型参数与其他设备确定的模型参数进行融合，需要说明的是，该其他设备确定的模型参数通过消息管理和通信模块传递。在模型参数融合中，还可以通过参与方管理模块对参与方进行管理，通过流程调度模块对模型参数融合流程进行调度。例如，确定由哪个设备开始发起哪个模型参数的加权合并流程和负责模型参数的加权合并等。
图4是本申请实施例提供的一种数据处理方法的流程图，该方法由数据处理***中的第一设备执行，该数据处理***包括至少三个设备，该数据处理***中的设备采用环形结构连接，该第一设备可以为终端或服务器，在此以该第一设备为终端为例，该方法包括以下步骤。
401、第一设备获取目标业务的第一样本数据。
本申请实施例提供的数据处理方法用于基于样本数据对业务处理模型进行训练，得到业务处理模型的目标模型参数，第一样本数据为第一设备上存储的样本数据。该数据处理方法由数据处理***实现，该数据处理***包括多个设备，第一设备属于该数据处理***，该数据处理方法由该多个设备协作完成，因而，该数据处理方法为一种联邦学习方法。在一些实施例中，也可以称该多个设备为多个参与方。
402、第一设备基于该第一样本数据对第一业务处理模型进行训练，得到该第一业务处理模型的第一模型参数。
在本实施例中，第一设备获取第一样本数据，在第一设备本地对第一业务处理模型进行模型训练，得到该第一模型参数。在一些实施例中，该第一模型参数在第一设备本地训练得到，也可以称之为第一设备的本地模型参数，或者第一设备确定的本地模型参数等。
同理的，该数据处理***中的其他设备(也即是第二设备)也可以获取第二样本数据，并在其本地训练模型，确定出第二模型参数。这样该数据处理***中每个设备基于自己的样本数据训练得到了模型参数。将数据处理***中的多个设备确定出的模型参数融合，即可得到该业务处理模型的目标模型参数。
在上述图2和图3所示的数据处理***中已经示出，数据处理***中的设备采用环形结构连接，可以构成一个或多个环，该数据处理***中的一个设备与至少两个设备连接，每个环中的设备与环内的两个设备连接，每个设备与邻居设备通信即可，这样一个设备与其他设备至少有两条通信链路，即使其中一条通信链路暂时中断，该设备也可以通过其他通信链路与其他设备进行通信，因而，该数据处理***不会因某条通信链路的暂时中断而中断数据处理流程，具有很好的稳定性和鲁棒性。
403、第一设备向与该第一设备连接的第二设备发送该第一模型参数，由该第二设备确定第一融合参数，将该第一融合参数发送给其他第二设备，该第一融合参数为该第二设备基于接收到的该第一模型参数与自身确定的第二模型参数得到的，该第二设备为该数据处理***中除该第一设备之外的设备。
该第一设备和第二设备确定出模型参数后，并非将模型参数均发送给同一个设备进行融合，而是由第一设备发起，向连接的第二设备发送第一模型参数，第二设备接收到第一模型参数后，将第一模型参数与自身确定的第二模型参数融合后，再继续发送给下一个第二设备，在传递过程中，每个设备均对接收到的模型参数与自身确定的模型参数融合后再传递，这样设备之间传递的数据量较小，能够有效避免过载和通信拥塞的问题，能够有效提高数据处理速度和效率，保证数据处理的稳定性。
404、第一设备获取第二融合参数，该第二融合参数包括该数据处理***中该至少三个设备确定的模型参数。
在该数据处理***中与第一设备连接的一个设备，在将自身确定的模型参数与接收到的融合模型参数融合后，得到了第二融合参数，该第二融合参数包括数据处理***中所有设备确定的模型参数。因而可以将该第二融合参数再传回第一设备，由第一设备进一步确定第一业务处理模型的目标模型参数。在一些实施例中，该第二融合参数包括该数据处理***中该至少三个设备确定的模型参数是指，该第二融合参数融合了该数据处理***中该至少三个设 备确定的模型参数。
405、第一设备根据该第二融合参数，确定该第一业务处理模型的目标模型参数。
本申请实施例，一方面，数据处理***中的设备采用环形结构连接，每个设备与邻居设备通信即可，这样一个设备与其他设备有两条通信链路，即使其中一条通信链路暂时中断，该设备也可以通过其他通信链路与其他设备进行通信，因而，该数据处理***不会因某条通信链路的暂时中断而中断数据处理流程，具有很好的稳定性和鲁棒性。另一方面，该数据处理***在进行数据处理时，每个设备确定的模型参数按照上述通信链路依次传递，且传递过程中，每个设备对接收到的模型参数与自身确定的模型参数融合后再传递，这样设备之间传递的数据量较小，且无需向一个设备集中发送模型参数，因而，能够有效避免过载和通信拥塞的问题，能够有效提高数据处理速度和效率，保证数据处理的稳定性。
图5是本申请实施例提供的一种数据处理方法的流程图，该方法由数据处理***中的第二设备执行，该数据处理***包括至少三个设备，该数据处理***中的设备采用环形结构连接，该第二设备可以为终端或服务器，在此以该第二设备为终端为例，该方法包括以下步骤。
501、第二设备获取目标业务的第二样本数据。
502、第二设备基于该第二样本数据对第二业务处理模型进行训练，得到该第二业务处理模型的第二模型参数。
该步骤501和步骤502与上述步骤401和步骤402同理，第二设备也可以获取自身的第二样本数据对第二业务处理模型进行模型训练，得到第二模型参数。
503、第二设备接收第一设备确定的第一模型参数。
其中，该至少三个设备包括该第一设备。
该步骤503与上述步骤403对应，第二设备接收上述步骤403中第一设备发送的第一模型参数。
504、第二设备将该第一模型参数与自身确定的第二模型参数进行融合，得到第一融合参数。
第二设备接收到第一设备确定的第一模型参数，本地也训练得到了第二模型参数，因而，可以将二者融合后再发送给其他设备，而不是将两个模型参数直接发送，这样能够减少第二设备与其他设备之间传输的数据量，能够有效避免过载和通信拥塞的问题，能够有效提高数据处理速度和效率。
505、第二设备将该第一融合参数发送给该数据处理***中的其他设备。
同理，其他设备接收到第二设备发送的第一融合参数，也可以基于该第一融合参数与自身确定的模型参数，得到新的融合模型参数再继续传递。后续得到包括数据处理***中所有设备的模型参数的第二融合参数后，可以传回第一设备确定第一业务处理模型的目标模型参数。
506、第二设备确定该第二业务处理模型的目标模型参数。
该目标模型参数由其他设备(比如第一设备)确定后，可以将目标模型参数发送给数据处理***中的任意设备。该第二设备则可以接收该第二业务处理模型的目标模型参数。
本申请实施例，一方面，数据处理***中的设备采用环形结构连接，每个设备与邻居设备通信即可，这样一个设备与其他设备有两条通信链路，即使其中一条通信链路暂时中断，该设备也可以通过其他通信链路与其他设备进行通信，因而，该数据处理***不会因某条通信链路的暂时中断而中断数据处理流程，具有很好的稳定性和鲁棒性。另一方面，该数据处理***在进行数据处理时，每个设备确定的模型参数按照上述通信链路依次传递，且传递过程中，每个设备对接收到的模型参数与自身确定的模型参数融合后再传递，这样设备之间传递的数据量较小，且无需向一个设备集中发送模型参数，因而，能够有效避免过载和通信拥塞的问题，能够有效提高数据处理速度和效率，保证数据处理的稳定性。
图6是本申请实施例提供的一种数据处理***的示意图，参见图6，该数据处理***包 括至少三个设备，该至少三个设备包括第一设备601和至少两个第二设备602，该第一设备为该数据处理***中的任一个；第二设备为该数据处理***中除该第一设备之外的设备。该数据处理***中的设备采用环形结构连接，一个设备与另外两个设备连接。该第一设备用于执行各个方法实施例中第一设备所执行的数据处理方法。第二设备用于执行各个方法实施例中第二设备所执行的数据处理方法。具体可以参见上述图4、图5以及图7所示方法实施例。
图7是本申请实施例提供的一种数据处理方法的流程图，参见图7，该方法应用于数据处理***，下面通过该数据处理***包括的多个设备协作完成数据处理的方式，对该数据处理方法进行说明。在下述实施例中，以数据处理***包括四个设备为例进行说明，将与第一设备连接的设备称为第二设备，将该第二设备之外的其他第二设备称为第三设备和第四设备。在此仅以该数据处理***包括四个设备为例进行说明，该数据处理***还可以包括更多设备或者更少的设备，其所执行的方法步骤均与下述实施例中的设备所执行的方法步骤同理。参见图7，该方法包括以下步骤。
701、第一设备获取目标业务的第一样本数据。
在本申请实施例中，该数据处理***中的每个设备能够获取该目标业务的样本数据以对对应的业务处理模型进行训练。在横向联邦学习中，每个设备能够获取到自己的样本数据，对样本数据进行处理得到的样本特征位于同一特征空间内。
该目标业务可以为任一种业务，例如，图像分类、语音识别等。相应地，该业务处理模型可以为任一种模型，例如，图像分类模型、语音识别模型等。在一些实施例中，该业务处理模型为深度学习模型，本申请实施例对于深度学习模型的模型结构不作限定。
对于不同的目标业务，样本数据也可能不同。例如，在图像分类业务中，样本数据为样本图像。在语音识别业务中，样本数据为样本语音。
在一些实施例中，该第一样本数据存储于该第一设备中，第一设备可以从本地存储中提取该第一样本数据。在另一些实施例中，该第一样本数据可以存储于数据库中，第一设备可以从数据库中提取该第一样本数据。其中，该第一样本数据是指第一设备对应的多条样本数据，或者称为第一设备对应的样本数据集。
702、第一设备基于该第一样本数据对第一业务处理模型进行训练，得到该第一业务处理模型的第一模型参数。
其中，第一业务处理模型也即是第一设备对应的业务处理模型，或者说是部署在第一设备上的业务处理模型。
第一设备获取到第一样本数据后，能够在本地对第一业务处理模型进行训练，得到第一业务处理模型的第一模型参数。在此将数据处理***中每个设备进行模型训练得到的模型参数称为模型参数，以此来表示该模型参数为一个设备模型训练得到的结果，最终我们需要将数据处理***中设备确定的模型参数综合得到最终的模型参数，在本申请实施例中将该最终的模型参数称为目标模型参数。
在一些实施例中，该模型训练的过程也即是由第一设备通过第一业务处理模型对第一样本数据进行特征提取，得到第一样本数据的特征，然后进一步对特征进行该目标业务对应的处理，得到第一样本数据的预测结果，然后基于第一样本数据中的目标结果和该预测结果，确定损失值，基于损失值对第一业务处理模型的模型参数进行调整，直至符合训练结束条件时停止，其中，目标结果也即是第一样本数据携带的标签。
对于该训练结束条件可以为损失值收敛，或者迭代次数达到目标次数等，本申请实施例对此不作限定。
其中，第一设备可以在本地对第一业务处理模型，进行一个或者几个epoch的模型训练。其中，1个epoch等于使用样本数据中的全部样本训练一次模型。通俗的讲，epoch的取值就是整个样本数据被使用的次数。
该第一设备可以使用任意训练方法和训练框架，例如PyTorch、TensorFlow、Keras、Horovod 或PyTorch-on-Angel等。其中，PyTorch是一个开源的Python机器学***台上运行的PyTorch项目。Angel是一种计算平台，使用Java和Scala语言开发，面向机器学习的高性能分布式计算框架。需要说明的是，该第一设备、第二设备、第三设备以及第四设备可以选择不同的训练方法和训练框架，也即是，该数据处理***中每个设备可以选择自己的训练方法和训练框架，具体如何选择，可以由相关技术人员根据需求或者设备的软硬件条件确定，本申请实施例对此不作限定。
703、第二设备获取该目标业务的第二样本数据。
704、第二设备基于该第二样本数据对第二业务处理模型进行训练，得到该第二业务处理模型的第二模型参数。
该步骤703和步骤704与上述步骤701和步骤702同理，第二设备也可以基于自身的第二样本数据进行模型训练，得到第二模型参数，该第二业务处理模型也即是第二设备对应的业务处理模型，该第二业务处理模型是部署在第二设备上的业务处理模型，该第二业务处理模型既可以与第一业务处理模型为相同结构的业务处理模型，也可以为与第一业务处理模型不同结构的业务处理模型，本申请实施例对此不做限定。
705、第三设备获取该目标业务的第三样本数据。
706、第三设备基于该第三样本数据对第三业务处理模型进行训练，得到该第三业务处理模型的第三模型参数。
该步骤705和步骤706与上述步骤701和步骤702同理，第三设备也可以基于自身的第三样本数据进行模型训练，得到第三模型参数，该第三业务处理模型也即是第三设备对应的业务处理模型，该第三业务处理模型是部署在第三设备上的业务处理模型，该第三业务处理模型既可以与第一业务处理模型为相同结构的业务处理模型，也可以为与第一业务处理模型不同结构的业务处理模型，本申请实施例对此不做限定。
707、第四设备获取目标业务的四样本数据。
708、第四设备基于该第四样本数据对第四业务处理模型进行训练，得到该第四业务处理模型的第四模型参数。
该步骤707和步骤708与上述步骤701和步骤702同理，第四设备也可以基于自身的第四样本数据进行模型训练，得到第四模型参数，该第四业务处理模型也即是第四设备对应的业务处理模型，该第四业务处理模型是部署在第四设备上的业务处理模型，该第四业务处理模型既可以与第一业务处理模型为相同结构的业务处理模型，也可以为与第一业务处理模型不同结构的业务处理模型，本申请实施例对此不做限定。
需要说明的是，上述步骤序号并不对步骤701、702，步骤703、704，步骤705、706，以及步骤707、708的执行顺序造成限定。数据处理***中的每个设备基于样本数据进行模型训练的过程彼此独立，能够并行执行。相较于由一个设备收集所有的样本数据进行训练，将模型训练的步骤划分到各个设备上，由各个设备并行处理，能够大大提高模型训练速度和效率，也能够减少设备在模型训练时的负担，避免设备出现高负荷导致***崩溃的情况。
709、第一设备向与该第一设备连接的第二设备发送该第一模型参数。
数据处理***中每个设备训练得到模型参数后，需要对多个设备确定的模型参数进行融合。该第一设备为数据处理***中的任一设备，在本实施例中，以该第一设备为模型参数融合的发起方以及对融合模型参数进行处理得到目标模型参数的设备为例进行说明，在一些实施例中，也可以选择其他设备作为模型参数融合的发起方，也可以选择其他设备作为对融合模型参数进行处理得到目标模型参数的设备，本申请实施例对此不作限定。
对于本申请实施例提供的四个设备，该四个设备的连接方式可以如图8所示。参见图8， 该数据处理***包括第一设备801、第二设备802、第三设备803和第四设备804。该四个设备之间采用环形结构，第一设备801分别与第二设备802和第四设备804连接，第二设备802分别与第三设备803和第一设备801连接，第三设备803分别与第二设备802和第四设备804连接，第四设备804分别与第三设备803和第一设备801连接。该四个设备也可以称为四个参与方，例如，可以分别称该四个设备为参与方1、参与方2、参与方3和参与方4。
第一设备确定了第一模型参数后，能够通过通信连接，将第一模型参数发送给其他设备。在本实施例中，与第一设备连接的有第二设备和第四设备，在此仅以向第二设备发送第一模型参数为例进行说明，在一些实施例中，第一设备也可以向第四设备发送该第一模型参数，后续由第二设备对第三设备确定的第二融合参数传递回第一设备。
在一些实施例中，数据处理***中各个设备之间传输数据时，对数据加密后再发送。比如，每个设备可以采用基于秘密分享的方法来保护本地训练得到的模型参数，例如，该秘密分享的方法可以通过添加随机掩码的方式实现。在该步骤709中，第一设备可以获取随机掩码，将该随机掩码和该第一模型参数进行融合，得到加密模型参数，然后向与该第一设备连接的第二设备发送该加密模型参数。通过将该随机掩码与第一模型参数融合后再发送，即使其他设备截获到发送的加密模型参数，由于不清楚随机掩码是什么，则无法推断出该第一设备确定的第一模型参数，保证了设备之间数据的传输安全。
另外，通过添加随机掩码，而不是基于同态加密的方式，也就不会出现密文的字节数比明文的字节数大很多倍(例如，大64倍)的情况，也即是不会出现密文膨胀问题，自然也就不会带来很大的计算量和数据传输量。上述添加随机掩码的方式，能够大大减小通信开销，进一步避免单点通信拥塞问题和链路故障问题。
在一些实施例中，每个设备确定出的模型参数可能为浮点数，在采用秘密分享时，每个设备可以将确定的模型参数从浮点数转化为整数，进而对取整后的整数进行取余，将余数发送给其他设备，这样在提高数据传输的安全性的同时，能够有效减小传输的数据量，提高数据处理效率。
在一些实施例中，上述随机掩码的获取过程为：第一设备获取候选随机掩码，然后对该候选随机掩码进行取整，得到该随机掩码。
相应的，第一设备获取到随机掩码后，还可以对第一模型参数进行上述取整，然后将随机掩码和取整后的第一模型参数进行融合。在一些实施例中，第一设备对该第一模型参数进行取整，对该随机掩码和取整后的第一模型参数进行求和，对求和结果进行取余，得到该加密模型参数。
例如，该随机掩码的获取过程可以通过
Figure PCTCN2021125214-appb-000001
实现。其中，
Figure PCTCN2021125214-appb-000002
为候选随机掩码，INT(x)表示对x取整。P是一个很大的整数，用于将候选随机掩码转化为整数，整数P的选择决定了对浮点数精度的保留程度，P越大，浮点数精度保留越高，本申请实施例对其取值不作限定。r 1为随机掩码。
对于第一模型参数，可以通过
Figure PCTCN2021125214-appb-000003
将浮点数的第一模型参数
Figure PCTCN2021125214-appb-000004
转换为整数。
在一些实施例中，在对随机掩码与第一模型参数进行求和时，可以为该第一模型参数赋予权重，这样每个设备对应一个权重，后续基于每个设备的权重，能够对数据处理***中的多个设备确定出的模型参数进行加权合并，得到加密模型参数。
例如，第一设备可以通过(r 1+d 1w 1)modQ，将随机掩码与取整之后的第一模型参数进行融合。其中，d 1是第一设备的权重。Q是一个非常大的素数，mod为取模函数。在一些实施例中，2d 1w 1<Q，这里d 1是整数。在d 1是浮点数的情况下，可以采用同样的方法将d 1也转换为整数。
第一设备融合得到加密模型参数后，可以将该加密模型参数发送至第二设备，由第二设备再在此基础上融合自身确定的第二模型参数。其中，该加密模型参数即为上述取余处理得到的余数。
710、第二设备接收第一设备确定的第一模型参数。
711、第二设备基于该第一模型参数与自身确定的第二模型参数进行融合，得到第一融合参数。
第二设备接收到第一设备发送的第一模型参数后，可以将其与自身确定的第二模型参数进行融合。
在上述第一设备发送加密模型参数的实施例中，该第二设备接收到该加密模型参数后，确定第一融合参数，将该第一融合参数发送给其他第二设备，该第一融合参数为该第二设备基于接收到的该第一模型参数与自身确定的第二模型参数得到的。这样后续设备之间传输的数据均包含有该随机掩码，有效保证了数据处理***中所有设备之间数据传输的安全性。
该融合过程与上述步骤709中的融合过程同理。在一些实施例中，该第二设备基于接收到的加密模型参数与自身确定并取整后的第二模型参数进行求和，对求和结果进行取余得到第一融合参数，将该第一融合参数发送给其他第二设备。
例如，第二设备可以对接收到的(r 1+d 1w 1)modQ，与d 2w 2求和，再取余处理，即可得到(r 1+d 1w 1+d 2w 2)modQ。(r 1+d 1w 1+d 2w 2)modQ即为第一融合参数。其中，d 2为第二设备的权重，w 2为第二设备确定的第二模型参数。
712、第二设备将该第一融合参数发送给第三设备。
713、第三设备接收第二设备发送的该第一融合参数。
714、第三设备基于第一融合参数和自身确定的第三模型参数进行融合，得到第三融合参数。
该步骤713与步骤714与上述步骤710和步骤711同理，在此不多做赘述。例如，第三设备可以执行同理的融合步骤，得到第三融合参数(r 1+d 1w 1+d 2w 2+d 3w 3)modQ。
715、第三设备将第三融合参数发送给第四设备。
716、第四设备接收第三设备发送的第三融合参数。
717、第四设备基于第三融合参数和自身确定的第四模型参数进行融合，得到第二融合参数。
步骤716与步骤717与上述步骤710和步骤711同理，在此不多做赘述。例如，第四设备可以执行同理的融合步骤，得到第二融合参数(r 1+d 1w 1+d 2w 2+d 3w 3+d 1w 4)modQ。当前第四设备已经将数据处理***中所有设备的模型参数均融合，因而，可以将融合得到的第二融合参数返回给第一设备。
718、第四设备将第二融合参数发送给第一设备。
719、第一设备接收该第二融合参数。
该步骤719为第一设备获取第二融合参数的过程。在上述将模型参数融合随机掩码发送的实施例中，第四设备融合得到的可以不是第二融合参数，而是候选融合参数，其中包含有随机掩码，第一设备还需要将该随机掩码去除后，得到第二融合参数。其中，该第二融合参数包括该数据处理***中该至少三个设备确定的模型参数，也即是包括上述第一设备、第二设备、第三设备以及第四设备确定的模型参数，在一些实施例中，该第二融合参数融合了上述第一设备、第二设备、第三设备以及第四设备确定的模型参数。
在一些实施例中，第一设备接收候选融合参数，该候选融合参数包括该随机掩码以及该数据处理***中该至少三个设备确定的模型参数，然后第一设备从该候选融合参数中去除该随机掩码，得到该第二融合参数。例如，第一设备可以计算(r 1+d 1w 1+d 2w 2+d 3w 3+d 4w 4-r 1)modQ。其中，r 1即为随机掩码。
720、第一设备根据该第二融合参数，确定该第一业务处理模型的目标模型参数。
第一设备得到第二融合参数后，可以对其进行处理，得到第一业务处理模型的目标模型参数。该第二融合参数为一种融合结果，如果想要获得目标模型参数，还需要将其转化为单个设备确定出的模型参数。
在一些实施例中，第一设备可以获取该第二融合参数与总权重的商，将该商作为该第一业务处理模型的目标模型参数，该总权重为该数据处理***中该至少三个设备的权重之和。
例如，第一设备可以计算(d 1w 1+d 2w 2+d 3w 3+d 4w 4)/(Pd)，其中，d＝(d 1+d 2+d 3+d 4)是所有设备的权重之和。在取整处理过程中，每个模型参数乘以P，因而在求平均时，需要将其除去，这样即可得到四个设备的模型参数的加权合并结果。
对于每个设备的权重，任一设备的权重可以为样本数据的数量，也可以为其他参数，比如数据质量指标、贡献度指标、奖励指标等，本申请实施例对此不作限定。
当然，上述通过对模型参数进行加权平均的方式得到了目标模型参数，在其它实施例中，也可以直接对模型参数进行平均，其实是加权平均的一种特殊情况。也即是上述d 1＝d 2＝d 3＝d 4＝1。第一设备计算(w 1+w 2+w 3+w 4)/4，将其作为模型参数。
在一些实施例中，上述模型参数以及目标模型参数可以包括N个分段；其中，该N为大于1的整数。这样每个设备负责一个目标模型参数的一个分段的计算，进而数据处理***中的多个设备可以并行计算目标模型参数的分段，能够进一步将计算步骤划分到多个设备中，并行处理能够提高数据处理速度和效率。对于分段，横向联邦学***均，可以并行的计算。也就是分段数量可以与设备数量相同，也可以与设备数量不同。
其中，1≤N≤K，K是横向联邦学习参与方的个数。分段数量可以小于设备数量，数据处理***中的部分设备进行分段计算，一些设备无需进行分段计算。
在一些实施例中，也可以选择模型参数分段个数N>K，分段数量可以大于设备数量，这样每个设备可以负责多个分段的计算，然后分多次计算，每次计算L个模型参数分段的加权平均，L≤K。
在分段并行处理的方式中，上述步骤709可以为：第一设备向与该第一设备连接的第二设备发送该第一模型参数中的第一分段，进而，步骤710中，第二设备接收到第一模型参数中的第一分段。然后步骤711至步骤712，该第二设备基于第一模型参数中的与自身确定的第二模型参数中的第一分段，得到第一融合分段，将第一融合分段发送给第三设备。第三设备和第四设备所执行的步骤同理。第四设备确定出第二融合分段后，能够将该第二融合分段发送至第一设备，进而第一设备获取第二融合参数，该第二融合分段包括该数据处理***中该至少三个设备确定的模型参数中的第一分段。最终步骤720中，第一设备可以根据该第二融合分段，确定该业务处理模型的目标模型参数中的目标第一分段，基于该目标第一分段，获取该业务处理模型的目标模型参数。
当然，在获取目标模型参数时，除了该目标第一分段之外，还需要确定出目标第二分段。该目标第二分段由数据处理***中第一设备之外的设备确定出，以实现由多个设备并行确定模型分段。
因而，第一设备还可以接收第二设备发送的模型参数中的第二分段或第一融合第二分段，对接收到的该第二分段或第一融合第二分段与该第一设备确定的该第一模型参数中的第二分段融合，得到第二融合第二分段，将该第一设备得到的该第二融合第二分段发送至其他第二设备。
在一些实施例中，对于多个分段的融合步骤，可以由第一设备执行，也可以由数据处理***中的其他设备执行。因而，基于该目标第一分段，获取该业务处理模型的目标模型参数的步骤可以包括以下两种方式。
方式一、第一设备接收该目标模型参数中的目标第二分段，该目标第二分段为该目标模型参数中除该目标第一分段之外的分段，第一设备对该目标模型参数中的目标第一分段和目标第二分段进行拼接，得到该第一业务处理模型的目标模型参数。该方式一中，第一设备负责所有目标分段的融合得到目标模型参数。
方式二、第一设备发送该目标第一分段，第一设备接收该第一业务处理模型的目标模型 参数，该目标模型参数由第二设备对该目标第一分段和目标第二分段拼接得到，该目标第二分段为该目标模型参数中除该目标第一分段之外的分段。在该方式二中，第一设备之外的其他设备负责所有目标分段的融合得到目标模型参数，再将目标模型参数同步给该第一设备。
对于该分段并行处理方式中，上述数据处理方法可以包括以下几个步骤。例如，如图9和图8所示，有K＝4个参与方的情况。每个参与方都将其本地模型参数组成的向量分成四段，该分段过程可以将向量平均分为四段，也可以不平均分为四段，本申请实施例对此不作限定。例如，模型参数向量w k＝[w k,1,w k,2,w k,3,w k,4]，k＝1，2，3，4(k表示参与方的索引)，分别由四个参与方来负责分别计算这四个分段的加权平均(即联邦模型参数平均)。由参与方1负责计算模型参数分段1的加权平均流程，如图9所示，参与方2负责计算模型参数分段2的加权平均流程，参与方3负责计算模型参数分段3的加权平均流程。参与方4负责计算模型参数分段4的加权平均流程。在此仅以参与方1的加权平均流程进行说明，其他参与方同理，该流程可以分为以下五个步骤。
步骤①：参与方1的加解密模块生成随机掩码1，记作
Figure PCTCN2021125214-appb-000005
对于其他参与方，则生成其自身的随机掩码，例如，参与方2生成随机掩码2，也即是r 2,2。参与方1的模型融合模块计算其拥有的参数分段1(也即是参与方1的第一模型参数的第一分段)和随机掩码之和，即计算出(r 1,1+d 1w 1,1)modQ，并将结果发送给参与方2。(r 1,1+d 1w 1,1)modQ为参与方1得到的加密分段1。其中，w 1,1是指参与方1的参数分段1，d 1可以选择为参与方1拥有的训练样本的数量或其他参数(如数据质量指标，贡献度指标，奖励指标等)，Q是一个非常大的素数，比如2d 1w 1,1<Q。
步骤②：参与方2将收到的来自参与方1的加密分段1与自己的参数分段1进行求和，即计算(r 1,1+d 1w 1,1+d 2w 2,1)modQ，并将结果发送给参与方3。(r 1,1+d 1w 1,1+d 2w 2,1)modQ为参与方2得到的加密分段1。其中，d 2可以选择为参与方2拥有的训练样本的数量或其他参数。
步骤③：参与方3将收到的来自参与方2的加密分段1与自己的参数分段1进行求和，即计算(r 1,1+d 1w 1,1+d 2w 2,1+d 3w 3,1)modQ，并将结果发送给参与方4。(r 1,1+d 1w 1,1+d 2w 2,1+d 3w 3,1)modQ为参与方3得到的加密分段1。其中，d 3可以选择为参与方3拥有的训练样本的数量或其他参数。
步骤④：参与方4将收到的来自参与方3的加密分段1与自己的参数分段1进行求和，即计算(r 1,1+d 1w 1,1+d 2w 2,1+d 3w 3,1+d 1w 4,1)modQ，并将结果发送给参与方1。(r 1,1+d 1w 1,1+d 2w 2,1+d 3w 3,1+d 1w 4,1)modQ为参与方4得到的加密分段1。其中，d 4可以选择为参与方4拥有的训练样本的数量或其他参数。
步骤⑤：参与方1将收到的来自参与方4的加密分段1减去自己生成的随机掩码1，即计算(r 1,1+d 1w 1,1+d 2w 2,1+d 3w 3,1+d 4w 4,1-r 1,1)modQ，就可获得参数分段1的加权和，即可获得d 1w 1,1+d 2w 2,1+d 3w 3,1+d 4w 4,1。进而，参与方1可以计算模型参数分段1的加权平均(d 1w 1,1+d 2w 2,1+d 3w 3,1+d 4w 4,1)/(Pd)。可以称之为平均分段1，在后续也可以称之为参数分段1，只是此时得到的参数分段1为四个设备的参数分段1的加权平均结果。其中，d＝(d 1+d 2+d 3+d 4)可以是所有参与方的总的训练样本数量或其他参数之和。
上述步骤中，加法和减法运算都是在整数域的运算，需要包括求模运算，也即是模型参数和随机掩码均进行了取整，最后步骤⑤的除法运算整数除以整数的运算，结果是浮点数。需要说明的是，由以上的描述可以可知，因为每个参与方分别负责计算一个模型参数分段的加权平均，所以参与方之间可以并行执行模型参数分段的加权平均步骤。
在一些实施例中，在图3所示的分组场景中，该第一设备为目标设备组中的管理设备；该第二设备为该目标设备组中的跟随设备。例如，在图3所示的双层环形结构的数据处理***中，包括10个参与方，其中，第一个设备组中，参与方3为管理设备，第一个设备组中每个参与方分别负责参数分段1至参数分段3的加权合并，并由参与方3负责将第一组参与方 中每个参与方得到的参数分段拼接得到该第一组参与方得到的模型参数分段1至参数分段3，即
Figure PCTCN2021125214-appb-000006
同理地，参与方6负责将第二个设备组中各个参与方加权合并的参数分段4至参数分段6合并，即
Figure PCTCN2021125214-appb-000007
参与方10负责将第三个设备组中各个参与方加权合并的参数分段7至参数分段10合并，即
Figure PCTCN2021125214-appb-000008
进而，参与方3、6、10则可以再将得到的参数分段加权合并，得到
Figure PCTCN2021125214-appb-000009
其中，
Figure PCTCN2021125214-appb-000010
本申请实施例，一方面，数据处理***中的设备采用环形结构连接，每个设备与邻居设备通信即可，这样一个设备与其他设备有两条通信链路，即使其中一条通信链路暂时中断，该设备也可以通过其他通信链路与其他设备进行通信，因而，该数据处理***不会因某条通信链路的暂时中断而中断数据处理流程，具有很好的稳定性和鲁棒性。另一方面，该数据处理***在进行数据处理时，每个设备确定的模型参数按照上述通信链路依次传递，且传递过程中，每个设备对接收到的模型参数与自身确定的模型参数融合后再传递，这样设备之间传递的数据量较小，且无需向一个设备集中发送模型参数，因而，能够有效避免过载和通信拥塞的问题，能够有效提高数据处理速度和效率，保证数据处理的稳定性。
在一些实施例中，上述图4、图5、图7所示实施例中的数据处理***可以为区块链***，数据处理***中的设备可以为区块链***中的节点设备。
下面对区块链***进行简单介绍。
区块链是分布式数据存储、点对点传输、共识机制、加密算法等计算机技术的新型应用模式。区块链(Blockchain)，本质上是一个去中心化的数据库，是一串使用密码学方法相关联产生的数据块，每一个数据块中包含了一批次网络交易的信息，用于验证其信息的有效性(防伪)和生成下一个区块。区块链可以包括区块链底层平台、平台产品服务层以及应用服务层。
本申请实施例涉及的***可以是由客户端、多个节点(接入网络中的任意形式的计算设备，如服务器、用户终端)通过网络通信的形式连接形成的分布式***。该分布式***可以为区块链***，参见图10，图10是本申请实施例提供的分布式***100应用于区块链***的结构示意图，由多个节点(接入网络中的任意形式的计算设备，如服务器、用户终端)和客户端形成，节点之间形成组成的点对点(P2P，Peer To Peer)网络，P2P协议是一个运行在传输控制协议(Transmission Control Protocol，TCP)协议之上的应用层协议。在分布式***中，任何机器如服务器、终端都可以加入而成为节点，节点包括硬件层、中间层、操作***层和应用层。
参见图10示出的区块链***中各节点的功能，涉及的功能包括以下几种。
1)路由，节点具有的基本功能，用于支持节点之间的通信。
节点除具有路由功能外，还可以具有以下功能。
2)应用，用于部署在区块链中，根据实际业务需求而实现特定业务，记录实现功能相关的数据形成记录数据，在记录数据中携带数字签名以表示任务数据的来源，将记录数据发送到区块链***中的其他节点，供其他节点在验证记录数据来源以及完整性成功时，将记录数据添加到临时区块中。
3)区块链，包括一系列按照产生的先后时间顺序相互接续的区块(Block)，新区块一旦加入到区块链中就不会再被移除，区块中记录了区块链***中节点提交的记录数据。
参见图11，图11是本申请实施例提供的区块结构(Block Structure)的示意图，每个区 块包括本区块存储交易记录的哈希值(本区块的哈希值)、以及前一区块的哈希值，各区块通过哈希值连接形成区块链。另外，区块中还可以包括有区块生成时的时间戳等信息。区块链(Blockchain)，本质上是一个去中心化的数据库，是一串使用密码学方法相关联产生的数据块，每一个数据块中包含了相关的信息，用于验证其信息的有效性(防伪)和生成下一个区块。
在本申请实施例中，上述第一设备、第二设备、第三设备和第四设备均可以为区块链***中的节点设备，多个节点设备采用环形结构连接，每个节点设备可以获取目标业务的样本数据，基于样本数据对业务处理模型进行训练，得到业务处理模型的模型参数。在得到模型参数后，节点设备可以向连接的下一个节点设备发送自身确定的模型参数，下一个节点设备可以将接收到的模型参数与自身确定的模型参数融合后，再发送给连接的下一个节点设备，直至最后一个节点设备传回第一个开始发送的节点设备。该节点设备基于融合的模型参数，能够确定最终的目标模型参数。该节点设备确定最终的目标模型参数后，可以基于该目标模型参数生成区块，将该区块向区块链***中其他节点设备广播该区块，由区块链***对该区块进行共识，在共识通过时，各个节点设备能够将该区块添加到区块链上，这样每个节点设备均同步了目标模型参数。
在分段并行方案中，每个节点设备在得到自己负责的目标分段后，可以由一个节点设备对其进行拼接，得到目标模型参数，并通过生成区块将其上链。每个节点设备也可以基于自己负责的目标分段，生成区块并上链。
在一些实施例中，节点设备也可以在得到自身确定的模型参数后，基于自身确定的模型参数生成区块，通过共识，将区块上链。
这样通过生成区块并上链，将最终确定的目标模型参数或者中间数据上链，能够利用区块链***中链上数据无法被篡改的特性，保证数据安全、准确和完整，且通过区块链共识机制，能够避免单个节点设备计算错误的情况，保证数据的准确性。
上述所有可选技术方案，能够采用任意结合形成本申请的可选实施例，在此不再一一赘述。
图12是本申请实施例提供的一种数据处理装置的结构示意图，参见图12，该装置包括以下功能模块：
获取模块1001，用于获取目标业务的第一样本数据。
训练模块1002，用于基于该第一样本数据对第一业务处理模型进行训练，得到该第一业务处理模型的第一模型参数。
发送模块1003，用于向与该第一设备连接的第二设备发送该第一模型参数，由该第二设备确定第一融合参数，将该第一融合参数发送给其他第二设备，该第一融合参数为该第二设备基于接收到的该第一模型参数与自身确定的第二模型参数得到的，该第二设备为该数据处理***中除该第一设备之外的设备，该数据处理***中的设备采用环形结构连接。
该获取模块1001，还用于获取第二融合参数，该第二融合参数包括该数据处理***中该至少三个设备确定的模型参数。
确定模块1004，用于根据该第二融合参数，确定该第一业务处理模型的目标模型参数。
在一些实施例中，该第一模型参数包括N个分段。其中，该N为大于1的整数。
该发送模块1003用于向与该第一设备连接的该第二设备发送该第一模型参数中的第一分段，由该第二设备确定第一融合分段，将该第一融合分段发送给其他第二设备，该第一融合分段为该第二设备基于接收到的该第一分段与自身确定的第二模型参数中的第一分段得到的。
该获取模块1001用于基于第二融合分段，获取该第二融合参数，该第二融合分段包括该数据处理***中该至少三个设备确定的模型参数中的第一分段。
该确定模块1004用于：
根据该第二融合分段，确定该第一业务处理模型的目标模型参数中的目标第一分段。
基于该目标第一分段，获取该第一业务处理模型的目标模型参数。
在一些实施例中，该获取模块1001用于执行下述任一项：
接收该目标模型参数中的目标第二分段，该目标第二分段为该目标模型参数中除该目标第一分段之外的分段。将该目标第一分段和该目标第二分段进行拼接，得到该第一业务处理模型的目标模型参数。
发送该目标第一分段。确定该第一业务处理模型的目标模型参数，该目标模型参数由该第二设备拼接该目标第一分段和该目标第二分段得到。
在一些实施例中，该获取模块1001还用于接收该第二设备发送的模型参数中的第二分段或第一融合第二分段。
该装置还包括：
融合模块，用于将该第二分段或该第一融合第二分段与该第一设备确定的该模型参数中的第二分段进行融合，得到第二融合第二分段。
该发送模块1003还用于将该第二融合第二分段发送至其他第二设备。
在一些实施例中，该数据处理***包括至少两个设备组，各设备组包括管理设备和跟随设备，各设备组中的设备采用环形结构连接。该至少两个设备组的管理设备采用环形结构连接。该第一设备为目标设备组中的管理设备。该第二设备为该目标设备组中的跟随设备，该目标设备组为该至少两个设备组中的任一设备组。
在一些实施例中，该发送模块1003用于向该目标设备组中的跟随设备发送该第一模型参数，由该跟随设备确定第一融合参数，将该第一融合参数发送给其他跟随设备，该第一融合参数为该跟随设备基于接收到的该第一模型参数与自身确定的第二模型参数得到的。
该获取模块1001用于：
获取第三融合参数，该第三融合参数包括该目标设备组确定的模型参数。
向其他设备组中的管理设备发送该第三融合参数，由该其他设备组中的管理设备确定第四融合参数，将该第四融合参数发送给该其他设备组中的跟随设备，该第四融合参数为该其他设备组中的管理设备基于接收到的该第三融合参数与自身所在组设备确定的融合参数得到的。
获取该第二融合参数，该第二融合参数包括该数据处理***中至少两个设备组确定的融合参数。
在一些实施例中，该发送模块1003用于：
获取随机掩码。
将该随机掩码和该第一模型参数进行融合，得到加密模型参数。
向与该第一设备连接的第二设备发送该加密模型参数，由该第二设备确定第一融合参数，将该第一融合参数发送给其他第二设备，该第一融合参数为该第二设备基于接收到的该加密模型参数与自身确定的第二模型参数得到的。
该获取模块1001用于：
接收候选融合参数，该候选融合参数包括该随机掩码以及该数据处理***中该至少三个设备确定的模型参数。
从该候选融合参数中去除该随机掩码，得到该第二融合参数。
在一些实施例中，该融合模块用于：
对该第一模型参数进行取整。
对该随机掩码和取整后的该第一模型参数进行求和。
对求和结果进行取余，得到该加密模型参数。
该由该第二设备确定第一融合参数，将该第一融合参数发送给其他第二设备，包括：
由该第二设备基于接收到的加密模型参数与自身确定并取整后的第二模型参数进行求和， 对求和结果进行取余得到该第一融合参数，将该第一融合参数发送给其他第二设备。
在一些实施例中，该融合模块用于：
对该第一模型参数进行取整。
获取整后的该第一模型参数与该第一设备的权重的第一乘积。
对该随机掩码和该第一乘积进行求和。
对求和结果进行取余，得到该加密模型参数。
由该第二设备基于接收到的加密模型参数与第二乘积进行求和，对求和结果进行取余，得到该第一融合参数，将该第一融合参数发送给其他第二设备，该第二乘积为该第二设备自身确定并取整后的第二模型参数与该第二设备的权重的乘积。
在一些实施例中，该发送模块1003用于：获取候选随机掩码。
对该候选随机掩码进行取整，得到该随机掩码。
在一些实施例中，该确定模块1004，用于获取该第二融合模型参数与总权重的商，将该商作为该第一业务处理模型的该目标模型参数，该总权重为该数据处理***中该至少三个设备的权重之和。
本申请实施例，一方面，数据处理***中的设备采用环形结构连接，每个设备与邻居设备通信即可，这样一个设备与其他设备有两条通信链路，即使其中一条通信链路暂时中断，该设备也可以通过其他通信链路与其他设备进行通信，因而，该数据处理***不会因某条通信链路的暂时中断而中断数据处理流程，具有很好的稳定性和鲁棒性。另一方面，该数据处理***在进行数据处理时，每个设备确定的模型参数按照上述通信链路依次传递，且传递过程中，每个设备对接收到的模型参数与自身确定的模型参数融合后再传递，这样设备之间传递的数据量较小，且无需向一个设备集中发送模型参数，因而，能够有效避免过载和通信拥塞的问题，能够有效提高数据处理速度和效率，保证数据处理的稳定性。
需要说明的是：上述实施例提供的数据处理装置在数据处理时，仅以上述各功能模块的划分进行举例说明，实际应用中，能够根据需要而将上述功能分配由不同的功能模块完成，即将数据处理装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的数据处理装置与数据处理方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
图13是本申请实施例提供的一种数据处理装置的结构示意图，参见图13，该装置包括：
获取模块1101，用于获取目标业务的第二样本数据。
训练模块1102，用于基于该第二样本数据对第二业务处理模型进行训练，得到该第二业务处理模型的第二模型参数。
接收模块1103，用于接收第一设备确定的第一模型参数，该至少三个设备包括该第一设备。
融合模块1104，用于将该第一模型参数与自身确定的第二模型参数进行融合，得到第一融合参数。
发送模块1105，用于将该第一融合参数发送给该数据处理***中的其他设备，该数据处理***中的设备采用环形结构连接。
该接收模块1103还用于确定该第二业务处理模型的目标模型参数。
在一些实施例中，该第一模型参数包括N个分段。其中，该N为大于1的整数。
该接收模块1103用于：
接收该第一设备确定的第一模型参数中的第一分段。
该接收模块1103还用于将该第一设备确定的第一分段与自身确定的第二模型参数中的第一分段进行融合，得到第一融合分段。
将该第一融合分段发送给该数据处理***中的其他设备。
获取第二融合分段，该目第二融合分段包括该数据处理***中该至少三个设备确定的模 型参数中的第二分段。
根据该第二融合分段，确定该第二业务处理模型的目标模型参数中的目标第二分段。
基于该目标第二分段，获取该第二业务处理模型的目标模型参数。
本申请实施例，一方面，数据处理***中的设备采用环形结构连接，每个设备与邻居设备通信即可，这样一个设备与其他设备有两条通信链路，即使其中一条通信链路暂时中断，该设备也可以通过其他通信链路与其他设备进行通信，因而，该数据处理***不会因某条通信链路的暂时中断而中断数据处理流程，具有很好的稳定性和鲁棒性。另一方面，该数据处理***在进行数据处理时，每个设备确定的模型参数按照上述通信链路依次传递，且传递过程中，每个设备对接收到的模型参数与自身确定的模型参数融合后再传递，这样设备之间传递的数据量较小，且无需向一个设备集中发送模型参数，因而，能够有效避免过载和通信拥塞的问题，能够有效提高数据处理速度和效率，保证数据处理的稳定性。
需要说明的是：上述实施例提供的数据处理装置在处理数据时，仅以上述各功能模块的划分进行举例说明，实际应用中，能够根据需要而将上述功能分配由不同的功能模块完成，即将数据处理装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的数据处理装置与数据处理方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
图14是本申请实施例提供的一种电子设备的结构示意图，该电子设备1200可因配置或性能不同而产生比较大的差异，能够包括一个或一个以上处理器(Central Processing Units，CPU)1201和一个或一个以上的存储器1202，其中，该存储器1202中存储有至少一条计算机程序，该至少一条计算机程序由该处理器1201加载并执行以实现上述各个方法实施例提供的数据处理方法。该电子设备还能够包括其他用于实现设备功能的部件，例如，该电子设备还能够具有有线或无线网络接口以及输入输出接口等部件，以便进行输入输出。本申请实施例在此不做赘述。
上述方法实施例中的电子设备能够实现为终端。例如，图15是本申请实施例提供的一种终端的结构框图。该终端1300可以是便携式移动终端，比如：智能手机、平板电脑、笔记本电脑或台式电脑。终端1300还可能被称为用户设备、便携式终端、膝上型终端、台式终端等其他名称。
通常，终端1300包括有：处理器1301和存储器1302。
处理器1301可以包括一个或多个处理核心，比如4核心处理器、8核心处理器等。处理器1301可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。在一些实施例中，处理器1301还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
存储器1302可以包括一个或多个计算机可读存储介质，该计算机可读存储介质可以是非暂态的。存储器1302还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器1302中的非暂态的计算机可读存储介质用于存储至少一个指令，该至少一个指令用于被处理器1301所执行以实现本申请中方法实施例提供的数据处理方法。
在一些实施例中，终端1300还可选包括有：***设备接口1303和至少一个***设备。处理器1301、存储器1302和***设备接口1303之间可以通过总线或信号线相连。各个***设备可以通过总线、信号线或电路板与***设备接口1303相连。在一些实施例中，***设备包括：射频电路1304、显示屏1305和电源1306中的至少一种。
***设备接口1303可被用于将I/O(Input/Output，输入/输出)相关的至少一个***设备连接到处理器1301和存储器1302。
射频电路1304用于接收和发射RF(Radio Frequency，射频)信号，也称电磁信号。射频 电路1304通过电磁信号与通信网络以及其他通信设备进行通信。射频电路1304将电信号转换为电磁信号进行发送，或者，将接收到的电磁信号转换为电信号。射频电路1304可以通过至少一种无线通信协议来与其它终端进行通信。该无线通信协议包括但不限于：万维网、城域网、内联网、各代移动通信网络(2G、3G、4G及5G)、无线局域网和/或WiFi(Wireless Fidelity，无线保真)网络。在一些实施例中，射频电路1304还可以包括NFC(Near Field Communication，近距离无线通信)有关的电路，本申请对此不加以限定。
显示屏1305用于显示UI(User Interface，用户界面)。该UI可以包括图形、文本、图标、视频及其它们的任意组合。当显示屏1305是触摸显示屏时，显示屏1305还具有采集在显示屏1305的表面或表面上方的触摸信号的能力。该触摸信号可以作为控制信号输入至处理器1301进行处理。
电源1306用于为终端1300中的各个组件进行供电。电源1306可以是交流电、直流电、一次性电池或可充电电池。当电源1306包括可充电电池时，该可充电电池可以是有线充电电池或无线充电电池。有线充电电池是通过有线线路充电的电池，无线充电电池是通过无线线圈充电的电池。该可充电电池还可以用于支持快充技术。
在一些实施例中，终端1300还包括有一个或多个传感器1307。该一个或多个传感器1307包括但不限于：压力传感器1308以及光学传感器1309。
压力传感器1308可以设置在终端1300的侧边框和/或显示屏1305的下层。当压力传感器1308设置在终端1300的侧边框时，可以检测用户对终端1300的握持信号，由处理器1301根据压力传感器1308采集的握持信号进行左右手识别或快捷操作。当压力传感器1308设置在显示屏1305的下层时，由处理器1301根据用户对显示屏1305的压力操作，实现对UI界面上的可操作性控件进行控制。可操作性控件包括按钮控件、滚动条控件、图标控件、菜单控件中的至少一种。
光学传感器1309用于采集环境光强度。在一个实施例中，处理器1301可以根据光学传感器1309采集的环境光强度，控制显示屏1305的显示亮度。在一些实施例中，当环境光强度较高时，调高显示屏1305的显示亮度；当环境光强度较低时，调低显示屏1305的显示亮度。
本领域技术人员可以理解，图15中示出的结构并不构成对终端1300的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
上述方法实施例中的电子设备能够实现为服务器。例如，图16是本申请实施例提供的一种服务器的结构示意图，该服务器1400可因配置或性能不同而产生比较大的差异，能够包括一个或一个以上处理器(Central Processing Units，CPU)1401和一个或一个以上的存储器1402，其中，该存储器1402中存储有至少一条计算机程序，该至少一条计算机程序由该处理器1401加载并执行以实现上述各个方法实施例提供的数据处理方法。当然，该服务器还能够具有有线或无线网络接口以及输入输出接口等部件，以便进行输入输出，该服务器还能够包括其他用于实现设备功能的部件，在此不做赘述。
在示例性实施例中，还提供了一种计算机可读存储介质，例如包括至少一条计算机程序的存储器，上述至少一条计算机程序由可由处理器执行以完成上述实施例中的数据处理方法。例如，计算机可读存储介质能够是只读存储器(Read-Only Memory，简称：ROM)、随机存取存储器(Random Access Memory，简称：RAM)、只读光盘(Compact Disc Read-Only Memory，简称：CD-ROM)、磁带、软盘和光数据存储设备等。
在示例性实施例中，还提供了一种计算机程序产品或计算机程序，该计算机程序产品或该计算机程序包括一条或多条程序代码，该一条或多条程序代码存储在计算机可读存储介质中。电子设备的一个或多个处理器能够从计算机可读存储介质中读取该一条或多条程序代码，该一个或多个处理器执行该一条或多条程序代码，使得电子设备能够执行上述数据处理方法。
本领域普通技术人员能够理解实现上述实施例的全部或部分步骤能够通过硬件来完成， 也能够通过程序来指令相关的硬件完成，该程序能够存储于一种计算机可读存储介质中，上述提到的存储介质能够是只读存储器，磁盘或光盘等。
以上描述仅为本申请的可选实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (17)

  1. 一种数据处理方法，由数据处理***中的第一设备执行，所述数据处理***包括至少三个设备，所述数据处理***中的设备采用环形结构连接，所述方法包括：
    获取目标业务的第一样本数据；
    基于所述第一样本数据对第一业务处理模型进行训练，得到所述第一业务处理模型的第一模型参数；
    向与所述第一设备连接的第二设备发送所述第一模型参数，由所述第二设备确定第一融合参数，将所述第一融合参数发送给其他第二设备，所述第一融合参数为所述第二设备基于接收到的所述第一模型参数与自身确定的第二模型参数得到的，所述第二设备为所述数据处理***中除所述第一设备之外的设备；
    获取第二融合参数，所述第二融合参数包括所述数据处理***中所述至少三个设备确定的模型参数；
    根据所述第二融合参数，确定所述第一业务处理模型的目标模型参数。
  2. 根据权利要求1所述的方法，其中，所述第一模型参数包括N个分段，所述N为大于1的整数；
    所述向与所述第一设备连接的第二设备发送所述第一模型参数，包括：
    向与所述第一设备连接的所述第二设备发送所述第一模型参数中的第一分段，由所述第二设备确定第一融合分段，将所述第一融合分段发送给其他第二设备，所述第一融合分段为所述第二设备基于接收到的所述第一分段与自身确定的第二模型参数中的第一分段得到的；
    所述获取第二融合参数，包括：
    基于第二融合分段，获取所述第二融合参数，所述第二融合分段包括所述数据处理***中所述至少三个设备确定的模型参数中的第一分段；
    所述根据所述第二融合参数，确定所述第一业务处理模型的目标模型参数，包括：
    根据所述第二融合分段，确定所述第一业务处理模型的目标模型参数中的目标第一分段；
    基于所述目标第一分段，获取所述第一业务处理模型的目标模型参数。
  3. 根据权利要求2所述的方法，其中，所述基于所述目标第一分段，获取所述第一业务处理模型的目标模型参数，包括下述任一项：
    接收所述目标模型参数中的目标第二分段，所述目标第二分段为所述目标模型参数中除所述目标第一分段之外的分段；将所述目标第一分段和所述目标第二分段进行拼接，得到所述第一业务处理模型的目标模型参数；
    发送所述目标第一分段；确定所述第一业务处理模型的目标模型参数，所述目标模型参数由所述第二设备拼接所述目标第一分段和所述目标第二分段得到。
  4. 根据权利要求2所述的方法，其中，所述方法还包括：
    接收所述第二设备发送的模型参数中的第二分段或第一融合第二分段；
    将所述第二分段或所述第一融合第二分段与所述第一设备确定的所述模型参数中的第二分段进行融合，得到第二融合第二分段；
    将所述第二融合第二分段发送至其他第二设备。
  5. 根据权利要求1所述的方法，其中，所述数据处理***包括至少两个设备组，各设备组包括管理设备和跟随设备，各设备组中的设备采用环形结构连接；所述至少两个设备组的管 理设备采用环形结构连接；所述第一设备为目标设备组中的管理设备；所述第二设备为所述目标设备组中的跟随设备，所述目标设备组为所述至少两个设备组中的任一设备组。
  6. 根据权利要求5所述的方法，其中，所述向与所述第一设备连接的第二设备发送所述第一模型参数，包括：
    向所述目标设备组中的跟随设备发送所述第一模型参数，由所述跟随设备确定第一融合参数，将所述第一融合参数发送给其他跟随设备，所述第一融合参数为所述跟随设备基于接收到的所述第一模型参数与自身确定的第二模型参数得到的；
    所述获取第二融合参数，包括：
    获取第三融合参数，所述第三融合参数包括所述目标设备组确定的模型参数；
    向其他设备组中的管理设备发送所述第三融合参数，由所述其他设备组中的管理设备确定第四融合参数，将所述第四融合参数发送给所述其他设备组中的跟随设备，所述第四融合参数为所述其他设备组中的管理设备基于接收到的所述第三融合参数与自身所在组设备确定的融合参数得到的；
    获取所述第二融合参数，所述第二融合参数包括所述数据处理***中至少两个设备组确定的融合参数。
  7. 根据权利要求1所述的方法，其中，所述向与所述第一设备连接的第二设备发送所述第一模型参数，包括：
    获取随机掩码；
    将所述随机掩码和所述第一模型参数进行融合，得到加密模型参数；
    向与所述第一设备连接的第二设备发送所述加密模型参数，由所述第二设备确定第一融合参数，将所述第一融合参数发送给其他第二设备，所述第一融合参数为所述第二设备基于接收到的所述加密模型参数与自身确定的第二模型参数得到的；
    所述获取第二融合参数，包括：
    接收候选融合参数，所述候选融合参数包括所述随机掩码以及所述数据处理***中所述至少三个设备确定的模型参数；
    从所述候选融合参数中去除所述随机掩码，得到所述第二融合参数。
  8. 根据权利要求7所述的方法，其中，所述将所述随机掩码和所述第一模型参数进行融合，得到加密模型参数，包括：
    对所述第一模型参数进行取整；
    对所述随机掩码和取整后的所述第一模型参数进行求和；
    对求和结果进行取余，得到所述加密模型参数；
    所述由所述第二设备确定第一融合参数，将所述第一融合参数发送给其他第二设备，包括：
    由所述第二设备基于接收到的加密模型参数与自身确定并取整后的第二模型参数进行求和，对求和结果进行取余得到所述第一融合参数，将所述第一融合参数发送给其他第二设备。
  9. 根据权利要求7所述的方法，其中，所述将所述随机掩码和所述第一模型参数进行融合，得到加密模型参数，包括：
    对所述第一模型参数进行取整；
    获取取整后的所述第一模型参数与所述第一设备的权重的第一乘积；
    对所述随机掩码和所述第一乘积进行求和；
    对求和结果进行取余，得到所述加密模型参数；
    所述由所述第二设备确定第一融合参数，将所述第一融合参数发送给其他第二设备，包 括：
    由所述第二设备基于接收到的加密模型参数与第二乘积进行求和，对求和结果进行取余，得到所述第一融合参数，将所述第一融合参数发送给其他第二设备，所述第二乘积为所述第二设备自身确定并取整后的第二模型参数与所述第二设备的权重的乘积。
  10. 根据权利要求1所述的方法，其中，所述获取随机掩码，包括：
    获取候选随机掩码；
    对所述候选随机掩码进行取整，得到所述随机掩码。
  11. 根据权利要求1所述的方法，其中，所述根据所述第二融合参数，确定所述第一业务处理模型的目标模型参数，包括：
    获取所述第二融合模型参数与总权重的商，将所述商作为所述第一业务处理模型的所述目标模型参数，所述总权重为所述数据处理***中所述至少三个设备的权重之和。
  12. 一种数据处理方法，由数据处理***中的第二设备执行，所述数据处理***包括至少三个设备，所述数据处理***中的设备采用环形结构连接，所述方法包括：
    获取目标业务的第二样本数据；
    基于所述第二样本数据对第二业务处理模型进行训练，得到所述第二业务处理模型的第二模型参数；
    接收第一设备确定的第一模型参数，所述至少三个设备包括所述第一设备；
    将所述第一模型参数与自身确定的第二模型参数进行融合，得到第一融合参数；
    将所述第一融合参数发送给所述数据处理***中的其他设备；
    确定所述第二业务处理模型的目标模型参数。
  13. 根据权利要求12所述的方法，其中，所述第二模型参数包括N个分段；其中，所述N为大于1的整数；
    所述接收第一设备确定的第一模型参数，包括：
    接收所述第一设备确定的第一模型参数中的第一分段；
    所述确定所述第二业务处理模型的目标模型参数包括：
    将所述第一设备确定的第一分段与自身确定的第二模型参数中的第一分段进行融合，得到第一融合分段；
    将所述第一融合分段发送给所述数据处理***中的其他设备；
    获取第二融合分段，所述目第二融合分段包括所述数据处理***中所述至少三个设备确定的模型参数中的第二分段；
    根据所述第二融合分段，确定所述第二业务处理模型的目标模型参数中的目标第二分段；
    基于所述目标第二分段，获取所述第二业务处理模型的目标模型参数。
  14. 一种数据处理***，所述数据处理***包括至少三个设备，所述至少三个设备包括第一设备和至少两个第二设备；所述数据处理***中的设备采用环形结构连接；
    第一设备用于获取目标业务的第一样本数据；基于所述第一样本数据对第一业务处理模型进行训练，得到所述第一业务处理模型的第一模型参数；向与所述第一设备连接的第二设备发送所述第一模型参数，由所述第二设备确定第一融合参数，将所述第一融合参数发送给其他第二设备，所述第一融合参数为所述第二设备基于接收到的所述第一模型参数与自身确定的第二模型参数得到的，所述第二设备为所述数据处理***中除所述第一设备之外的设备；获取第二融合模型参数，所述第二融合模型参数包括所述数据处理***中所述至少三个设备确定的模型参数；根据所述第二融合模型参数，确定所述第一业务处理模型的目标模型参数；
    任一第二设备用于获取所述目标业务的第二样本数据；基于所述第二样本数据对第二业务处理模型进行训练，得到所述第二业务处理模型的第二模型参数；接收所述第一设备确定的第一模型参数；将所述第一模型参数与自身确定的第二模型参数进行融合，得到第一融合参数；将所述第一融合参数发送给所述数据处理***中的其他设备；确定所述第二业务处理模型的目标模型参数。
  15. 一种数据处理装置，所述装置包括：
    获取模块，用于获取目标业务的第一样本数据；
    训练模块，用于基于所述第一样本数据对第一业务处理模型进行训练，得到所述第一业务处理模型的第一模型参数；
    发送模块，用于向与所述第一设备连接的第二设备发送所述第一模型参数，由所述第二设备确定第一融合参数，将所述第一融合参数发送给其他第二设备，所述第一融合参数为所述第二设备基于接收到的所述第一模型参数与自身确定的第二模型参数得到的，所述第二设备为所述数据处理***中除所述第一设备之外的设备，所述数据处理***中的设备采用环形结构连接；
    所述获取模块，还用于获取第二融合参数，所述第二融合参数包括所述数据处理***中所述至少三个设备确定的模型参数；
    确定模块，用于根据所述第二融合参数，确定所述第一业务处理模型的目标模型参数。
  16. 一种电子设备，所述电子设备包括一个或多个处理器和一个或多个存储器，所述一个或多个存储器中存储有至少一条计算机程序，所述至少一条计算机程序由所述一个或多个处理器加载并执行以实现如权利要求1至权利要求11任一项所述的数据处理方法，或如权利要求12或权利要求13任一项所述的数据处理方法。
  17. 一种计算机可读存储介质，所述存储介质中存储有至少一条计算机程序，所述至少一条计算机程序由处理器加载并执行以实现如权利要求1至权利要求11任一项所述的数据处理方法，或如权利要求12或权利要求13任一项所述的数据处理方法。
PCT/CN2021/125214 2020-12-02 2021-10-21 数据处理方法、装置、设备及存储介质 WO2022116725A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2023513949A JP7512518B2 (ja) 2020-12-02 2021-10-21 データ処理方法、装置、電子機器及びコンピュータプログラム
EP21899759.1A EP4203398A4 (en) 2020-12-02 2021-10-21 DATA PROCESSING METHOD, APPARATUS, DEVICE AND STORAGE MEDIUM
US17/969,159 US20230040570A1 (en) 2020-12-02 2022-10-19 Data processing method, apparatus, and device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011385627.5A CN112217706B (zh) 2020-12-02 2020-12-02 数据处理方法、装置、设备及存储介质
CN202011385627.5 2020-12-02

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/969,159 Continuation US20230040570A1 (en) 2020-12-02 2022-10-19 Data processing method, apparatus, and device, and storage medium

Publications (1)

Publication Number Publication Date
WO2022116725A1 true WO2022116725A1 (zh) 2022-06-09

Family

ID=74068109

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/125214 WO2022116725A1 (zh) 2020-12-02 2021-10-21 数据处理方法、装置、设备及存储介质

Country Status (5)

Country Link
US (1) US20230040570A1 (zh)
EP (1) EP4203398A4 (zh)
JP (1) JP7512518B2 (zh)
CN (1) CN112217706B (zh)
WO (1) WO2022116725A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112217706B (zh) * 2020-12-02 2021-03-19 腾讯科技（深圳）有限公司 数据处理方法、装置、设备及存储介质
CN112733967B (zh) * 2021-03-30 2021-06-29 腾讯科技（深圳）有限公司 联邦学习的模型训练方法、装置、设备及存储介质
CN112801307B (zh) * 2021-04-13 2021-07-06 深圳索信达数据技术有限公司 基于区块链的联邦学习方法、装置和计算机设备

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090219818A1 (en) * 2008-03-03 2009-09-03 Masahiko Tsuchiya Node device, packet switch device, communication system and method of communicating packet data
CN106789083A (zh) * 2017-01-12 2017-05-31 成都信息工程大学 基于可信第三方的环状量子信任评估模型
CN109871702A (zh) * 2019-02-18 2019-06-11 深圳前海微众银行股份有限公司 联邦模型训练方法、***、设备及计算机可读存储介质
CN110263921A (zh) * 2019-06-28 2019-09-20 深圳前海微众银行股份有限公司 一种联邦学习模型的训练方法及装置
CN110472745A (zh) * 2019-08-06 2019-11-19 深圳前海微众银行股份有限公司 一种联邦学习中的信息传输方法及装置
CN110490738A (zh) * 2019-08-06 2019-11-22 深圳前海微众银行股份有限公司 一种混合联邦学习方法及架构
US10666342B1 (en) * 2019-05-01 2020-05-26 Qualcomm Incorporated Beam management using adaptive learning
CN111695629A (zh) * 2020-06-11 2020-09-22 腾讯科技（深圳）有限公司 用户特征获取方法、装置、计算机设备及存储介质
CN112217706A (zh) * 2020-12-02 2021-01-12 腾讯科技（深圳）有限公司 数据处理方法、装置、设备及存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2016129363A1 (ja) * 2015-02-12 2018-01-11 学校法人東京理科大学 分散装置、演算・復元装置、及びプログラム
JP6961324B2 (ja) 2015-08-25 2021-11-05 株式会社日立製作所 検索可能暗号処理システム
JP6753874B2 (ja) 2018-02-16 2020-09-09 日本電信電話株式会社 分散深層学習システム
JP7135468B2 (ja) 2018-06-11 2022-09-13 日本電信電話株式会社 分散処理システムおよび分散処理方法
CN111612168B (zh) * 2020-06-30 2021-06-15 腾讯科技（深圳）有限公司 一种机器学习任务的管理方法以及相关装置

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090219818A1 (en) * 2008-03-03 2009-09-03 Masahiko Tsuchiya Node device, packet switch device, communication system and method of communicating packet data
CN106789083A (zh) * 2017-01-12 2017-05-31 成都信息工程大学 基于可信第三方的环状量子信任评估模型
CN109871702A (zh) * 2019-02-18 2019-06-11 深圳前海微众银行股份有限公司 联邦模型训练方法、***、设备及计算机可读存储介质
US10666342B1 (en) * 2019-05-01 2020-05-26 Qualcomm Incorporated Beam management using adaptive learning
CN110263921A (zh) * 2019-06-28 2019-09-20 深圳前海微众银行股份有限公司 一种联邦学习模型的训练方法及装置
CN110472745A (zh) * 2019-08-06 2019-11-19 深圳前海微众银行股份有限公司 一种联邦学习中的信息传输方法及装置
CN110490738A (zh) * 2019-08-06 2019-11-22 深圳前海微众银行股份有限公司 一种混合联邦学习方法及架构
CN111695629A (zh) * 2020-06-11 2020-09-22 腾讯科技（深圳）有限公司 用户特征获取方法、装置、计算机设备及存储介质
CN112217706A (zh) * 2020-12-02 2021-01-12 腾讯科技（深圳）有限公司 数据处理方法、装置、设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4203398A4 *

Also Published As

Publication number Publication date
EP4203398A1 (en) 2023-06-28
EP4203398A4 (en) 2024-05-01
CN112217706A (zh) 2021-01-12
CN112217706B (zh) 2021-03-19
JP2023538967A (ja) 2023-09-12
JP7512518B2 (ja) 2024-07-08
US20230040570A1 (en) 2023-02-09

Similar Documents

Publication Publication Date Title
WO2022116725A1 (zh) 数据处理方法、装置、设备及存储介质
CN113177645B (zh) 联邦学习方法、装置、计算设备及存储介质
CN112084422B (zh) 一种账号数据智能处理方法和装置
CN110570283A (zh) 基于区块链的购物方法及***
CN108777625A (zh) 签名的验证方法、装置和***、存储介质、电子装置
Bahutair et al. Multi-perspective trust management framework for crowdsourced IoT services
CN110659134A (zh) 一种应用于人工智能平台的数据处理方法及装置
Zhang et al. A data set accuracy weighted random forest algorithm for IoT fault detection based on edge computing and blockchain
CN112989399B (zh) 数据处理***及方法
Yuan et al. A multi-source feedback based trust calculation mechanism for edge computing
CN111090631A (zh) 分布式环境下的信息共享方法、装置和电子设备
Jin et al. A distributed fog computing architecture supporting multiple migrating mode
CN115049070A (zh) 联邦特征工程数据的筛选方法及装置、设备及存储介质
Zhang et al. A trustworthy safety inspection framework using performance-security balanced blockchain
CN110489955A (zh) 应用于电子设备的图像处理、装置、计算设备、介质
Rashid et al. Edgestore: Towards an edge-based distributed storage system for emergency response
CN112231768B (zh) 数据处理方法、装置、计算机设备及存储介质
Priyadharshini et al. Light chain consensus reinforcement machine learning: an effective blockchain model for internet of things using for its advancement and challenges
Zhang et al. Can Transformer and GNN Help Each Other?
Wang et al. C3Meta: a context-aware cloud-edge-end collaboration framework toward green metaverse
Huang et al. AFLChain: blockchain-enabled asynchronous federated learning in edge computing network
Zhang et al. Deploying GIS services into the edge: A study from performance evaluation and optimization viewpoint
US10564601B2 (en) Method and system for image processing and data transmission in network-based multi-camera environment
CN117519996B (zh) 一种数据处理方法、装置、设备以及存储介质
Bandara et al. Lightweight, geo-scalable deterministic blockchain design for 5G networks sliced applications with hierarchical CFT/BFT consensus groups, IPFS and novel hardware design

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21899759

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023513949

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021899759

Country of ref document: EP

Effective date: 20230323

NENP Non-entry into the national phase

Ref country code: DE