CN109543725A - A kind of method and device obtaining model parameter - Google Patents
A kind of method and device obtaining model parameter Download PDFInfo
- Publication number
- CN109543725A CN109543725A CN201811313450.0A CN201811313450A CN109543725A CN 109543725 A CN109543725 A CN 109543725A CN 201811313450 A CN201811313450 A CN 201811313450A CN 109543725 A CN109543725 A CN 109543725A
- Authority
- CN
- China
- Prior art keywords
- calculate node
- data acquisition
- acquisition system
- client device
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present application discloses a kind of method and device for obtaining model parameter, wherein method includes: the M calculate node by selecting to be stored with data acquisition system from block catenary system, client device can select K calculate node to carry out model training from M calculate node, and obtain the training result of P calculate node in K calculate node；Further, client device can obtain model parameter according to the training result of data acquisition system.In the embodiment of the present application, by using the calculate node training pattern in block catenary system, it may not need the exclusive machine learning algorithm group of building and reduce the waste of social resources so as to save cost；And by having in selection block catenary system, relatively the calculate node of storage and processing ability runs algorithm by force, the training effectiveness of model can be improved, and then obtain preferable model training effect.
Description
Technical field
This application involves technical field of data processing more particularly to a kind of method and devices for obtaining model parameter.
Background technique
At this stage, with the deep development of artificial intelligence technology, model is trained gradually using machine learning method
Popular research direction as algorithm development field.Specifically, can usually develop on a client device can by developer
With the algorithm for training pattern, and using client device acquisition for the data acquisition system of training pattern and for training mould
The algorithm of type carries out model training, obtains model parameter.However, this kind of method depends on the process performance of client device, it is right
It may be such that in the bigger data of more complicated algorithm or treating capacity if the process performance of client device is inadequate
The runing time of algorithm is longer, lower so as to cause the operational efficiency of algorithm, and model training effect is poor.
In order to improve the operational efficiency of algorithm, relatively good model training effect is obtained, many scientific research institutions would generally structure
Exclusive machine learning algorithm cluster is built, and using the multiple client equipment in exclusive machine learning algorithm cluster to privately owned
Data are trained.Using this kind of method, although can guarantee that the process performance of client device is met the requirements, building is special
The machine learning algorithm cluster of category needs to expend a large amount of man power and material, so that the higher cost of exploitation；And due to exclusive
Machine learning algorithm cluster may can not be used by other scientific research institutions, it is thus possible to can make different scientific research machines
Structure constructs different machine learning algorithm clusters, to cause the waste of resource.
To sum up, a kind of method for obtaining model parameter is needed at present, and exclusive machine is constructed using the prior art for solving
Device learning algorithm cluster obtains the technical issues of higher cost, the wasting of resources caused by model parameter.
Summary of the invention
The embodiment of the present application provides a kind of method for obtaining model parameter, exclusive for solving to construct using the prior art
Machine learning algorithm cluster obtains the technical issues of higher cost, the wasting of resources caused by model parameter.
A kind of method obtaining model parameter provided by the embodiments of the present application, comprising:
Client device obtains the mark of the data acquisition system for training pattern and the algorithm for training pattern；
The client device is according to the mark of the data acquisition system, and selection is stored with the data from block catenary system
M calculate node of set；
The client device is selected from the M calculate node according to the first attribute information of the M calculate node
Select K calculate node；In the M calculate node the first attribute information of each calculate node include the calculate node
Line stability, calculated performance calculate cost, are in the calculated result confidence level of the calculate node one or more；
The mark of the algorithm and the data acquisition system is sent to the K calculate node by the client device, with
Make the K calculate node respectively using in the data acquisition system data and the algorithm model is trained；
The client device obtains the corresponding data acquisition system of P calculate node in the K calculate node
Training result, and according to the training result of the data acquisition system, obtain model parameter；
Wherein, M >=K >=P, P >=2, M, K and P are integer.
Optionally, the client device selects the M calculating section for being stored with the data acquisition system from block catenary system
Point, comprising:
The client device obtains and the data from the block catenary system according to the mark of the data acquisition system
The corresponding storage relation table of the mark of set, the storage relation table include multiple calculate nodes where the data acquisition system
Mark；
The client device is according to the storage relation table, and selection is stored with the data acquisition system from block catenary system
M calculate node.
Optionally, the client device gets the corresponding number of P calculate node in the K calculate node
After the training result of set, further includes:
The client device analyzes the training result of the corresponding data acquisition system of the P calculate node,
If it is determined that there are incredible first calculate nodes of training result for the P calculate node, then first is sent to block catenary system
Information, the first information are used to indicate the block catenary system and carry out to the calculated result confidence level of first calculate node
It updates.
Optionally, the method also includes:
First distribution of earnings strategy is sent respectively to the K calculate node by the client device, and described first receives
Entering allocation strategy includes the shortest getable income of P calculate node institute of duration needed for training pattern.
A kind of method obtaining model parameter provided by the embodiments of the present application, comprising:
Calculate node in block catenary system receive that client device sends for the algorithm of training pattern and for instructing
Practice the mark of the data acquisition system of model；
The calculate node obtains the data acquisition system of storage, uses the number according to the mark of the data acquisition system
According in set data and the algorithm model is trained, and the instruction information for obtaining training result is sent to institute
State client device；Data in the data acquisition system are to be obtained by the public data on downloading network.
The embodiment of the present application provides a kind of client device, which includes:
Module is obtained, is used for the mark of the data acquisition system of training pattern and for the algorithm of training pattern for obtaining；
Processing module, for the mark according to the data acquisition system, selection is stored with the data from block catenary system
M calculate node of set；
According to the first attribute information of the M calculate node, K calculate node is selected from the M calculate node；
The first attribute information of each calculate node includes the online stability of the calculate node, calculating in the M calculate node
Performance calculates cost, is in the calculated result confidence level of the calculate node one or more；
The mark of the algorithm and the data acquisition system is sent to the K calculate node, so that the K calculating saves
Point respectively using in the data acquisition system data and the algorithm model is trained；
The acquisition module is also used to obtain the corresponding data set of P calculate node in the K calculate node
The training result of conjunction, and according to the training result of the data acquisition system, obtain model parameter；
Wherein, M >=K >=P, P >=2, M, K and P are integer.
Optionally, the processing module, is specifically used for:
According to the mark of the data acquisition system, obtained from the block catenary system corresponding with the mark of the data acquisition system
Storage relation table, the storage relation table includes the mark of multiple calculate nodes where the data acquisition system；
According to the storage relation table, selection is stored with M calculate node of the data acquisition system from block catenary system.
Optionally, the processing module is also used to the training knot to the corresponding data acquisition system of the P calculate node
Fruit is analyzed, however, it is determined that there are incredible first calculate nodes of training result for the P calculate node, then to block linkwork
System sends the first information, and the first information is used to indicate the block catenary system to the calculated result of first calculate node
Confidence level is updated.
Optionally, the processing module is also used to the first distribution of earnings strategy being sent respectively to the K calculate node,
The first distribution of earnings strategy includes the shortest getable income of P calculate node institute of duration needed for training pattern.
The embodiment of the present application provides a kind of calculate node, which includes:
Transceiver module, the algorithm for training pattern for receiving client device transmission and the number for training pattern
According to the mark of set；
It obtains module and obtains the data acquisition system of storage for the mark according to the data acquisition system；
Processing module, for using in the data acquisition system data and the algorithm model is trained, and
The instruction information for obtaining training result is sent to the client device；Data in the data acquisition system are to pass through lower support grid
What the public data on network obtained.
In above-described embodiment of the application, by obtaining the mark of the data acquisition system for training pattern, client device
M calculate node for being stored with data acquisition system can be selected from block catenary system, and according to the first attribute of M calculate node
Information selects K calculate node from M calculate node, and then the mark of algorithm and data acquisition system is sent to K calculating section
Point；Correspondingly, each of K calculate node calculate node can be used data in data acquisition system and algorithm to model into
Row training, and the instruction information for successfully obtaining training result is sent to client device, so that the available K of client device
The training result of the corresponding data acquisition system of P calculate node in a calculate node；Further, client device can basis
The training result of data acquisition system, obtains model parameter.In the embodiment of the present application, by using the calculate node in block catenary system
Training pattern may not need the exclusive machine learning algorithm group of building and reduce the waste of resource so as to save cost；And
Algorithm is run by the calculate node with stronger storage capacity and processing capacity in selection block catenary system, it is ensured that algorithm
Processing speed, improve the training effectiveness of model, and then obtain preferable model training effect；Since block catenary system can be deposited
The online stability of each calculate node is stored up and updates, averaged historical handles the letter such as calculated result confidence level of time and node
Breath, so as to avoid client device and/or calculate node from faking during model training, so that model training
Process become open, transparent, and then can effectively guarantee the right of user.In addition, being used for the data acquisition system of training pattern
It can be public data, carry out model training by using the calculate node in block catenary system, may not need client device
Downloading data set dramatically saves the cost of model training.
Detailed description of the invention
In order to more clearly explain the technical solutions in the embodiments of the present application, make required in being described below to embodiment
Attached drawing is briefly introduced, it should be apparent that, the drawings in the following description are only some examples of the present application, for this
For the those of ordinary skill in field, without any creative labor, it can also be obtained according to these attached drawings
His attached drawing.
Fig. 1 is a kind of system architecture schematic diagram that the embodiment of the present application is applicable in；
Fig. 2 is a kind of corresponding flow diagram of method for obtaining model parameter in the embodiment of the present application；
Fig. 3 is a kind of structural schematic diagram of client device provided by the embodiments of the present application；
Fig. 4 is a kind of structural schematic diagram of calculate node provided by the embodiments of the present application.
Specific embodiment
In order to keep the purposes, technical schemes and advantages of the application clearer, below in conjunction with attached drawing to the application make into
It is described in detail to one step, it is clear that described embodiments are only a part of embodiments of the present application, rather than whole implementation
Example.Based on the embodiment in the application, obtained by those of ordinary skill in the art without making creative efforts
All other embodiment, shall fall in the protection scope of this application.
Fig. 1 is a kind of system architecture schematic diagram that the embodiment of the present application is applicable in, as shown in Figure 1, can be in the system architecture
Including one or more calculate nodes in block chain network (than calculate node 101, the calculate node gone out as schematically shown in Figure 1
102, calculate node 103 and calculate node 104) and client device 200.Wherein, one or more meters in block chain network
Operator node can safeguard block chain network jointly.Client device 200 can by access network in block chain network
One or more calculate nodes are communicated.
In the embodiment of the present application, block chain network can be point-to-point (the Peer To being made of multiple calculate nodes
Peer, P2P) network.P2P is that one kind operates in transmission control protocol (Transmission Control Protocol, TCP)
Application layer protocol on agreement, the calculate node in block chain network can be reciprocity each other, and middle scheming is not present in network
Operator node, therefore each calculate node can randomly connect other calculate nodes.
In specific implementation, the calculate node in block chain network can have multiple functions, for example, routing function, transaction
Function, block chain function and common recognition function etc..Specifically, the calculate node in block chain network can be other calculate nodes
The information such as the transaction data sent send more calculate nodes to realize the communication between calculate node；Alternatively, area
Calculate node in block chain network can be used for that user is supported to trade；Alternatively, calculate node in block chain network can be with
All Activity on log history；Alternatively, the calculate node in block chain network can be by verifying and recording transaction life
At the new block in block chain.In practical application, routing function is that each calculate node in block chain network must have
Function, and other functions can be configured according to actual needs by those skilled in the art.
In the embodiment of the present application, a calculate node in block chain network can on a physical machine (server),
And a calculate node can specifically refer to a series of process or processes run in server.For example, 1 net of block chain
Calculate node 101 in network can be the process run on a server.
It should be noted that calculate node described herein can refer to the server where calculate node.
Based on system architecture illustrated in Figure 1, Fig. 2 is a kind of method for obtaining model parameter provided by the embodiments of the present application
Corresponding flow diagram, this method comprises:
Step 201, client device obtains the mark of the algorithm for training pattern and the data acquisition system for training pattern
Know.
Herein, for may include public data in the data acquisition system of training pattern, public data refers to public on network
The data (for example, can be video data, image data or voice data etc.) opened, it can by way of searching for network
To obtain.In the embodiment of the present application, the algorithm for training pattern may include the program in machine code that user writes, or can also be with
The algorithm (for example user is downloaded by network) got by other means for user, the embodiment of the present application does not limit this
It is fixed.
In specific implementation, client device obtain for training pattern algorithm mode can there are many.Show at one
In example, algorithm can be stored in advance in the hard disk of client device (or being stored in internal storage), in this way,
Client device can be directly obtained algorithm from hard disk；In yet another example, client device can send to equipment a and use
In the request message of request algorithm, and the response message that receiving device a is returned, it include algorithm in the response message, in this way, client
End equipment message can get algorithm according to response.The embodiment of the present application does not limit this.
Step 202, the client device selects to be stored with from block catenary system according to the mark of the data acquisition system
M calculate node of the data acquisition system.
In the embodiment of the present application, data acquisition system, client can store in multiple calculate nodes of block catenary system
Equipment select to be stored with from block catenary system M calculate node of data acquisition system mode can there are many, in a kind of possibility
Implementation in, storage relation table can be set in block catenary system, which, which can serve to indicate that, is stored with
Multiple calculate nodes of data acquisition system.In one example, as shown in table 1, storing in relation table may include data acquisition system
Mark, the mark of the type of data acquisition system and multiple calculate nodes where data acquisition system.
A kind of table 1: storage relation table signal of block catenary system
The storage relation table illustrated according to table 1, the type of data acquisition system can be Image, the mark of data acquisition system
It can be Test, and the calculate node that data acquisition system Test is stored in block catenary system has 20, respectively calculate node J1,
Calculate node J2 ..., calculate node J20.
In the embodiment of the present application, client device can after getting the mark for the data acquisition system of training pattern,
According to the mark of data acquisition system, storage relation table corresponding with the mark of data acquisition system, Jin Ergen are obtained from block catenary system
According to storage relation table, the M calculate node that selection is stored with data acquisition system forms the first alternative calculate node set.For example, base
In the storage relation table that table 1 is illustrated, the alternative calculate node set of the first of data acquisition system Test can for calculate node J1,
Calculate node J2 ... ..., calculate node J20 }.
In the embodiment of the present application, the information of each calculate node can also be stored in block catenary system, for example calculate section
The online stability of point, calculated performance, the calculated result for handling handling capacity, the history average handling time of data, calculate node
Confidence level calculates the information such as cost.And block catenary system can also be nervous according to the intensity of traffic and resource of calculate node
Degree etc. is updated the multiple information of calculate node.Further, client device can be by handing over block catenary system
Mutually, the information of each calculate node stored in block catenary system is obtained.
Step 203, the client device is calculated according to the first attribute information of the M calculate node from described M
K calculate node is selected in node.
Herein, client device can be according to the first attribute information of calculate node each in M calculate node, from M
K calculate node, the calculate node to be trained as the first data subset are selected in calculate node.Wherein, in M calculate node
First attribute information of each calculate node may include the online stability of calculate node, calculated performance, calculate cost, calculates
It is one or more in the calculated result confidence level of node.Optionally, the first attribute information of each calculate node can also wrap
Include the history average computation processing time of processing handling capacity and data.
In the embodiment of the present application, it is more that client device selects the mode of K calculate node that may have from M calculate node
Kind, for example, first attribute information of each calculate node may include the calculating cost of calculate node in example 1, in this way,
Client device can select K according to the calculating cost of calculate node each in M calculate node from M calculate node
Calculate node.In example 2, the first attribute information of each calculate node may include the online stability of calculate node, meter
The calculated result for calculating performance, calculating cost, processing handling capacity, the history average computation of data processing time and calculate node is credible
Degree, in this way, client device can online stability, calculated performance, calculating according to calculate node each in M calculate node
Cost, processing handling capacity, the calculated result confidence level of the history average computation of data processing time and calculate node, are counted from M
K calculate node is selected in operator node.It should be noted that the first attribute of each calculate node is believed in the embodiment of the present application
Breath may include that online stability, calculated performance, calculating cost, processing handling capacity, the history of data of calculate node are averagely counted
One or more in the calculated result confidence level of calculation processing time and calculate node, the embodiment of the present application does not limit this.
By taking above-mentioned example 2 as an example, in one possible implementation, client device can preset a highest
Calculating cost (such as highest calculate excitation value be 100), which can serve to indicate that client device can be distributed
To the highest excitation value for the calculate node for using data acquisition system training pattern.Specifically, the available M calculating of client device
The calculating cost of each calculate node in node, and the calculating cost of each calculate node is compared with 100, select M
Calculate node of the carrying cost no more than 100 forms the second alternative calculate node set in calculate node.For example, it is based on table
The 1 storage relation table illustrated, it is assumed that deposited in 20 calculate nodes of the alternative calculate node set of the first of data acquisition system Test
4 calculate nodes (for example, calculate node J4, calculate node J9, calculate node J14, calculate node J19) calculating cost not
Greater than 100, then can there are calculate node J4, calculate node J9, calculate node J14 in the second alternative calculate node set, calculate
This 4 calculate nodes of node J19.It should be noted that if the calculating cost of 20 calculate nodes is all larger than client device and sets
The highest calculating cost set, then client device can wait the calculating cost of some calculate node to drop to client and set
The highest calculating cost or client device of standby setting can be fed directly to the information of user model failure to train.
It should be noted that excitation value can be a kind of form of expression of cluster excitation, specifically, it can be client
End equipment expenditure and can be used for rewarding data subset that calculate node sends in storage client device (or can also be with
By data acquisition system) and/or the cost paid when carrying out model training using data subset.In the embodiment of the present application, excitation value
It can be all members of cluster (herein, i.e., multiple calculate nodes in block catenary system and one or more client devices) institute
Generally acknowledged internet or real-life valuable object, or may be internet recognized by all members of cluster
Or real-life universal equivalent, the embodiment of the present application are not especially limited this.
Further, client device can also obtain second and alternatively calculate section by interacting with block catenary system
The online stability of each calculate node, calculated performance, calculating cost, processing handling capacity, the history of data are average in point set
The information such as the calculated result confidence level of calculating treatmenting time and algorithm, and be data acquisition system choosing by preset algorithm and pre-set level
It selects K calculate node and carries out model training, wherein K is more than or equal to 2.
Step 204, the mark of the algorithm and the data acquisition system is sent to the K calculating by the client device
Node so that the K calculate node respectively using in the data acquisition system data and the algorithm model is trained.
In the embodiment of the present application, after selecting K calculate node for data acquisition system, client device can will be used to train
The algorithm of model and the mark of data acquisition system are sent respectively to K calculate node.It is directed to any of K calculate node meter
Operator node (for example, calculate node J14) calculates after receiving the mark of algorithm and data acquisition system of client device transmission
The data acquisition system corresponding with the mark of data acquisition system Test saved in the available calculate node J14 of node J14.
It should be noted that the mark of algorithm and data acquisition system can be with transmitted in parallel to different in the embodiment of the present application
Calculate node, that is to say, that K calculating section can be sent to simultaneously for the algorithm of training pattern and the mark of data acquisition system
Point, in this way, the efficiency of data transmission can be promoted, save the time.
In the embodiment of the present application, client device is same the mark to K calculate node transmission algorithm and data acquisition system
When, the first distribution of earnings strategy of P calculate node in K calculate node can also be sent, wherein the first distribution of earnings strategy
It may include the shortest getable reward of P calculate node institute of duration needed for training pattern.In one example, client is set
The standby corresponding total excitation value of pre-set first distribution of earnings strategy is 500, and determines and carry out model instruction using data acquisition system
First 2 obtain the calculating of training result in experienced 3 calculate nodes (calculate node J4, calculate node J14, calculate node J19)
The excitation value that node can obtain is respectively 300,200, then client device can will be used for algorithm, the data set of training pattern
The mark of conjunction and the first distribution of earnings strategy are sent to this 3 calculate nodes jointly.Correspondingly, it is directed in 3 calculate nodes
Any one calculate node (for example, calculate node J14), after the data acquisition system saved in obtaining calculate node J14, calculate section
Data in data acquisition system can be used in point J14 and algorithm is trained model, obtain training result, and then by training result
It is recorded in block catenary system；Meanwhile the instruction information for obtaining training result can be sent to client and set by calculate node J14
It is standby.Further, client device can in receiving 3 calculate nodes preceding 2 calculate nodes (for example, receiving instruction
The sequence of information is successively are as follows: calculate node J14, calculate node J4) the instruction information that sends respectively, and in block catenary system net
It can be that the excitation value of calculate node J14 reward is 300, is calculating after the training result for getting calculate node write-in on network
The excitation value of node J4 reward is 200.
It should be noted that P calculate node using in data acquisition system data and algorithm model is trained
In the process, block catenary system can recorde each calculate node in P calculate node and be trained to obtain the time of training result,
And the indexs such as history average computation processing time for updating the calculated performance of each calculate node, processing handling capacity and data.
Step 205, the client device obtains the corresponding number of P calculate node in the K calculate node
According to the training result of set, and according to the training result of the data acquisition system, model parameter is obtained.
In the embodiment of the present application, client device can be analyzed the training result of P calculate node, however, it is determined that P
There are incredible first calculate nodes of training result for a calculate node, then can send the first information to block catenary system.Its
In, the first information can serve to indicate that block catenary system is updated the calculated result confidence level of the first calculate node.One
In a example, however, it is determined that within a preset range, then client is set the error range of the corresponding P training result of P calculate node
It is believable for that can determine P training result；If it is determined that there is the instruction of one or more calculate nodes in P training result
Practice result and the error of other training results is larger, then client device can be by the training including one or more calculate nodes
As a result the wrong first information is sent to block catenary system.Correspondingly, block catenary system is receiving the first of client transmission
After information, the history training result of one or more calculate nodes can be inquired, multiple history training result is wrong if it exists, i.e.,
Thinking one or more calculate nodes, there are imitation behaviors during training, then client device can reduce by one or more
The calculated result confidence level of a calculate node.If the calculated result confidence level of some calculate node is reduced to preset threshold,
The calculate node will not be actively supplied to client device again and the service such as be calculated or stored.
Further, obtain model parameter mode can there are many, in one possible implementation, client is set
It is standby that a most suitable training result can be chosen from P training result of data acquisition system, obtain the parameter of model.Another
In the possible implementation of kind, client device can also carry out integration processing to P training result of data acquisition system, obtain mould
The parameter of type.In other possible embodiments, the parameter of model is also possible to instructions any number of in 0~P training result
Practice result and carry out what integration was handled, the embodiment of the present application is not especially limited this.
In above-described embodiment of the application, by obtaining the mark of the data acquisition system for training pattern, client device
M calculate node for being stored with data acquisition system can be selected from block catenary system, and according to the first attribute of M calculate node
Information selects K calculate node from M calculate node, and then the mark of algorithm and data acquisition system is sent to K calculating section
Point；Correspondingly, each of K calculate node calculate node can be used data in data acquisition system and algorithm to model into
Row training, and the instruction information for successfully obtaining training result is sent to client device, so that the available K of client device
The training result of the corresponding data acquisition system of P calculate node in a calculate node；Further, client device can basis
The training result of data acquisition system, obtains model parameter.In the embodiment of the present application, by using the calculate node in block catenary system
Training pattern may not need the exclusive machine learning algorithm group of building and reduce the waste of resource so as to save cost；And
Algorithm is run by the calculate node with stronger storage capacity and processing capacity in selection block catenary system, it is ensured that algorithm
Processing speed, improve the training effectiveness of model, and then obtain preferable model training effect；Since block catenary system can be deposited
The online stability of each calculate node is stored up and updates, averaged historical handles the letter such as calculated result confidence level of time and node
Breath, so as to avoid client device and/or calculate node from faking during model training, so that model training
Process become open, transparent, and then can effectively guarantee the right of user.In addition, being used for the data acquisition system of training pattern
It can be public data, carry out model training by using the calculate node in block catenary system, may not need client device
Downloading data set dramatically saves the cost of model training.
For above method process, the embodiment of the present application also provides a kind of client device, the client device it is specific
Content is referred to above method implementation.
Fig. 3 is a kind of structural schematic diagram of client device provided by the embodiments of the present application, comprising:
Module 301 is obtained, is used for the mark of the data acquisition system of training pattern and for the algorithm of training pattern for obtaining；
Processing module 302, for the mark according to the data acquisition system, selection is stored with the number from block catenary system
According to M calculate node of set；
According to the first attribute information of the M calculate node, K calculate node is selected from the M calculate node；
The first attribute information of each calculate node includes the online stability of the calculate node, calculating in the M calculate node
Performance calculates cost, is in the calculated result confidence level of the calculate node one or more；
The mark of the algorithm and the data acquisition system is sent to the K calculate node, so that the K calculating saves
Point respectively using in the data acquisition system data and the algorithm model is trained；
The acquisition module 301 is also used to obtain the corresponding number of P calculate node in the K calculate node
According to the training result of set, and according to the training result of the data acquisition system, model parameter is obtained；
Wherein, M >=K >=P, P >=2, M, K and P are integer.
Optionally, the processing module 302, is specifically used for:
According to the mark of the data acquisition system, obtained from the block catenary system corresponding with the mark of the data acquisition system
Storage relation table, the storage relation table includes the mark of multiple calculate nodes where the data acquisition system；
According to the storage relation table, selection is stored with M calculate node of the data acquisition system from block catenary system.
Optionally, the processing module 302 is also used to the instruction to the corresponding data acquisition system of the P calculate node
Practice result to be analyzed, however, it is determined that there are incredible first calculate nodes of training result for the P calculate node, then to block
Catenary system sends the first information, and the first information is used to indicate calculating of the block catenary system to first calculate node
Result credibility is updated.
Optionally, the processing module 302 is also used to for the first distribution of earnings strategy being sent respectively to the K calculating section
Point, the first distribution of earnings strategy include the shortest getable income of P calculate node institute of duration needed for training pattern.
Fig. 4 is a kind of structural schematic diagram of calculate node provided by the embodiments of the present application, comprising:
Transceiver module 401, for receive client device transmission for training pattern algorithm and be used for training pattern
Data acquisition system mark；
It obtains module 402 and obtains the data acquisition system of storage for the mark according to the data acquisition system；
Processing module 403, for using in the data acquisition system data and the algorithm model is trained,
And the instruction information for obtaining training result is sent to the client device；Data in the data acquisition system are to pass through downloading
What the public data on network obtained.
It can be seen from the above: in above-described embodiment of the application, the data set of training pattern is used for by obtaining
The mark of conjunction, client device can select M calculate node for being stored with data acquisition system from block catenary system, and according to M
First attribute information of a calculate node selects K calculate node from M calculate node, and then by algorithm and data acquisition system
Mark be sent to K calculate node；Correspondingly, data acquisition system can be used in each of K calculate node calculate node
In data and algorithm model is trained, and the instruction information for successfully obtaining training result is sent to client device,
So that the training result of the corresponding data acquisition system of P calculate node in the available K calculate node of client device；Into one
Step ground, client device can obtain model parameter according to the training result of data acquisition system.In the embodiment of the present application, by adopting
With the calculate node training pattern in block catenary system, it may not need and construct exclusive machine learning algorithm group, so as to save
The waste of resource is reduced in cost-saving；And pass through the calculating with stronger storage capacity and processing capacity in selection block catenary system
Node runs algorithm, it is ensured that the processing speed of algorithm improves the training effectiveness of model, and then obtains preferable model training
Effect；Since block catenary system can store and update the online stability of each calculate node, averaged historical handle the time and
The information such as the calculated result confidence level of node, so as to avoid client device and/or calculate node in the mistake of model training
It fakes in journey, so that the process of model training becomes open, transparent, and then can effectively guarantee the right of user.This
Outside, it can be public data for the data acquisition system of training pattern, carry out mould by using the calculate node in block catenary system
Type training may not need client device downloading data set, dramatically save the cost of model training.
It should be understood by those skilled in the art that, embodiments herein can provide as method or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application
Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although the preferred embodiment of the application has been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the application range.
Obviously, those skilled in the art can carry out various modification and variations without departing from the essence of the application to the application
Mind and range.In this way, if these modifications and variations of the application belong to the range of the claim of this application and its equivalent technologies
Within, then the application is also intended to include these modifications and variations.
Claims (10)
1. a kind of method for obtaining model parameter, which is characterized in that this method comprises:
Client device obtains the mark of the data acquisition system for training pattern and the algorithm for training pattern；
The client device is according to the mark of the data acquisition system, and selection is stored with the data acquisition system from block catenary system
M calculate node；
The client device selects K from the M calculate node according to the first attribute information of the M calculate node
A calculate node；The first attribute information of each calculate node includes the online of the calculate node in the M calculate node
Stability, calculated performance calculate cost, are in the calculated result confidence level of the calculate node one or more；
The mark of the algorithm and the data acquisition system is sent to the K calculate node by the client device, so that institute
State K calculate node respectively using in the data acquisition system data and the algorithm model is trained；
The client device obtains the training of the corresponding data acquisition system of P calculate node in the K calculate node
As a result, and according to the training result of the data acquisition system, obtain model parameter；
Wherein, M >=K >=P, P >=2, M, K and P are integer.
2. the method according to claim 1, wherein the client device selects storage from block catenary system
There is M calculate node of the data acquisition system, comprising:
The client device obtains and the data acquisition system from the block catenary system according to the mark of the data acquisition system
The corresponding storage relation table of mark, the storage relation table includes the mark of multiple calculate nodes where the data acquisition system
Know；
The client device is according to the storage relation table, and selection is stored with the M of the data acquisition system from block catenary system
A calculate node.
3. the method according to claim 1, wherein the client device gets the K calculate node
In the corresponding data acquisition system of P calculate node training result after, further includes:
The client device analyzes the training result of the corresponding data acquisition system of the P calculate node, if really
There are incredible first calculate nodes of training result for the fixed P calculate node, then send the first information to block catenary system,
The first information is used to indicate the block catenary system and is updated to the calculated result confidence level of first calculate node.
4. according to the method in any one of claims 1 to 3, which is characterized in that the method also includes:
First distribution of earnings strategy is sent respectively to the K calculate node, first income point by the client device
It include the shortest getable income of P calculate node institute of duration needed for training pattern with strategy.
5. a kind of method for obtaining model parameter, which is characterized in that the described method includes:
Calculate node in block catenary system receive that client device sends for the algorithm of training pattern and for training mould
The mark of the data acquisition system of type；
The calculate node obtains the data acquisition system of storage, uses the data set according to the mark of the data acquisition system
Data and the algorithm in conjunction are trained the model, and the instruction information for obtaining training result is sent to the visitor
Family end equipment；Data in the data acquisition system are to be obtained by the public data on downloading network.
6. a kind of client device, which is characterized in that the client device includes:
Module is obtained, is used for the mark of the data acquisition system of training pattern and for the algorithm of training pattern for obtaining；
Processing module, for the mark according to the data acquisition system, selection is stored with the data acquisition system from block catenary system
M calculate node；
According to the first attribute information of the M calculate node, K calculate node is selected from the M calculate node；It is described
In M calculate node the first attribute information of each calculate node include the online stability of the calculate node, calculated performance,
It is one or more in calculating cost, the calculated result confidence level of the calculate node；
The mark of the algorithm and the data acquisition system is sent to the K calculate node, so that the K calculate node point
Not using in the data acquisition system data and the algorithm model is trained；
The acquisition module is also used to obtain the corresponding data acquisition system of P calculate node in the K calculate node
Training result, and according to the training result of the data acquisition system, obtain model parameter；
Wherein, M >=K >=P, P >=2, M, K and P are integer.
7. client device according to claim 6, which is characterized in that the processing module is specifically used for:
According to the mark of the data acquisition system, deposit corresponding with the mark of the data acquisition system is obtained from the block catenary system
Relation table is stored up, the storage relation table includes the mark of multiple calculate nodes where the data acquisition system；
According to the storage relation table, selection is stored with M calculate node of the data acquisition system from block catenary system.
8. client device according to claim 6, which is characterized in that the processing module is also used to:
The training result of the corresponding data acquisition system of the P calculate node is analyzed, however, it is determined that the P calculating section
There are incredible first calculate nodes of training result for point, then send the first information to block catenary system, and the first information is used
The calculated result confidence level of first calculate node is updated in the instruction block catenary system.
9. the client device according to any one of claim 6 to 8, which is characterized in that the processing module is also used to:
First distribution of earnings strategy is sent respectively to the K calculate node, the first distribution of earnings strategy includes training
Model takes the getable income of long shortest P calculate node institute.
10. a kind of calculate node, which is characterized in that the calculate node includes:
Transceiver module, the algorithm for training pattern for receiving client device transmission and the data set for training pattern
The mark of conjunction；
It obtains module and obtains the data acquisition system of storage for the mark according to the data acquisition system；
Processing module, for using in the data acquisition system data and the algorithm model is trained, and will
Instruction information to training result is sent to the client device；Data in the data acquisition system are by downloading network
Public data obtain.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811313450.0A CN109543725A (en) | 2018-11-06 | 2018-11-06 | A kind of method and device obtaining model parameter |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811313450.0A CN109543725A (en) | 2018-11-06 | 2018-11-06 | A kind of method and device obtaining model parameter |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109543725A true CN109543725A (en) | 2019-03-29 |
Family
ID=65844426
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811313450.0A Pending CN109543725A (en) | 2018-11-06 | 2018-11-06 | A kind of method and device obtaining model parameter |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109543725A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110619317A (en) * | 2019-09-26 | 2019-12-27 | 联想(北京)有限公司 | Model training method, model training device and electronic equipment |
| CN116402275A (en) * | 2023-03-03 | 2023-07-07 | 河海大学 | Unmanned carrier dynamic selection method for intelligent cooperative inspection of dam |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101719930A (en) * | 2009-11-27 | 2010-06-02 | 南京邮电大学 | Cloud money-based hierarchical cloud computing system excitation method |
| CN103345572A (en) * | 2013-06-13 | 2013-10-09 | 东北大学 | Node trust value assessment method for volunteering computing environment |
| CN103746957A (en) * | 2013-10-10 | 2014-04-23 | 安徽师范大学 | Trust evaluation system based on privacy protection and construction method thereof |
| CN106815644A (en) * | 2017-01-26 | 2017-06-09 | 北京航空航天大学 | Machine learning method and from node |
| CN107864198A (en) * | 2017-11-07 | 2018-03-30 | 济南浪潮高新科技投资发展有限公司 | A kind of block chain common recognition method based on deep learning training mission |
| CN108123936A (en) * | 2017-12-13 | 2018-06-05 | 北京科技大学 | A kind of access control method and system based on block chain technology |
| CN108323200A (en) * | 2018-01-25 | 2018-07-24 | 深圳前海达闼云端智能科技有限公司 | Data training method and device based on block chain, storage medium and block chain link points |
| US20180253539A1 (en) * | 2017-03-05 | 2018-09-06 | Ronald H. Minter | Robust system and method of authenticating a client in non-face-to-face online interactions based on a combination of live biometrics, biographical data, blockchain transactions and signed digital certificates. |
| CN108664222A (en) * | 2018-05-11 | 2018-10-16 | 北京奇虎科技有限公司 | A kind of block catenary system and its application process |
-
2018
- 2018-11-06 CN CN201811313450.0A patent/CN109543725A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101719930A (en) * | 2009-11-27 | 2010-06-02 | 南京邮电大学 | Cloud money-based hierarchical cloud computing system excitation method |
| CN103345572A (en) * | 2013-06-13 | 2013-10-09 | 东北大学 | Node trust value assessment method for volunteering computing environment |
| CN103746957A (en) * | 2013-10-10 | 2014-04-23 | 安徽师范大学 | Trust evaluation system based on privacy protection and construction method thereof |
| CN106815644A (en) * | 2017-01-26 | 2017-06-09 | 北京航空航天大学 | Machine learning method and from node |
| US20180253539A1 (en) * | 2017-03-05 | 2018-09-06 | Ronald H. Minter | Robust system and method of authenticating a client in non-face-to-face online interactions based on a combination of live biometrics, biographical data, blockchain transactions and signed digital certificates. |
| CN107864198A (en) * | 2017-11-07 | 2018-03-30 | 济南浪潮高新科技投资发展有限公司 | A kind of block chain common recognition method based on deep learning training mission |
| CN108123936A (en) * | 2017-12-13 | 2018-06-05 | 北京科技大学 | A kind of access control method and system based on block chain technology |
| CN108323200A (en) * | 2018-01-25 | 2018-07-24 | 深圳前海达闼云端智能科技有限公司 | Data training method and device based on block chain, storage medium and block chain link points |
| CN108664222A (en) * | 2018-05-11 | 2018-10-16 | 北京奇虎科技有限公司 | A kind of block catenary system and its application process |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110619317A (en) * | 2019-09-26 | 2019-12-27 | 联想(北京)有限公司 | Model training method, model training device and electronic equipment |
| CN116402275A (en) * | 2023-03-03 | 2023-07-07 | 河海大学 | Unmanned carrier dynamic selection method for intelligent cooperative inspection of dam |
| CN116402275B (en) * | 2023-03-03 | 2023-12-15 | 河海大学 | Unmanned carrier dynamic selection method for intelligent cooperative inspection of dam |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109543726A (en) | A kind of method and device of training pattern | |
| US11291917B2 (en) | Artificial intelligence (AI) model training using cloud gaming network | |
| US9787745B1 (en) | Content delivery | |
| US10737179B2 (en) | Predictive recommendations for skills development | |
| US20180240062A1 (en) | Collaborative algorithm development, deployment, and tuning platform | |
| CN110352584A (en) | Across the automatic priority ranking of the equipment flow of local network | |
| US20070094325A1 (en) | Hybrid peer-to-peer data communication and management | |
| CN107209872A (en) | The distributed training of reinforcement learning system | |
| CN102075352A (en) | Method and device for predicting network user behavior | |
| CN108920948A (en) | A kind of anti-fraud streaming computing device and method | |
| Vuculescu et al. | How to solve problems with crowds: A computer‐based simulation model | |
| US11551119B2 (en) | Machine learning based ranking of private distributed data, models and compute resources | |
| WO2021007006A1 (en) | Dual machine learning pipelines for transforming data and optimizing data transformation | |
| CN109543725A (en) | A kind of method and device obtaining model parameter | |
| CN114745392A (en) | Flow scheduling method | |
| Nguyen et al. | An improved sea lion optimization for workload elasticity prediction with neural networks | |
| Nguyen et al. | A novel nature-inspired algorithm for optimal task scheduling in fog-cloud blockchain system | |
| CN109558950A (en) | A kind of method and device of determining model parameter | |
| CN113543160B (en) | 5G slice resource allocation method, device, computing equipment and computer storage medium | |
| Dubey et al. | Bayesian network based trust model with time window for pure P2P computing systems | |
| Zeng et al. | Approximating behavioral equivalence of models using top-k policy paths. | |
| Hegedus et al. | Gossip-based learning under drifting concepts in fully distributed networks | |
| Li et al. | Search experience-based search adaptation in artificial bee colony algorithm | |
| CN109558894A (en) | A kind of method and client device of determining model parameter | |
| Gilbert | The role of artificial intelligence for network automation and security |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190329 |