CN104598425A - General multiprocessor parallel calculation method and system - Google Patents
General multiprocessor parallel calculation method and system Download PDFInfo
- Publication number
- CN104598425A CN104598425A CN201310530649.XA CN201310530649A CN104598425A CN 104598425 A CN104598425 A CN 104598425A CN 201310530649 A CN201310530649 A CN 201310530649A CN 104598425 A CN104598425 A CN 104598425A
- Authority
- CN
- China
- Prior art keywords
- computing node
- calculation
- task
- node
- calculation task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Multi Processors (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a general multiprocessor parallel calculation method and system. The method comprises the steps that a calculation task is obtained, hardware information and software information of all calculation nodes of a calculation resource are obtained, and the processing capacity of each the calculation node is calculated according to the hardware information and software information; according to the internal parallelizability of the calculation task, the calculation task is decomposed into multiple sub calculation tasks; according to the processing capacity of each calculation node, the sub calculation tasks are matched with all the nodes; according to a basic communication protocol, the data format of the sub calculation tasks is converted to a protocol format, and according to the matching relation, the sub calculation tasks are distributed to all the calculation nodes; according to the basic communication protocol, calculation results, of the protocol format, fed back by the calculation nodes are obtained, the data format of the calculation results is converted to the format of the original calculation task from the protocol format, and then the calculation results are fed back. Multiprocessor parallel calculation under the heterogeneous environment is achieved.
Description
Technical field
The present invention relates to the communication technical field under isomerous environment, particularly the general multiprocessing parallel calculation method and system of one.
Background technology
Along with progress and the development of geophysical prospecting technology equipment, the changes in demand that domestic and international earthquake-capturing software is also faced with big data quantity process, high-efficient high performance calculates, as three-dimensional illumination, tomographic statics, Wave equation forward modeling etc., calculated amount is large, length consuming time, conventional individual tupe far can not meet high-performance calculation needs, supports that big data quantity process, high-performance, efficient calculation are trends of the times.Current seismic acquisition configuration is but faced with complicated, isomery, that performance difference is huge computing environment, instead of the group system of the isomorphism based on costliness required for traditional parallel computation.Address this problem, need a kind of multiprocessing parallel calculation framework be adapted under isomery difference network computing environment, the carrying out intellectuality of parallel task is decomposed, scheduling, management and control, adapt to different parallel tasks.
Traditional physical prospecting parallel computation has had a lot of ripe solution, but for isomery difference computing environment, feature often based on different Parallel applications carries out parallel processing targetedly, meet and specifically apply needs, the general multiprocessing parallel calculation being applicable to isomery difference computing environment that still neither one passes through at present solves pattern and scheme.
For earthquake-capturing application system, calculation task mostly have inherent can concurrency, calculation task can resolve into mutual essentially independent calculating subtask, thus runs on different computational resources relatively independently.In earthquake-capturing field produces, in the network computing environment be made up of different computational resources, device hardware (comprises CPU multinuclear, the many core of CPU, GPU, network route and structure etc.), software environment difference is huge, computing power also differs greatly, simultaneously, its data processing method of different calculation tasks, calculate control procedure, computational accuracy, efficiency variance is also very large, be suitable for different calculation tasks, its core goes to solidify the key link in parallel computation with a kind of general computing architecture, solve the otherness of different parallel computation task in each link again neatly simultaneously.This is also the gordian technique of isomerous environment parallel computation frame, is also the key factor improving earthquake-capturing application software counting yield.
Summary of the invention
For solving the problems of the prior art, the application proposes a kind of general multiprocessing parallel calculation method and system, by basic communication protocol, format conversion is carried out to calculation task, to adapt to each calculation level in heterogeneous computing environment, thus realize adapting to physical prospecting acquisition applications feature and the general multiprocessing parallel calculation that can adapt to isomerous environment.
For achieving the above object, this application provides a kind of general multiprocessing parallel calculation method, comprising:
Obtain calculation task;
Obtain hardware information and the software information of each computing node of computational resource, and calculate the processing power of each computing node according to this hardware information and software information;
Can concurrency according to the inherence of described calculation task, this calculation task is decomposed into multiple calculating subtask;
According to the processing power of described each computing node, described multiple calculating subtask and described each node are matched;
By basic communication protocol, be protocol format by the Data Format Transform of described multiple calculating subtask, and according to matching relationship, be distributed to each computing node;
By described basic communication protocol, obtain the result of calculation of the protocol format that described each computing node returns;
The data layout of described result of calculation is converted to the form of former calculation task by protocol format and returns.
Optionally, described according to matching relationship, after being distributed to each computing node, also comprise:
By multiple calculating subtasks of described protocol format, be converted to the form of the computing node needs matched with it respectively;
Described each computing node performs calculation task respectively, and obtains result of calculation;
This result of calculation is converted to protocol format and returns.
Wherein, the described processing power according to described each computing node, matches described multiple calculating subtask and described each node and comprises:
Processing power according to described each computing node is screened, and obtains qualified computing node and participates in calculating;
Matched with described qualified computing node by described multiple calculating, the coupling between described calculating subtask and computing node comprises one to one or multi-to-multi;
Discharge ineligible computing node.
Optionally, each computing node described is heterogeneous nodes.
Optionally, the method also comprises: computing node performs during calculation task, when certain computing node taken the lead in distribution technical assignment after, be its Distribution Calculation task again;
Optionally, the method also comprises: computing node monitors the state of described each computing node during performing calculation task, when certain node occurs abnormal, the calculation task that this node is responsible for is reassigned to other node.
Invention additionally provides a kind of general multiprocessing parallel calculation system, comprising:
Task acquiring unit, for obtaining calculation task;
Computing power information acquisition unit, for obtaining hardware information and the software information of each computing node of computational resource, and calculates the processing power of each computing node according to this hardware information and software information;
Task-decomposing unit, for can concurrency according to the inherence of described calculation task, be decomposed into multiple calculating subtask by this calculation task;
Task matching unit, for the processing power according to described each computing node, matches described multiple calculating subtask and described each node;
Task Dispatching Unit, for by basic communication protocol, is protocol format by the Data Format Transform of described multiple calculating subtask, and according to matching relationship, is distributed to each computing node;
Result recovery unit, for by described basic communication protocol, obtains the result of calculation of the protocol format that described each computing node returns;
Format conversion unit, for being converted to the form of former calculation task by the data layout of described result of calculation by protocol format and returning.
Optionally, also comprise calculation server, specifically comprise:
Second format conversion unit, for the multiple calculating subtasks by described protocol format, is converted to the form of the computing node needs matched with it respectively;
Multiple computing node, for performing calculation task, and obtains result of calculation;
Feedback unit, for being converted to protocol format by this result of calculation and returning.
Wherein, task matching unit comprises:
Computing node screening subelement, for screening according to the processing power of described each computing node, obtaining qualified computing node and participating in calculating;
Coupling subelement, for being matched with described qualified computing node by described multiple calculating, the coupling between described calculating subtask and computing node comprises one to one or multi-to-multi;
Release subelement, for discharging ineligible computing node.
Optionally, each computing node described is heterogeneous nodes.
Optionally, also comprise:
Secondary distribution unit, performs during calculation task for computing node, when certain computing node taken the lead in distribution technical assignment after, be its Distribution Calculation task again.
Exception processing unit: for monitoring the state of described each node, when certain node occurs abnormal, is reassigned to other node by the calculation task that this node is responsible for.
The present invention can reach following beneficial effect: by obtaining calculation task; Obtain hardware information and the software information of each computing node of computational resource, and calculate the processing power of each computing node according to this hardware information and software information; Can concurrency according to the inherence of described calculation task, this calculation task is decomposed into multiple calculating subtask; According to the processing power of described each computing node, described multiple calculating subtask and described each node are matched; By basic communication protocol, be protocol format by the Data Format Transform of described multiple calculating subtask, and according to matching relationship, be distributed to each computing node; By described basic communication protocol, obtain the result of calculation of the protocol format that described each computing node returns; The data layout of described result of calculation is converted to the form of former calculation task by protocol format and returns, achieve the multiprocessing parallel calculation under isomerous environment, support the process of abnormality, guarantee, when individual equipment occurs abnormal, not affect completing of whole calculation task.
Accompanying drawing explanation
Fig. 1 is the process flow diagram of a kind of general multiprocessing parallel calculation method of the present invention;
Fig. 2 is the structural drawing of a kind of general multiprocessing parallel calculation system of the present invention.
Embodiment
Be described below by way of specific embodiment:
Embodiment one:
As shown in Figure 1, be the process flow diagram of a kind of general multiprocessing parallel calculation method of the present embodiment, comprise the following steps:
Step 101, obtains calculation task;
Step 102, obtains the ability parameter of each computing node of computational resource, and calculates the processing power of each computing node according to this ability parameter;
Physical prospecting gathers parallel computation and usually has intensive, and input data are little, the feature that result data is huge, and also there is some difference in dissimilar application, therefore defines an application type factor f
a, for be characterized in certain type application in input size of data d
i, calculated amount c
a, result data d
othe mutual relationship of size is an important reference items in Parallel Task Scheduling process.The initial application type factor calculates adopts with the following method:
F
a=F
a(c
a, d
o, d
i)=c
a/ d
i*ρ
a+ d
o/ d
i; Wherein, ρ
ait is coefficient.
After obtaining computational resource list and calculation task list, management node just can carry out task scheduling and management, according to the calculated amount size of calculation task and the arithmetic capability of computing node, carries out task matching.
In parallel system, the calculated amount of calculation task, input data size that is how many and result data all will respond the computing time of whole concurrent job, in order to the task amount of each calculation task of more authentic and valid reflection, we will calculate the task equivalent of each task, and are arranged according to task equivalent size by calculation task to be allocated.Task equivalent is tried to achieve by the application type factor that the Meta task of each calculation task is how many and initial.Task equivalent calculation formula is as follows:
e
n=f
a*c
n；
Because many-sided factors such as cpu performance, IO handling capacity, the network bandwidth are all by the processing time of RESPONSE CALCULATION node to calculation task, in order to reflect computing node more really in parallel framework to the processing power of task, we will calculate the weighting task processing power of each node, calculate the initial weighting task processing power of each computing node, try to achieve primarily of the computing power obtained, network transmission speed, initial weighting task processing power formula is as follows.After having calculated the weighting task processing power of each computing node, according to this result, computational resource list is sorted.
p
m=t
c+t
t
Wherein t
cfor computing node m is to the processing time of identical element task, t
tfor computing node m is to the transmission time of identical element task data.
Effective prior imformation is obtained less before calculating by concurrent job, so use minimum task precedence method to distribute when carrying out task matching first, namely from calculation task list, choose front M the calculation task that task equivalent is minimum, and distribute successively according to the weighting task processing power of each computing node.
Management node upgrades the respective handling ability parameter of computing node according to the situation that computing node is reported, as data rate, and the stand-by period etc. of data transmission, and calculate the weighted mean processing power of this computing node on this basis.
p
m=t
c+t
i+t
o
Wherein t
ifor the input data weighting transmission time of unit Meta task, t
ofor the output data weighting transmission time of unit Meta task.
Step 103, can concurrency according to the inherence of described calculation task, and this calculation task is decomposed into multiple calculating subtask;
A kind of task state of loose coupling is adopted concrete parallel computation application to be separated with the framework that walks abreast.Task division can as required by big gun, divide by road or by frequency etc., and the size of each calculation task is not fixed, and can comprise the calculation task of one or more yuan of granularity.The mode of task division is determined by concrete should being used for usually, parallel framework is only dispatched according to this task division and computational resource situation, target is exactly make each calculation task reasonably on the balanced each computing node be assigned on isomery lattice, so that the operating load of each computing node keeps relative equilibrium, and obtain preferably parallel processing performance, thus shorten the execution time of whole concurrent program.Also to reduce internodal exchanges data simultaneously, reduce data communication cost.
Step 104, by basic communication protocol, is assigned to described each computing node by described multiple calculating subtask according to the processing power of described each computing node; Described calculating subtask can be converted to described computing node and can identify and the form processed by this basic communication protocol;
Processing power according to described each computing node is screened, and obtains qualified computing node and participates in calculating; Matched in described multiple calculating subtask and described qualified computing node, the coupling between described calculating subtask and computing node comprises one to one or multi-to-multi; Discharge ineligible computing node; Optionally, between each computing node can be heterogeneous nodes.
Matched in multiple calculating subtask and computing node, undertaken by Task Assigned Policy, concrete Task Assigned Policy has: press and wait task amount, described multiple calculating subtask is assigned to described each computing node; By the computing velocity of computing node, described multiple calculating subtask is assigned to described each computing node; By the smallest synchronization time, described multiple calculating subtask is assigned to described each computing node; By max calculation efficiency, described multiple calculating subtask is assigned to described each computing node.Wherein, described allocation strategy comprises static state and allocates in advance and run duration dynamic assignment; Described run duration dynamic assignment is specially: after certain computing node completes the technical assignment of distribution, is its Distribution Calculation task again; Wherein, the distribution support of strategy is manually changed.Require that scheduling strategy can the change of adaptive strategy and adjustment, support customization, require that coding realizes changing minimum, even do not change, only describe the amendment of strategy, adjustment by corresponding configuration text and change.
This basic communication protocol is important base layer support technology of the present invention.Parallel computation under multi-machine surroundings must relate to message communicating between different computational resource and data are transmitted, and must follow certain communication specification.The present invention is based on existing mature technology, standard a nd norm basis, build a unification, open, efficient network service supervisory packet.As application layer protocol, energy high level overview and the most basic communication function, the operational order of specification within the scope of parallel computation frame, and the parameter format of standardization, can not be too complicated, must change by adaption demand again, change is controlled in less scope.
Therefore communication protocol comprises following three contents: communication functions, operational order and parameter format.
Communication message can be taken out from the content of communication protocol, comprise message packet and data message, communication message is made up of heading and header body, the parameter format of communication, here XML is adopted to be described, the semanteme transmitted based on data representation and the data of XML and the good state property feature of form, the form in simplified message communication represents and conversion operation.Be do not need to carry out Parameter analysis of electrochemical for parallel framework, design parameter serializing and unserializing operate is gone to define and complete by concrete Parallel application.Between two communication entities, the content of Internet Transmission is called communication message, can be divided into message packet and data message again according to content, and in general, message packet transmission quantity is little, and data message data volume is general comparatively large, and transmission reliability requires higher.
Communication functions has and sends function and receiver function, and comprise message with data transmits, wherein data transmission is generally based on file, and data volume is large, needs to consider big data quantity transmission, except performance is wanted soon, also to have breakpoint transmission and error handling processing machine-processed.
According to the feature of parallel framework, make 6 kinds of basic communication instructions.Broadcasting instructions: broadcast(overall situation broadcast), execute instruction: report(is point-to-point, passive report to broadcast), inquiry instruction: inquire (point-to-point inquiry can not be replied), response instruction: answer(is point-to-point, response to access), the point-to-point access of access instruction: visit(, imperative operation, require response), response instruction: response(is to the response of access).
The telecommunication management bag on basis is the support of parallel providing the foundation property of framework calculating.
Effective prior imformation is obtained less before calculating by concurrent job, so use minimum task precedence method to distribute when carrying out task matching first, namely from calculation task list, choose front M the calculation task that task equivalent is minimum, and distribute successively according to the weighting task processing power of each computing node.
Management node upgrades the respective handling ability parameter of computing node according to the situation that computing node is reported, as data rate, and the stand-by period etc. of data transmission, and calculate the weighted mean processing power of this computing node on this basis.
p
m=t
c+t
i+t
o；
Wherein t
ifor the input data weighting transmission time of unit Meta task, t
ofor the output data weighting transmission time of unit Meta task.
The disposition that management node is reported according to computing node upgrades the application type factor.
Wherein
for utilizing to obtain the application type factor that calculates of m computing node.
Management node distributes remaining task according to the current weighted mean processing power of each computing node, has made the shortest time of all residue required by task.
Concrete dispatching algorithm
for m node completes the time needed for the n-th calculation task.
for m node completes its n run
mthe T.T. that individual required by task is wanted.
t
tfor the minimum time required for overall concurrent job, it is also the target of Parallel Task Scheduling.
Owing to there is many uncertain factors in parallel framework, as the deadlock of computing node and adding of new node, all be according to the processing power of current each computing node remaining calculation task distributed when therefore dispatching at every turn, guaranteed the shortest time that residue task is altogether required.
After each computing node receives the subtask of distribution, each calculating subtask of this protocol format is converted into the form that target computing nodes can identify; Each computing node performs calculation task, obtains checkout result, and this result is changed into protocol format.
Computing node performs during calculation task, when certain computing node taken the lead in distribution technical assignment after, be its Distribution Calculation task again;
Computing node monitors the state of described each computing node during performing calculation task, when certain node occurs abnormal, the calculation task that this node is responsible for is reassigned to other node.After computing node abnormal restoring, can again participate in calculating.After user node is abnormal, operation is interrupted, and after user node recovers, operation continues.After management node is abnormal, operation is interrupted, and after recovery, continues to run whole operation from the state before exception.
Step 105, by described basic communication protocol, obtains the multiple sub-result of calculation that described each computing node returns;
Step 106, is undertaken integrating and returning by described multiple sub-result of calculation.
Embodiment two:
As shown in Figure 2, be the structural drawing of a kind of general multiprocessing parallel calculation system of the present embodiment, comprise:
Task acquiring unit 201, for obtaining calculation task;
Computing power information acquisition unit 202, for obtaining hardware information and the software information of each computing node of computational resource, and calculates the processing power of each computing node according to this hardware information and software information;
Task-decomposing unit 203, for can concurrency according to the inherence of described calculation task, be decomposed into multiple calculating subtask by this calculation task;
Task matching unit 204, for the processing power according to described each computing node, matches described multiple calculating subtask and described each node;
Computing node screening subelement 2041, for screening according to the processing power of described each computing node, obtaining qualified computing node and participating in calculating;
Coupling subelement 2042, for being matched with described qualified computing node by described multiple calculating, the coupling between described calculating subtask and computing node comprises one to one or multi-to-multi;
Release subelement 2043, for discharging ineligible computing node.
Task Dispatching Unit 205, for by basic communication protocol, is protocol format by the Data Format Transform of described multiple calculating subtask, and according to matching relationship, is distributed to each computing node;
Calculation server 206, specifically comprises:
Format conversion subelement 2061, for the multiple calculating subtasks by described protocol format, is converted to the form of the computing node needs matched with it respectively;
Computing node 2062, for performing calculation task, and obtains result of calculation;
This computing node has multiple, can be heterogeneous nodes or isomorphism node.
Feedback subelement 2063, for being converted to protocol format by this result of calculation and returning;
Result recovery unit 207, for by described basic communication protocol, obtains the result of calculation of the protocol format that described each computing node returns;
Format conversion unit 208, for being converted to the form of former calculation task by the data layout of described result of calculation by protocol format and returning.
Optionally, described protocol format is XML format.
Also comprise secondary distribution unit 209 and exception processing unit 210, be connected with described computing node 2062, secondary distribution unit 209 performs during calculation task for computing node, when certain computing node taken the lead in distribution technical assignment after, be its Distribution Calculation task again; The calculation task that this node is responsible for, for monitoring the state of described each node, when certain node occurs abnormal, is reassigned to other node by exception processing unit 210; Concrete, this Elementary Function can comprise the process after the state detecting each node, abnormal appearance and the recovery after exception.
Optionally, this exception processing unit 210 can also be used for monitoring management node and user node, and wherein, management node is used for the state of monitoring calculation node and user node.
When computing node occurs abnormal, the task that it runs by exception processing unit 210 re-starts distributes to other computing node, guarantees completing of whole calculation task; After computing node abnormal restoring, can again participate in calculating.
After user node is abnormal, operation is interrupted, and after user node recovers, operation continues.
After management node is abnormal, operation is interrupted, and after recovery, continues to run whole operation from the state before exception.
The present invention proposes a set of general parallel computation frame being suitable for heterogeneous computing environment, solve the parallel computation problem under isomery difference environment, provide the succinct solution of a unification, solve conventional parallel computation Problems existing, and frame stability, reliably, different to physical prospecting application has very strong adaptive faculty.
The present invention to based on process and parallel computation pattern based on thread both provide good support.For the process mode of independent operating, parallel framework is based on the communication protocol of bottom, and relative to the computation schema based on thread, except a little bigger, the volume of transmitted data multiple spot of expense in storage, mode of operation is on all four.
For serial program, when not revising any code, multiprocessing parallel calculation can be realized, based on parallel computation frame, making serial application possess computation capability.
For the multi-core parallel concurrent calculation procedure of unit, also when not revising any code, multiprocessing parallel calculation can be realized, promoting multi-core parallel concurrent counting yield, strengthen computation capability.
Different parallel tasks, the procedure definition provided by parallel framework and parameter configuration, can define different parallel tasks flexibly and easily efficiently, meets the almost main Parallel application demand of physical prospecting application.
Persons skilled in the art under this design philosophy do any not creative transformation, all should be considered as within protection scope of the present invention.
Claims (12)
1. a general multiprocessing parallel calculation method, is characterized in that, comprising:
Obtain calculation task;
Obtain hardware information and the software information of each computing node of computational resource, and calculate the processing power of each computing node according to this hardware information and software information;
Can concurrency according to the inherence of described calculation task, this calculation task is decomposed into multiple calculating subtask;
By basic communication protocol, described multiple calculating subtask is assigned to described each computing node according to the processing power of described each computing node; Described calculating subtask can be converted to described computing node and can identify and the form processed by this basic communication protocol;
By described basic communication protocol, obtain the result of calculation of the protocol format that described each computing node returns;
The data layout of described result of calculation is converted to the form of former calculation task by protocol format and returns.
2. the method for claim 1, is characterized in that, described according to matching relationship, after being distributed to each computing node, also comprises:
By multiple calculating subtasks of described protocol format, be converted to the form of the computing node needs matched with it respectively;
Described each computing node performs calculation task respectively, and obtains result of calculation;
This result of calculation is converted to protocol format and returns.
3. the method for claim 1, is characterized in that, the described processing power according to described each computing node, and being matched in described multiple calculating subtask and described each node comprises:
Processing power according to described each computing node is screened, and obtains qualified computing node and participates in calculating;
Matched in described multiple calculating subtask and described qualified computing node, the coupling between described calculating subtask and computing node comprises one to one or multi-to-multi;
Discharge ineligible computing node.
4. the method for claim 1, is characterized in that, each computing node described is heterogeneous nodes.
5. method as claimed in claim 2, is characterized in that, also comprise:
Computing node performs during calculation task, when certain computing node taken the lead in distribution technical assignment after, be its Distribution Calculation task again.
6. method as claimed in claim 2, is characterized in that, also comprise:
Computing node monitors the state of described each computing node during performing calculation task, when certain node occurs abnormal, the calculation task that this node is responsible for is reassigned to other node.
7. a general multiprocessing parallel calculation system, is characterized in that, comprising:
Task acquiring unit, for obtaining calculation task;
Computing power information acquisition unit, for obtaining hardware information and the software information of each computing node of computational resource, and calculates the processing power of each computing node according to this hardware information and software information;
Task-decomposing unit, for can concurrency according to the inherence of described calculation task, be decomposed into multiple calculating subtask by this calculation task;
Task matching unit, for the processing power according to described each computing node, matches described multiple calculating subtask and described each node;
Task Dispatching Unit, for by basic communication protocol, is protocol format by the Data Format Transform of described multiple calculating subtask, and according to matching relationship, is distributed to each computing node;
Result recovery unit, for by described basic communication protocol, obtains the result of calculation of the protocol format that described each computing node returns;
Format conversion unit, for being converted to the form of former calculation task by the data layout of described result of calculation by protocol format and returning.
8. system as claimed in claim 7, is characterized in that, also comprise calculation server, specifically comprise:
Second format conversion unit, for the multiple calculating subtasks by described protocol format, is converted to the form of the computing node needs matched with it respectively;
Multiple computing node, for performing calculation task, and obtains result of calculation;
Feedback unit, for being converted to protocol format by this result of calculation and returning.
9. system as claimed in claim 7, it is characterized in that, task matching unit comprises:
Computing node screening subelement, for screening according to the processing power of described each computing node, obtaining qualified computing node and participating in calculating;
Coupling subelement, for being matched with described qualified computing node by described multiple calculating, the coupling between described calculating subtask and computing node comprises one to one or multi-to-multi;
Release subelement, for discharging ineligible computing node.
10. system as claimed in claim 7, it is characterized in that, each computing node described is heterogeneous nodes.
11. systems as claimed in claim 8, is characterized in that, also comprise:
Secondary distribution unit, performs during calculation task for computing node, when certain computing node taken the lead in distribution technical assignment after, be its Distribution Calculation task again.
12. systems as claimed in claim 8, is characterized in that, also comprise:
Exception processing unit: for monitoring the state of described each node, when certain node occurs abnormal, is reassigned to other node by the calculation task that this node is responsible for.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310530649.XA CN104598425B (en) | 2013-10-31 | 2013-10-31 | A kind of general multiprocessing parallel calculation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310530649.XA CN104598425B (en) | 2013-10-31 | 2013-10-31 | A kind of general multiprocessing parallel calculation method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104598425A true CN104598425A (en) | 2015-05-06 |
| CN104598425B CN104598425B (en) | 2018-03-13 |
Family
ID=53124230
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310530649.XA Active CN104598425B (en) | 2013-10-31 | 2013-10-31 | A kind of general multiprocessing parallel calculation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104598425B (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104866378A (en) * | 2015-05-29 | 2015-08-26 | 北京京东尚科信息技术有限公司 | System and method for coordinating execution tasks |
| CN105208136A (en) * | 2015-11-04 | 2015-12-30 | 苏州墨华高科信息技术有限公司 | Elasticity parallel CFD (computational fluid dynamics) cloud computing system |
| CN105677486A (en) * | 2016-01-08 | 2016-06-15 | 上海交通大学 | Data parallel processing method and system |
| CN105912399A (en) * | 2016-04-05 | 2016-08-31 | 杭州嘉楠耘智信息科技有限公司 | Task processing method, device and system |
| CN106168939A (en) * | 2016-07-21 | 2016-11-30 | 中铁二院工程集团有限责任公司 | Support the calculating collaboration method calculating task and calculating node more more |
| CN106528292A (en) * | 2016-10-21 | 2017-03-22 | 杭州嘉楠耘智信息科技有限公司 | Task processing method and device |
| CN106547620A (en) * | 2016-10-21 | 2017-03-29 | 杭州嘉楠耘智信息科技有限公司 | Task processing method and device |
| CN106598735A (en) * | 2016-12-13 | 2017-04-26 | 广东金赋科技股份有限公司 | Distributive calculation method, main control node, calculation node and system |
| TWI608360B (en) * | 2016-09-02 | 2017-12-11 | 群暉科技股份有限公司 | Task Execution Method and Task Dispatch System |
| CN108172299A (en) * | 2017-12-25 | 2018-06-15 | 华中科技大学同济医学院附属协和医院 | A kind of medical data distal end computing system and method |
| CN108319495A (en) * | 2017-01-16 | 2018-07-24 | 阿里巴巴集团控股有限公司 | Task processing method and device |
| CN109523022A (en) * | 2018-11-13 | 2019-03-26 | Oppo广东移动通信有限公司 | Terminal data processing method, apparatus and terminal |
| CN110516738A (en) * | 2019-08-23 | 2019-11-29 | 佳都新太科技股份有限公司 | It is a kind of distributed to compare clustering method, device, electronic equipment and storage medium |
| CN110968320A (en) * | 2018-09-30 | 2020-04-07 | 上海登临科技有限公司 | Joint compiling method and compiling system for heterogeneous hardware architecture |
| CN111106946A (en) * | 2018-10-29 | 2020-05-05 | 深圳市云帆加速科技有限公司 | Method and system for reconstructing edge computing node of networking equipment |
| CN111488205A (en) * | 2019-01-25 | 2020-08-04 | 上海登临科技有限公司 | Scheduling method and scheduling system for heterogeneous hardware architecture |
| CN113391917A (en) * | 2020-03-13 | 2021-09-14 | 中国石油天然气集团有限公司 | Multi-machine heterogeneous parallel computing method and device for geophysical prospecting application |
| WO2022141513A1 (en) * | 2020-12-31 | 2022-07-07 | 华为技术有限公司 | Processing method and apparatus for model |
| CN116996513A (en) * | 2023-08-09 | 2023-11-03 | 无锡斑目信息技术有限公司 | Resource scheduling method of equipment asset management system |
| CN117176830A (en) * | 2023-11-03 | 2023-12-05 | 三未信安科技股份有限公司 | Adaptive conversion method of extensible multi-format protocol password application data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101217564A (en) * | 2008-01-16 | 2008-07-09 | 上海理工大学 | A parallel communication system and the corresponding realization method of simple object access protocol |
| CN102063336A (en) * | 2011-01-12 | 2011-05-18 | 国网电力科学研究院 | Distributed computing multiple application function asynchronous concurrent scheduling method |
| CN102209074A (en) * | 2011-05-30 | 2011-10-05 | 中国电力科学研究院 | All-digital dynamic simulation system of power system |
| CN102360310A (en) * | 2011-09-28 | 2012-02-22 | 中国电子科技集团公司第二十八研究所 | Multitask process monitoring method and system in distributed system environment |
| KR101285078B1 (en) * | 2009-12-17 | 2013-07-17 | 한국전자통신연구원 | Distributed parallel processing system and method based on incremental MapReduce on data stream |
-
2013
- 2013-10-31 CN CN201310530649.XA patent/CN104598425B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101217564A (en) * | 2008-01-16 | 2008-07-09 | 上海理工大学 | A parallel communication system and the corresponding realization method of simple object access protocol |
| KR101285078B1 (en) * | 2009-12-17 | 2013-07-17 | 한국전자통신연구원 | Distributed parallel processing system and method based on incremental MapReduce on data stream |
| CN102063336A (en) * | 2011-01-12 | 2011-05-18 | 国网电力科学研究院 | Distributed computing multiple application function asynchronous concurrent scheduling method |
| CN102209074A (en) * | 2011-05-30 | 2011-10-05 | 中国电力科学研究院 | All-digital dynamic simulation system of power system |
| CN102360310A (en) * | 2011-09-28 | 2012-02-22 | 中国电子科技集团公司第二十八研究所 | Multitask process monitoring method and system in distributed system environment |
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104866378A (en) * | 2015-05-29 | 2015-08-26 | 北京京东尚科信息技术有限公司 | System and method for coordinating execution tasks |
| CN105208136A (en) * | 2015-11-04 | 2015-12-30 | 苏州墨华高科信息技术有限公司 | Elasticity parallel CFD (computational fluid dynamics) cloud computing system |
| CN105677486A (en) * | 2016-01-08 | 2016-06-15 | 上海交通大学 | Data parallel processing method and system |
| CN105677486B (en) * | 2016-01-08 | 2019-03-22 | 上海交通大学 | Data parallel processing method and system |
| CN105912399A (en) * | 2016-04-05 | 2016-08-31 | 杭州嘉楠耘智信息科技有限公司 | Task processing method, device and system |
| CN105912399B (en) * | 2016-04-05 | 2020-02-07 | 杭州嘉楠耘智信息科技有限公司 | Task processing method, device and system |
| CN106168939A (en) * | 2016-07-21 | 2016-11-30 | 中铁二院工程集团有限责任公司 | Support the calculating collaboration method calculating task and calculating node more more |
| CN106168939B (en) * | 2016-07-21 | 2019-11-12 | 中铁二院工程集团有限责任公司 | Support the calculating collaboration method of more calculating tasks and more calculate nodes |
| TWI608360B (en) * | 2016-09-02 | 2017-12-11 | 群暉科技股份有限公司 | Task Execution Method and Task Dispatch System |
| CN106528292B (en) * | 2016-10-21 | 2022-04-22 | 上海嘉楠捷思信息技术有限公司 | Task processing method and device |
| CN106547620A (en) * | 2016-10-21 | 2017-03-29 | 杭州嘉楠耘智信息科技有限公司 | Task processing method and device |
| CN106547620B (en) * | 2016-10-21 | 2020-05-19 | 杭州嘉楠耘智信息科技有限公司 | Task processing method and device |
| CN106528292A (en) * | 2016-10-21 | 2017-03-22 | 杭州嘉楠耘智信息科技有限公司 | Task processing method and device |
| CN106598735A (en) * | 2016-12-13 | 2017-04-26 | 广东金赋科技股份有限公司 | Distributive calculation method, main control node, calculation node and system |
| CN108319495A (en) * | 2017-01-16 | 2018-07-24 | 阿里巴巴集团控股有限公司 | Task processing method and device |
| CN108172299A (en) * | 2017-12-25 | 2018-06-15 | 华中科技大学同济医学院附属协和医院 | A kind of medical data distal end computing system and method |
| CN110968320A (en) * | 2018-09-30 | 2020-04-07 | 上海登临科技有限公司 | Joint compiling method and compiling system for heterogeneous hardware architecture |
| US10963229B2 (en) * | 2018-09-30 | 2021-03-30 | Shanghai Denglin Technologies Co., Ltd | Joint compilation method and system for heterogeneous hardware architecture |
| CN111106946B (en) * | 2018-10-29 | 2022-12-30 | 金山云(深圳)边缘计算科技有限公司 | Method and system for reconstructing edge computing node of networking equipment |
| CN111106946A (en) * | 2018-10-29 | 2020-05-05 | 深圳市云帆加速科技有限公司 | Method and system for reconstructing edge computing node of networking equipment |
| CN109523022B (en) * | 2018-11-13 | 2022-04-05 | Oppo广东移动通信有限公司 | Terminal data processing method and device and terminal |
| CN109523022A (en) * | 2018-11-13 | 2019-03-26 | Oppo广东移动通信有限公司 | Terminal data processing method, apparatus and terminal |
| CN111488205A (en) * | 2019-01-25 | 2020-08-04 | 上海登临科技有限公司 | Scheduling method and scheduling system for heterogeneous hardware architecture |
| CN111488205B (en) * | 2019-01-25 | 2024-03-19 | 上海登临科技有限公司 | Scheduling method and scheduling system for heterogeneous hardware architecture |
| CN110516738A (en) * | 2019-08-23 | 2019-11-29 | 佳都新太科技股份有限公司 | It is a kind of distributed to compare clustering method, device, electronic equipment and storage medium |
| CN110516738B (en) * | 2019-08-23 | 2022-09-16 | 佳都科技集团股份有限公司 | Distributed comparison clustering method and device, electronic equipment and storage medium |
| CN113391917A (en) * | 2020-03-13 | 2021-09-14 | 中国石油天然气集团有限公司 | Multi-machine heterogeneous parallel computing method and device for geophysical prospecting application |
| CN113391917B (en) * | 2020-03-13 | 2024-04-30 | 中国石油天然气集团有限公司 | Multi-machine heterogeneous parallel computing method and device for geophysical prospecting application |
| WO2022141513A1 (en) * | 2020-12-31 | 2022-07-07 | 华为技术有限公司 | Processing method and apparatus for model |
| CN116996513A (en) * | 2023-08-09 | 2023-11-03 | 无锡斑目信息技术有限公司 | Resource scheduling method of equipment asset management system |
| CN116996513B (en) * | 2023-08-09 | 2024-04-02 | 无锡斑目信息技术有限公司 | Resource scheduling method of equipment asset management system |
| CN117176830A (en) * | 2023-11-03 | 2023-12-05 | 三未信安科技股份有限公司 | Adaptive conversion method of extensible multi-format protocol password application data |
| CN117176830B (en) * | 2023-11-03 | 2024-01-26 | 三未信安科技股份有限公司 | Adaptive conversion method of extensible multi-format protocol password application data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104598425B (en) | 2018-03-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104598425A (en) | General multiprocessor parallel calculation method and system | |
| Tran et al. | Task placement on fog computing made efficient for IoT application provision | |
| CN103607466B (en) | A kind of wide-area multi-stage distributed parallel grid analysis method based on cloud computing | |
| CN104375882B (en) | The multistage nested data being matched with high-performance computer structure drives method of calculation | |
| Malik et al. | An optimistic parallel simulation protocol for cloud computing environments | |
| CN104123182A (en) | Map Reduce task data-center-across scheduling system and method based on master-slave framework | |
| CN105096181A (en) | E-commerce transaction method and E-commerce transaction system for big data | |
| CN105069702B (en) | A kind of power grid integrated information processing method | |
| Al-Sinayyid et al. | Job scheduler for streaming applications in heterogeneous distributed processing systems | |
| Malik et al. | Optimistic synchronization of parallel simulations in cloud computing environments | |
| CN102063329B (en) | Multi-agent coordination based management task dynamic decomposition method and system | |
| CN105005503A (en) | Cellular automaton based cloud computing load balancing task scheduling method | |
| CN103414784B (en) | Support the cloud computing resource scheduling method of contingency mode | |
| Liu et al. | Reliability modelling and optimization for microservice‐based cloud application using multi‐agent system | |
| CN114490049A (en) | Method and system for automatically allocating resources in containerized edge computing | |
| Zhang | Storage optimization algorithm design of cloud computing edge node based on artificial intelligence technology | |
| Amoretti et al. | Efficient autonomic cloud computing using online discrete event simulation | |
| CN113806606A (en) | Three-dimensional scene-based electric power big data rapid visual analysis method and system | |
| Singh et al. | Combining malleability and i/o control mechanisms to enhance the execution of multiple applications | |
| Lin et al. | Task scheduling algorithm based on Pre-allocation strategy in cloud computing | |
| CN113010296A (en) | Task analysis and resource allocation method and system based on formalized model | |
| CN104484230B (en) | More satellite data central task stream dispatching algorithms based on nearly data calculating principle | |
| CN106357800A (en) | Cloud computing service architecture based on QoE | |
| CN105190599A (en) | Cloud application bandwidth modeling | |
| CN114422455B (en) | Multidimensional resource management architecture and method based on space-air-ground integrated network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |