CN104281636B - The concurrent distributed approach of magnanimity report data - Google Patents
The concurrent distributed approach of magnanimity report data Download PDFInfo
- Publication number
- CN104281636B CN104281636B CN201410187511.9A CN201410187511A CN104281636B CN 104281636 B CN104281636 B CN 104281636B CN 201410187511 A CN201410187511 A CN 201410187511A CN 104281636 B CN104281636 B CN 104281636B
- Authority
- CN
- China
- Prior art keywords
- formulary
- report data
- computing
- computer
- fragment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
A kind of concurrent distributed approach of magnanimity report data, including：Obtain report data；Report data formulary is generated, and multiple formulary fragments are cut into by row to the report data formulary that is generated, wherein each formulary fragment includes multirow report data formula；Report data is pushed to each computer node in computer cluster；Multiple computer nodes computing to formulary fragment being assigned in computer cluster carry out calculation process；Preserve the state snapshot of multiple computer node calculation process；And when the computing to any formulary fragment is interrupted, the computing state before interrupting is recovered according to state snapshot, the computing of interruption is continued executing with.Pass through above-mentioned technical proposal, report data formulary is cut into formulary fragment, formulary is set to be handled in the form of fragment in different processing nodes, each processing node only handles a part of formulary fragment and corresponding report data, drastically increases the treatment effeciency of report data.
Description
Technical field
The present invention relates to data processing field, in particular it relates to a kind of concurrent distributed approach of magnanimity report data.
Background technology
At present, to the processing of report data, traditional non-distributed computing technique is typically used.It is this traditional non-
Distributed computing technique is only applicable to handle a small amount of report data, when the quantity of report data constantly expands, reaches flood tide very
To be the stage of magnanimity when, using the traditional approach go handle report data, various drawbacks just occur.First, this tradition
Requirement of the non-distributed technology to software and hardware platform it is all very high, this will bring very high cost pressure to user.Its
It is secondary, even if user is ready to pay high cost, in most cases, the processing speed of this traditional non-distributed technology
Degree and treatment effeciency are all very low.Sometimes, the processing procedure of some report datas, generally requires time-consuming a few hours even
A couple of days could complete.
Currently, a small number of enterprises at home, also using some traditional cluster computings to realize to report data
Processing.Some calculate nodes (server) with high-performance calculation ability are combined into a computing cluster, cluster is used
In calculate node, to share the computational load of whole system.
This traditional cluster computing, although can partly improve the processing speed and treatment effeciency of report data.
But, because its handling principle is that whole report data is all pushed into each calculate node to carry out calculating processing.It is to collection
The hardware requirement of each calculate node is very high in group, and can not make full use of the computing capability of each calculate node.Moreover,
When report data is expanded into some degree (flood tide or magnanimity), treatment effeciency and the bottleneck of speed also occurs.I.e. in sea
Measure under data, by increasing the high-performance calculation node in cluster, the efficiency of the processing of whole system can not be improved.
In view of the above-mentioned problems, there is no good solution in the prior art.
The content of the invention
It is an object of the invention to provide a kind of method, it can be realized by this method and magnanimity report data is quickly located
Reason.
To achieve these goals, the present invention provides a kind of concurrent distributed approach of magnanimity report data, this method
Including：Obtain report data；Report data formulary is generated, and the report data formulary generated is cut into by row multiple
Formulary fragment, wherein each formulary fragment includes multirow report data formula；And the report data is pushed to meter
Each computer node in calculation machine cluster；Computing to formulary fragment is assigned to multiple meters in the computer cluster
Calculation machine node carries out calculation process；Preserve the state snapshot of the multiple computer node calculation process；And when to any public affairs
When the computing of formula collection fragment is interrupted, the computing state before interrupting is recovered according to the state snapshot, and continue executing with interruption
Computing.
Further, the multiple computer node, which carries out calculation process, includes formula operation and carries out the to operation result
One-level merges, to obtain multiple first order amalgamation results；And this method also includes：The result merged to the multiple first order
Carry out second level merging；And the final data result after the second level is merged is exported to intended application.
Further, the step of generation report data formulary, including generation report data check formula collection, and
Report data by verification is generated into report data conversion formula collection.
Further, this method also includes：Heartbeat detection is carried out to the multiple computer node；And the heart will be assigned to
The computing for jumping computer node of the detection without response is redistributed to other computer nodes.
Further, this method also includes：By the calculation process result of the multiple computer node be saved in it is described
The shared memory of all computer nodes connection in computer cluster.
Further, this method also includes：When all computer node calculation process of the current formulary fragment of computing are complete
Cheng Hou, the computing to next formulary fragment is allocated.
Further, this method also includes：The computing to formulary fragment is distributed according to greedy algorithm.
Further, this method also includes：After the computing to last formulary fragment is completed, computing knot is exported
Really.
Further, the computer cluster is made up of the computer node of deployment cloud computing platform.
Further, the cloud computing platform is HADOOP cloud computing platforms.
Further, the computer node is LINUX system server.
By above-mentioned technical proposal, report data formulary is cut into formulary fragment, enables formulary with fragment
Form handled in different processing nodes, each processing node only handles a part of formulary fragment and corresponding report
Table data, drastically increase the treatment effeciency of report data.
Other features and advantages of the present invention will be described in detail in subsequent embodiment part.
Brief description of the drawings
Accompanying drawing is, for providing a further understanding of the present invention, and to constitute a part for specification, with following tool
Body embodiment is used to explain the present invention together, but is not construed as limiting the invention.In the accompanying drawings：
Fig. 1 is the concurrent distributed approach flow chart of magnanimity report data according to embodiment of the present invention；
Fig. 2 is the concurrent distributed approach flow chart of magnanimity report data according to exemplary embodiment of the invention；
Fig. 3 is the concurrent distributed approach flow chart of magnanimity report data according to exemplary embodiment of the invention；
Fig. 4 is the concurrent distributed approach flow chart of magnanimity report data according to exemplary embodiment of the invention；
Fig. 5, Fig. 6 and Fig. 7 are the concurrent distributed treatment sides of magnanimity report data according to exemplary embodiment of the invention
The program implementation example figure of method.
Embodiment
The embodiment of the present invention is described in detail below in conjunction with accompanying drawing.It should be appreciated that this place is retouched
The embodiment stated is merely to illustrate and explain the present invention, and is not intended to limit the invention.
Processing to report data, typically based on two kinds of processing of report data verification and report data conversion.Form number
The accuracy to report data is referred to according to checking treatment, preciseness is checked, find the data having a question, it is ensured that form number
According to the process of accuracy.Report data conversion process refers to extracting the data specified from some specific forms, then
These data are carried out to calculate and process accordingly, the process of the data of specific format is converted into.Wherein whether to form number
According to checking treatment, or to the conversion process of report data, can all be related to the calculating of large amount of complex.
In order to improve the computational efficiency of large amount of complex calculating, the present invention is provided at a kind of concurrent distribution of magnanimity report data
Reason method, as shown in figure 1, this method includes：S101, obtains report data；S102, generates report data formulary, and to institute
The report data formulary of generation is cut into multiple formulary fragments by row, wherein each formulary fragment includes multirow form number
According to formula；And S103, the report data is pushed to each computer node in computer cluster；S104, will be to public affairs
Multiple computer nodes that the computing of formula collection fragment is assigned in the computer cluster carry out calculation process；S105, preserves institute
State the state snapshot of multiple computer node calculation process；And S106, when the computing to any formulary fragment is interrupted, root
Recover the computing state before interrupting according to the state snapshot, and continue executing with the computing of interruption.
By above-mentioned technical proposal, report data formulary is cut into formulary fragment, enables formulary with fragment
Form handled in different processing nodes, each processing node only handles a part of formulary fragment and corresponding report
Table data, drastically increase the treatment effeciency of report data., can be from the public affairs failed when the computing failure of formulary fragment
The computing of formula collection fragment is continued executing with, without being repeated from original state, is entered without to other formularies not failed
Row repetitive operation, improves task treatment effeciency, the reduction wasting of resources.In embodiments can be after report data be obtained just
Acquired report data is pushed to each computer node in computer cluster, with when the computing point to formulary fragment
After matching somebody with somebody, computer node can immediately begin to computing without waiting pending data.
In embodiments, the operation result of each computer node can be returned by network to carry out collecting conjunction
And.It is contemplated that operation result data volume extremely huge situation, network possibly can not quickly transmit huge operation result
Data volume, as short slab.Therefore, in a preferred embodiment, multiple computer nodes, which carry out calculation process, can include public affairs
Formula computing and to operation result carry out first order merging, to obtain multiple first order amalgamation results；And method can also include：
Second level merging is carried out to the result that the multiple first order merges；And final data result (the example after the second level is merged
Such as, Credential data) export to intended application.Formulary is handled in different processing nodes, it is therefore desirable to which processing is obtained
Result merge, it is contemplated that a processing node is there may be multiple results that can merge, therefore the merging of result can
Collect and merge with the result for being included in collecting and merging in a processing node and multiple processing nodes.For closing
And, one or more computers specified in computer cluster or idle can be selected to carry out.
In embodiments, method can also include：Heartbeat detection is carried out to the multiple computer node；And will divide
The computing for being fitted on computer node of the heartbeat detection without response is redistributed to other computer nodes.Can be true by heartbeat detection
Surely the working condition of the computer node of computing is carried out.In order to ensure the computing for being assigned to each computer node can be complete
Into when there is computer of the heartbeat detection without response, the computing for being assigned to the computer node being re-assigned into it
The normal computer node of his heartbeat detection.Preferably, the computer node being re-assigned to can complete allocated
The computer node of computing, can so make full use of idle computing resources to complete computing.
Call and collect with operation result for the ease of formula and/or formulary, in embodiments, this method can also be wrapped
Include：The calculation process result of the multiple computer node is saved in and all computer nodes in the computer cluster
The shared memory of connection.
In embodiments, this method can also include：When all computer nodes of the current formulary fragment of computing are transported
After the completion of calculation processing, the computing to next formulary fragment is allocated, to utilize whole computer node resource processing one
Individual formulary fragment, improves processing speed.
Fig. 2 is the concurrent distributed approach flow chart of magnanimity report data according to exemplary embodiment of the invention.
As shown in Fig. 2 can be included according to the concurrent distributed approach of magnanimity report data of exemplary embodiment of the invention：Root
The form handled according to report data, is divided into two kinds of such as " data check " and " data conversion " different by report data processing
Handle type.The rule of classification that type sets data is handled according to above two：" data check rule of classification " and " data conversion
Rule of classification ", or the condition to be verified or being changed to data.Preferably, set can be to being carried out by the data of verification
Conversion.It is multiple data slots by data cutting it is then possible to carry out cutting to data according to the rule of classification of setting.If (
" data check rule of classification ", then carry out cutting to " the data check formula " that has set.Wherein it is possible to according to financial data school
Proved recipe formula sets up data check formula to determine whether data are accurate；If " data conversion rule of classification ", then to having set
" data conversion formula " carries out cutting.) data slot after cutting and the original report data are assigned in computer cluster
Multiple computer nodes carry out Distributed Parallel Computing, and form interim findings collection.Then, collected according to different rules of classification
Interim findings collection is (if for example, " data check rule of classification ", then can collect the information of verification failure；If " data conversion
Rule of classification ", then can collect the data set after conversion), and form final data result.Finally, final data result is united
One exports to intended application for providing form or data error reporting.
In the embodiment shown in figure 2, it is to draw aggregate data according to rule of classification for the angle of aggregate data
Being divided into needs to be verified and needs to carry out two kinds of classifications of conversion；And for the angle of a certain data, the data need advanced
Row verification, is changed again in the case where verification passes through.Therefore, in embodiments, the step of report data formulary is generated
Suddenly, generation report data check formula collection can be included, and the report data generation report data conversion by verification is public
Formula collection.
For same data, the concurrent distributed approach of magnanimity report data that the present invention is provided can be according to task chain
Formula is carried out by sequence of steps.Aggregate data is probably magnanimity, and needs to verify and/or need the data progress operand changed
It may similarly be magnanimity rank.Such operand, common system and hardware is difficult to complete, and generally requires high performance hard
Part is supported, and high performance hardware certainly will need high cost input.Therefore, being provided in embodiments of the present invention with lower section
Method solves the contradiction between big data quantity and the not high hardware system of performance.
Fig. 3 is the concurrent distributed approach flow chart of magnanimity report data according to exemplary embodiment of the invention.
As shown in figure 3, the concurrent distributed approach of magnanimity report data that embodiment of the present invention is provided, can include：S301, will
Each step is divided into multiple subtask nodes；S302, is assigned computing needed for the node of subtask to the computer collection
Multiple computer nodes in group carry out calculation process；S303, the state for preserving the multiple computer node calculation process is fast
According to；And S304, when the subtask node interrupts, the subtask node shape before interrupting is recovered according to the state snapshot
State, and continue executing with the subtask node of interruption.
By above-mentioned technical proposal, the computing needed for completing a task chain step is assigned to multiple computer nodes and entered
Row calculation process, can break the whole up into parts operand, improve task run speed；By preserving the state snapshot of calculation process,
When task chain step is interrupted for some reason, task chain step can be returned to state before interruption, so that since state before interrupting
Continue executing with task chain.Therefore,, can be from the step failed when task chain step or node failure by above-mentioned technical proposal
Rapid or node continues executing with task chain, without being repeated from original state, improves task treatment effeciency, reduction resource wave
Take.
In a preferred embodiment, the above method can also include：Heartbeat detection is carried out to multiple computer nodes；With
And the computing for being assigned to computer node of the heartbeat detection without response is redistributed to other computer nodes.Pass through heartbeat detection
The working condition of the computer node of progress computing can be determined.In order to ensure the computing for being assigned to each computer node can
It is enough to complete, when there is computer of the heartbeat detection without response, the computing for being assigned to the computer node can be redistributed
To the normal computer node of other heartbeat detections.Preferably, the computer node being re-assigned to can complete to be divided
The computer node for the computing matched somebody with somebody, can so make full use of idle computing resources to complete computing.
In order that completing the computer node of the computing of each task node or subtask node in task chain can obtain
Data needed for computing, in embodiments, method also includes：The calculation process result of the multiple computer node is preserved
To the shared memory being connected with all computer nodes in the computer cluster.So all computer nodes can be
When computing starts from shared memory obtain operational data, and when computing is completed into shared memory storage computing knot
Really.Explanation is needed exist for, the state snapshot of computer node calculation process can also be stored in shared memory, or
Snapshot can also be separately provided.The example of memory includes but is not limited to read-only storage (ROM), arbitrary access and deposited
Reservoir (RAM), register, buffer storage, semiconductor memory apparatus etc..
In various embodiments, this method can also include：When all computers for performing current subtask node
After the completion of node calculation process, the computing needed for completing next subtask node is allocated.In embodiments, exist
A variety of situations need to perform above-mentioned steps.For example, for a large amount of computings, it is necessary to use all computer sections in computer cluster
The computing for the subtask node that point is participated in, could only continue distribution next after all computer nodes all complete calculation process
The computing of subtask node.For another example, it could distribute next after whole operation results that upper subtask node is obtained for needs
It is also required to include such step in the situation of the computing of subtask node, method.Certainly, in embodiments, can also be by
The different computer node groups that the computing of different subtask nodes is assigned in computer cluster simultaneously are handled.
The processing procedure in the processing of magnanimity report data is illustrated with reference to Fig. 4.As shown in figure 4, appointing at one
Can have in multiple tasks node (or net-shape processed node), each task node and can include between chain be engaged in from start to end
(such as with fully connected topology) multiple subtask nodes, the computing of each subtask node can be assigned to computer
One or more of cluster computer node (not shown) is handled.Each computer node computing to be carried out
It can be carried out, the Master Control Unit can be a computer node in computer cluster, born simultaneously by unified Master Control Unit
Blame the heartbeat detection of other all computer nodes in computer cluster.The context shared data related to computing is (for example, meter
The calculation process result of calculation machine node) it can be saved to and being total to that all computer nodes in the computer cluster are connected
Enjoy memory.After the computing of last net-shape processed node is completed, operation result can be exported, being for example output to target should
With.It should be noted that being shown in Fig. 4 includes the task chain of three net-shape processed nodes, but accompanying drawing is merely for exemplary
Purpose, not the length to task chain limit.For example according to the concurrent distributed treatment side of magnanimity report data of the present invention
Method can use two net-shape processed nodes, and data check and data conversion are handled respectively.
In a preferred embodiment, can be according to the computing needed for greedy algorithm distribution completion subtask node.
Handled i.e., it is possible to give computing capability most strong computer node by most complicated computing, and then ensure whole computing
Processing speed.
In a preferred embodiment, computer cluster can be made up of the computer node of deployment cloud computing platform, from
And the resources advantage of cloud computing platform can be utilized, reduce hardware requirement of a large amount of computings to computer node.For example, can make
HADOOP cloud computing platforms are used, and computer node can be to deploy cloud computing platform (such as HADOOP) LINUX system
Server.Hereinafter, with reference to HADOOP cloud computing platforms to the concurrent distributed approach of magnanimity report data according to the present invention
Implementation illustrate.
A kind of illustrative embodiments of the present invention realize that magnanimity report data is concurrent using HADOOP cloud computing platforms
Distributed approach.Its specific embodiment is as follows：
(1) 5 to 10 common servers (LINUX operating systems) are chosen, processing node is calculated as report data；
(2) HADOOP platforms are disposed on these common servers, by these machine assemblies into a Distributed Calculation
Cluster；
(3) distributed file system (HDFS) is initialized on Distributed Calculation cluster；
(4) form that system is handled according to report data, according to " data check rule of classification " and " data conversion packet rule
Then " to needing magnanimity report data to be processed to be grouped；(following steps (5)-(14) are carried out by taking " data conversion " processing as an example
Description)
(5) if " data conversion " is handled, then system is by the content of magnanimity report data and data conversion formula set
Content push is to distributed file system (HDFS)；
(6) system by the content of the data conversion set of formulas pushed in distributed file system (HDFS) according to
The quantity of machine in Distributed Calculation cluster and the computing capability of machine carry out cutting.In in data conversion set of formulas
Appearance is divided into 5 to 10 data slots (can be identical with participating in processing number of nodes)；
(7) 5 to 10 data slots segmented together with magnanimity report data, are pushed to distributed meter by system together
Calculate in cluster each calculate node (whole report datas are pushed to each calculate node so that each calculate node according to
Formula set operation demand is used)；
(8) system initiation task allocation schedule program, calls each calculate node in Distributed Calculation cluster, while right
Distribution to the data slot in the calculate node and magnanimity report data carries out calculating processing；
(9) system calls each calculate node in Distributed Calculation cluster, and data calculating in the calculate node is handled
Result, carry out small range and collect and merge, in each calculate node, produce interim findings collection；
(10) free time of each calculate node in system calling task allocation schedule program, observation Distributed Calculation cluster
State, enables the more idle calculate node of some of which (2 to 3 calculate nodes) as the calculate node for merging work, is
Merge work to prepare；
(11) system calling task allocation schedule program, starts the merging in the calculate node for the execution merging work chosen
Program；
(12) system will be called and perform the calculate node for merging work, collect the interim findings produced in previous step
Collection, sort and merge on a large scale；
(13) each execution of systematic collection merges the result set produced in the calculate node of work, is closed on a large scale again
And and collect, and by the data conversion result set eventually formed push in HDSF file system preserve.And stop it is all just
In the parallel computation task of the redundancy of execution；And
(14) system obtains the last data conversion result being stored in HDFS file system, and last lattice are carried out to it
Formula is changed, and the result after conversion is uniformly output in traditional relevant database (ORACLE) database.For other
Application use.
The embodiment of the embodiment of " data check " and " data conversion " is similar.
It can be realized by the above method and magnanimity report data is quickly handled.
Understand below for ease of the principle to the present invention, one embodiment is provided with reference to Fig. 5-Fig. 7.It is described as follows：
Assuming that now with a form for needing to carry out data conversion, form is entitled：" project under construction detail list (one) ".Should
The data of form are as shown in Figure 5.
Assuming that being provided with the set such as next number according to transformation rule in system.The structure of the data conversion rule set is
The matrix of one 1 × 72 (row × row).In the set of the data conversion rule, only 2 are provided with the 69th row and the 70th row
Transformation rule.69th row transformation rule be：YBZC041！AA [one, construction project # two, technological transformation project]；The conversion of 72nd row
Rule is also：YBZC041！AB [one, construction project # two, technological transformation project].The data conversion rule set is as shown in Figure 6.
Data conversion rule explanation：
Transformation rule " the YBZC041 of the 68th row in data conversion rule set！AA [one, construction project # two, technological transformation
Project] " implication be：From taking out that AA in Fig. 5 arranges in " project under construction detail list (one) " from the 10th row to the 26th row 17
The data of cell, and the data of this 17 cells are filled into the 68th row of data conversion rule set successively.
Transformation rule " the YBZC041 of the 70th row in data conversion rule set！AB [one, construction project # two, technological transformation
Project] " implication be：From taking out that AB in Fig. 5 arranges in " project under construction detail list (one) " from the 10th row to the 26th row 17
The data of cell, and the data of this 17 cells are filled into the 70th row of data conversion rule set successively.
The result that final system is changed out is as shown in Figure 7.
The preferred embodiment of the present invention is described in detail above in association with accompanying drawing, still, the present invention is not limited to above-mentioned reality
The detail in mode is applied, in the range of the technology design of the present invention, a variety of letters can be carried out to technical scheme
Monotropic type, these simple variants belong to protection scope of the present invention.For example, computer node can be changed into calculate node
Or computing unit.
It is further to note that each particular technique feature described in above-mentioned embodiment, in not lance
In the case of shield, it can be combined by any suitable means.In order to avoid unnecessary repetition, the present invention to it is various can
The combination of energy no longer separately illustrates.
In addition, various embodiments of the present invention can be combined randomly, as long as it is without prejudice to originally
The thought of invention, it should equally be considered as content disclosed in this invention.
Claims (10)
1. a kind of concurrent distributed approach of magnanimity report data, it is characterised in that this method includes：
Obtain report data；
Report data formulary is generated, and multiple formulary fragments are cut into by row to the report data formulary that is generated, its
In each formulary fragment include multirow report data formula；And
The report data is pushed to each computer node in computer cluster；
Multiple computer nodes computing to formulary fragment being assigned in the computer cluster carry out calculation process；
Preserve the state snapshot of the multiple computer node calculation process；And
When the computing to any formulary fragment is interrupted, the computing state before interrupting is recovered according to the state snapshot, and
Continue executing with the computing of interruption；
Wherein, the multiple computer node, which carries out calculation process, includes formula operation and carries out first order conjunction to operation result
And, to obtain multiple first order amalgamation results；And
This method also includes：Second level merging is carried out to the result that the multiple first order merges；And after the second level is merged
Final data result export to intended application.
2. according to the method described in claim 1, it is characterised in that the step of the generation report data formulary, including life
Report data conversion formula collection is generated into report data check formula collection, and by the report data by verification.
3. according to the method described in claim 1, it is characterised in that this method also includes：
Heartbeat detection is carried out to the multiple computer node；And
The computing for being assigned to computer node of the heartbeat detection without response is redistributed to other computer nodes.
4. according to the method described in claim 1, it is characterised in that this method also includes：
The calculation process result of the multiple computer node is saved in and all computer sections in the computer cluster
The shared memory of point connection.
5. according to the method described in claim 1, it is characterised in that this method also includes：
After the completion of all computer node calculation process of the current formulary fragment of computing, to the fortune of next formulary fragment
It is allocated.
6. according to the method described in claim 1, it is characterised in that this method also includes：
The computing to formulary fragment is distributed according to greedy algorithm.
7. according to the method described in claim 1, it is characterised in that this method also includes：
After the computing to last formulary fragment is completed, operation result is exported.
8. according to the method described in claim 1, it is characterised in that the computer cluster by deployment cloud computing platform calculating
Machine node is constituted.
9. method according to claim 8, it is characterised in that the cloud computing platform is HADOOP cloud computing platforms.
10. method according to claim 8, it is characterised in that the computer node is LINUX system server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410187511.9A CN104281636B (en) | 2014-05-05 | 2014-05-05 | The concurrent distributed approach of magnanimity report data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410187511.9A CN104281636B (en) | 2014-05-05 | 2014-05-05 | The concurrent distributed approach of magnanimity report data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104281636A CN104281636A (en) | 2015-01-14 |
| CN104281636B true CN104281636B (en) | 2017-09-08 |
Family
ID=52256512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410187511.9A Active CN104281636B (en) | 2014-05-05 | 2014-05-05 | The concurrent distributed approach of magnanimity report data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104281636B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107239327A (en) * | 2016-03-29 | 2017-10-10 | 平安科技（深圳）有限公司 | The optimization method and device of declaration form processing |
| CN106874080B (en) * | 2016-07-07 | 2020-05-12 | 阿里巴巴集团控股有限公司 | Data calculation method and system based on distributed server cluster |
| CN108241806A (en) * | 2016-12-23 | 2018-07-03 | 航天星图科技（北京）有限公司 | A kind of concurrent distributed validation method for structural data |
| CN111209301A (en) * | 2019-12-29 | 2020-05-29 | 南京云帐房网络科技有限公司 | Method and system for improving operation performance based on dependency tree splitting |
| CN113010590B (en) * | 2021-02-24 | 2023-07-07 | 光大兴陇信托有限责任公司 | Unified supervision reporting method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1755633A (en) * | 2004-09-27 | 2006-04-05 | 微软公司 | Method and system for multithread processing of spreadsheet chain calculations |
| CN101441557A (en) * | 2008-11-08 | 2009-05-27 | 腾讯科技（深圳）有限公司 | Distributed parallel calculating system and method based on dynamic data division |
| CN101616028A (en) * | 2009-06-25 | 2009-12-30 | 中兴通讯股份有限公司 | A kind of communication program service does not interrupt upgrade method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8868848B2 (en) * | 2009-12-21 | 2014-10-21 | Intel Corporation | Sharing virtual memory-based multi-version data between the heterogenous processors of a computer platform |
-
2014
- 2014-05-05 CN CN201410187511.9A patent/CN104281636B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1755633A (en) * | 2004-09-27 | 2006-04-05 | 微软公司 | Method and system for multithread processing of spreadsheet chain calculations |
| CN101441557A (en) * | 2008-11-08 | 2009-05-27 | 腾讯科技（深圳）有限公司 | Distributed parallel calculating system and method based on dynamic data division |
| CN101616028A (en) * | 2009-06-25 | 2009-12-30 | 中兴通讯股份有限公司 | A kind of communication program service does not interrupt upgrade method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104281636A (en) | 2015-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104281636B (en) | The concurrent distributed approach of magnanimity report data | |
| CN106406896B (en) | Block chain block building method based on parallel Pipeline technology | |
| EP3477556A1 (en) | Method and apparatus for performing operations in convolutional neural network | |
| CN104536937B (en) | Big data all-in-one machine realization method based on CPU GPU isomeric groups | |
| CN103970851B (en) | The method that magnanimity Credential data directly provides general headquarters of large-size enterprise group financial statement | |
| CN103699440A (en) | Method and device for cloud computing platform system to distribute resources to task | |
| CN104809168A (en) | Partitioning and parallel distribution processing method of super-large scale RDF graph data | |
| CN107070645A (en) | Compare the method and system of the data of tables of data | |
| Senger et al. | BSP cost and scalability analysis for MapReduce operations | |
| WO2013168495A1 (en) | Hierarchical probability model generating system, hierarchical probability model generating method, and program | |
| Stützle et al. | New benchmark instances for the QAP and the experimental analysis of algorithms | |
| CN103970611A (en) | Task processing method based on computer cluster | |
| CN104778088A (en) | Method and system for optimizing parallel I/O (input/output) by reducing inter-progress communication expense | |
| Fioretto et al. | GD-GIBBS: a GPU-based sampling algorithm for solving distributed constraint optimization problems. | |
| Yin et al. | VF2x: Fast, efficient virtual network mapping for real testbed workloads | |
| CN104699799A (en) | Data transmission method based on cross system | |
| CN106412125A (en) | Parallelization cloud monitoring system based on load balancing and construction method | |
| CN109710314B (en) | A method of based on graph structure distributed parallel mode construction figure | |
| CN104462023B (en) | The method of ultra-large sparse matrix multiplication computing based on mapreduce frameworks | |
| Tian et al. | Recovery mechanism of large-scale damaged edge computing net-work in industrial internet of things | |
| CN111177267A (en) | Data transmission method and device, electronic equipment and storage medium | |
| Wu et al. | Improved simulated annealing algorithm for task allocation in real-time distributed systems | |
| Deng et al. | A Solution Framework for All-to-All Comparison Data Distribution Strategy Based on Tabu Search | |
| Chung et al. | Optimizing a MapReduce module of preprocessing high-throughput DNA sequencing data | |
| CN108241806A (en) | A kind of concurrent distributed validation method for structural data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |