CN103246497B - A kind of real time data method for parallel processing based on Data Placement - Google Patents
A kind of real time data method for parallel processing based on Data Placement Download PDFInfo
- Publication number
- CN103246497B CN103246497B CN201310138224.4A CN201310138224A CN103246497B CN 103246497 B CN103246497 B CN 103246497B CN 201310138224 A CN201310138224 A CN 201310138224A CN 103246497 B CN103246497 B CN 103246497B
- Authority
- CN
- China
- Prior art keywords
- data
- data processor
- plant stand
- external server
- processor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 30
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000008569 process Effects 0.000 claims description 18
- 238000004891 communication Methods 0.000 claims description 3
- 238000003672 processing method Methods 0.000 claims 5
- 238000012544 monitoring process Methods 0.000 abstract description 3
- 230000001737 promoting effect Effects 0.000 abstract description 2
- 230000004044 response Effects 0.000 abstract description 2
- 230000033772 system development Effects 0.000 abstract description 2
- 230000008901 benefit Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
Landscapes
- Hardware Redundancy (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Supply And Distribution Of Alternating Current (AREA)
Abstract
The invention discloses a kind of real time data method for parallel processing based on Data Placement, by arranging the external server of one group of data processor walked abreast and two redundancy each other, realizing parallel data processing.The present invention has adapted to the demand of electric system development, meets the demand that automation system for the power network dispatching monitoring scale constantly expands.And the present invention can the technical merit of effective elevator system, and the processing power that the back-up system that develops skill greatly is monitored in real time and response speed, for dispatching of power netwoks and large operation provide strong technical support, have good promotional value.
Description
Technical field
The invention belongs to field of power, be specifically related to a kind of real time data method for parallel processing based on Data Placement.
Background technology
Along with the fast development of electrical network, its structure is day by day complicated, scale expanding day, corresponding supporting system technology also needs the computing ability possessing ultra-large electrical network, solves huge, data throughout huge because of electrical network scale and the computing speed, the system running speed decline problem that cause.But, the active and standby mechanism of traditional scheduler system is subject to the limitation of technology, be difficult to the requirement of adapt to development, therefore, in order to improve travelling speed and the monitoring scale of system, make it to be scheduling decision service more efficiently, some scholars are studying and introducing real time data parallel processing mode in electrical networks, it can reasonable utilization and management resource, gives full play to parallel advantage, provides high performance real-time monitor service.
Summary of the invention
For the deficiencies in the prior art, the present invention proposes a kind of real time data method for parallel processing based on Data Placement, takes full advantage of machine and Internet resources, and plays parallel advantage, improve real time data processing ability and treatment effeciency, high performance real-time monitor service is provided.
A kind of real time data method for parallel processing based on Data Placement provided by the invention, its improvements are, described method comprises the steps:
(1) the external server of one group of data processor walked abreast and two redundancy is each other set; Data processor respectively with front end processor and external server communication;
(2) lead external server obtains current data processor from resource management module status information by interface, the plant stand average mark in plant stand table is given current data processor by task dispatch; When newly-increased plant stand, newly-increased plant stand is distributed to the lightest data processor of the task of being calculated by task scheduling modules by main external server;
(3) described data processor carries out data processing according to the task of external server-assignment, and data processed result is fed back and be aggregated into external server.
Wherein, when described front end processor sends message to described data processor, send according to the mode of plant stand grouping;
Data processor receives whole message, but only the data of the plant stand of the machine are distributed in process.
Wherein, described front end processor sends message and divides into groups according to plant stand, often the corresponding plant stand of bag data, and in heading, arrange the information of plant stand described in this bag data.
Wherein, one group of data processor walked abreast described in step (1) pretend all down with, complete electric network model, but according to the task process corresponding data of every number of units according to multi-frame system.
Wherein, in step (2), main external server carries out pre-configured to the difference that the number of current data processor and each plant stand are counted, consistent for ensureing the treatment capacity of each data processor;
The task scheduling modules of main external server distributes to certain data processor factory station data on the one hand, specifies data processor for subsequent use on the other hand described factory station data.
Wherein, when step (2) increases plant stand newly, described task scheduling modules obtains current data processor status information and newly-increased power plant and substation's information according to the mode of recursive call interface and receipt message notice from resource management module, determines the data processor that task is the lightest.
Wherein, when the every platform data processor of step (3) carries out data processing, adopt multi-threading parallel process mode, the data of the plant stand belonging to the machine process are divided equally again, distributes to each thread;
When newly-increased plant stand, newly-increased plant stand is distributed to the lightest thread of task.
Wherein, in step (3), the local real-time database interface of described data processor is revised as real-time database this locality and Network Synchronization amendment interface, data processor passes through this locality and the Network Synchronization amendment interface of real-time database, the real-time database of data processor and external server is modified, ensures that the real-time database in external server standby machine has final complete process result.
Compared with the prior art, beneficial effect of the present invention is:
The present invention has adapted to the demand of electric system development, and meets the demand that constantly expands of automation system for the power network dispatching monitoring scale.
The present invention makes full use of machine and Internet resources, and plays parallel advantage, improves real time data processing ability and treatment effeciency, provides high performance real-time monitor service.
Be the data of the different plant stand of synchronization in the advance data bag that in conventional individual pattern, real time data processing module receives, do not associate mutually between them, also do not need to carry out Message Transmission.SCADA server is divided into internal processor and external server two class by the present invention, adopt data parallel model, real time data in each advance data bag is divided according to plant stand, distribute to different internal processor to process, while parallel data processing, ensure that the treatment capacity of each internal processor is as far as possible consistent.After often bag real time data processing completes, every platform internal processor is synchronized to the data result of the machine process by this locality of real-time database and Network Synchronization amendment interface in the external server standby machine in system, ensures the data consistency on external server.
The present invention can the technical merit of effective elevator system, and the processing power that the back-up system that develops skill greatly is monitored in real time and response speed, for dispatching of power netwoks and large operation provide strong technical support, have good promotional value.
Accompanying drawing explanation
Fig. 1 is the process flow diagram realizing parallel processing provided by the invention.
Fig. 2 is the system schematic of real time data method for parallel processing provided by the invention.
Embodiment
Below in conjunction with accompanying drawing, the specific embodiment of the present invention is described in further detail.
A kind of real time data method for parallel processing based on Data Placement that the present embodiment provides, as shown in Figure 1, as shown in Figure 2, it comprises the steps: total system schematic diagram its Integral Thought
(1) the external server of principal and subordinate of one group (N number of) parallel data processor and two redundancy is each other set; Data processor respectively with front end processor and external server communication;
N number of parallel data processor configuration status is impartial, under pretend with, complete electric network model, be responsible for partial task separately, after processing, result be synchronized to the external server of every platform; External server configures two, gathers the result of internal processor, and two external servers have complete, consistent electric network model and real time data, and main frame is responsible for task scheduling and is externally provided data, services.Its overall flow is, front end processor receives the data message of plant stand (comprising transformer station or power plant etc.), and send to data processor, parallel data processor carries out data processing, and result gathers to external server, and external server externally provides data, services.
(2) lead external server obtains current data processor from resource management module status information by interface, the plant stand average mark in plant stand table is given current data processor by task dispatch, when distributing, main external server carries out pre-configured to the difference that the number of current data processor and each plant stand are counted, consistent for ensureing the treatment capacity of each data processor, consider for robustness, the task scheduling modules of the present embodiment is except distributing to plant stand except certain data processor, specify data processor for subsequent use also can to each plant stand, general standby mode considers that node number adopts the mode of equally loaded for subsequent use, once master processor breaks down, spare processor can take over Processing tasks at once, consider that the convenience of engineering construction can adopt overall standby mode, namely two computing nodes are standby mutually one by one.When newly-increased plant stand, described task scheduling modules obtains current data processor status information and newly-increased power plant and substation's information according to the mode of recursive call interface and receipt message notice from resource management module, thus determine the data processor that task is the lightest, and newly-increased plant stand is distributed to the lightest data processor of task;
(3) described data processor carries out data processing according to the task of external server-assignment, and data processed result is fed back and be aggregated into external server.When every platform data processor carries out data processing, adopt multi-threading parallel process mode, the data of the plant stand belonging to the machine process are divided equally again, distributes to each thread; And when newly-increased plant stand, newly-increased plant stand is distributed to the lightest thread of task.The local real-time database interface of data processor is revised as real-time database this locality and Network Synchronization amendment interface, by this locality and the Network Synchronization amendment interface of real-time database, can ensure that the real-time database in SCADA external server standby machine has final complete process result, the final real time data parallel processing realized based on Data Placement.
Wherein, the front end processor of the present embodiment, when sending message to data processor, sends according to the mode of plant stand grouping; The often corresponding plant stand of bag data, and the information of plant stand described in this bag data is set in heading.Accordingly, data processor receives whole message, but only the data of the plant stand of the machine are distributed in process.Due to the preposition information adding plant stand belonging to this bag data in heading, real time data processing program only needs to read heading and just knows whether these bag data need the machine process, and does not need to unpack, and accelerates data processing speed.
Finally should be noted that: above embodiment is only in order to illustrate that technical scheme of the present invention is not intended to limit, although with reference to above-described embodiment to invention has been detailed description, those of ordinary skill in the field are to be understood that: still can modify to the specific embodiment of the present invention or equivalent replacement, and not departing from any amendment of spirit and scope of the invention or equivalent replacement, it all should be encompassed in the middle of right of the present invention.
Claims (6)
1. based on a real time data method for parallel processing for Data Placement, it is characterized in that, described method comprises the steps:
(1) the external server of one group of data processor walked abreast and two redundancy is each other set; Data processor respectively with front end processor and external server communication; When described front end processor sends message to described data processor, send according to the mode of plant stand grouping; Described front end processor sends message and divides into groups according to plant stand, often the corresponding plant stand of bag data, and in heading, arrange the information of plant stand described in this bag data; Data processor receives whole message, but only the data of the plant stand of the machine are distributed in process;
(2) lead external server obtains current data processor from resource management module status information by interface, the plant stand average mark in plant stand table is given current data processor by task dispatch; When newly-increased plant stand, newly-increased plant stand is distributed to the lightest data processor of the task of being calculated by task scheduling modules by main external server;
(3) described data processor carries out data processing according to the task of external server-assignment, and data processed result is fed back and be aggregated into external server.
2. data parallel processing method as claimed in claim 1, it is characterized in that, one group of data processor walked abreast described in step (1) pretend all down with, complete electric network model, but according to the task process corresponding data of every number of units according to multi-frame system.
3. data parallel processing method as claimed in claim 1, it is characterized in that, in step (2), main external server carries out pre-configured to the difference that the number of current data processor and each plant stand are counted, consistent for ensureing the treatment capacity of each data processor;
The task scheduling modules of main external server distributes to certain data processor factory station data on the one hand, specifies data processor for subsequent use on the other hand described factory station data.
4. data parallel processing method as claimed in claim 1, it is characterized in that, when step (2) increases plant stand newly, described task scheduling modules obtains current data processor status information and newly-increased power plant and substation's information according to the mode of recursive call interface and receipt message notice from resource management module, determines the data processor that task is the lightest.
5. data parallel processing method as claimed in claim 1, it is characterized in that, when the every platform data processor of step (3) carries out data processing, adopt multi-threading parallel process mode, the data of the plant stand belonging to the machine process are divided equally again, distributes to each thread;
When newly-increased plant stand, newly-increased plant stand is distributed to the lightest thread of task.
6. data parallel processing method as claimed in claim 1, it is characterized in that, in step (3), the local real-time database interface of described data processor is revised as real-time database this locality and Network Synchronization amendment interface, data processor passes through this locality and the Network Synchronization amendment interface of real-time database, the real-time database of data processor and external server is modified, ensures that the real-time database in external server standby machine has final complete process result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310138224.4A CN103246497B (en) | 2013-04-19 | 2013-04-19 | A kind of real time data method for parallel processing based on Data Placement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310138224.4A CN103246497B (en) | 2013-04-19 | 2013-04-19 | A kind of real time data method for parallel processing based on Data Placement |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103246497A CN103246497A (en) | 2013-08-14 |
| CN103246497B true CN103246497B (en) | 2016-01-20 |
Family
ID=48926032
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310138224.4A Active CN103246497B (en) | 2013-04-19 | 2013-04-19 | A kind of real time data method for parallel processing based on Data Placement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103246497B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103618671B (en) * | 2013-11-20 | 2018-05-22 | 国家电网公司 | The multigroup dissemination system of larger scale data acquisition business and its distribution method |
| CN104463465B (en) * | 2014-12-05 | 2019-01-04 | 国家电网公司 | A kind of real time monitoring clustering methods based on distributed model |
| CN105071994B (en) * | 2015-08-27 | 2018-08-03 | 许继集团有限公司 | A kind of mass data monitoring system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1630160A (en) * | 2004-08-27 | 2005-06-22 | 江苏省电力公司南京供电公司 | Method for establishing electric network monitoring disaster back-up system |
| CN201518418U (en) * | 2009-10-16 | 2010-06-30 | 巩义市供电公司 | On-line distributed voltage reactive control system |
| CN103034534A (en) * | 2011-09-29 | 2013-04-10 | 阿尔斯通电网公司 | Electric power system analysis parallel computing method and system based on grid computation |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2934142B2 (en) * | 1993-12-27 | 1999-08-16 | 株式会社日立製作所 | Distribution line system monitoring control method and apparatus |
| CN101776711B (en) * | 2010-01-19 | 2012-10-03 | 华北电网有限公司 | Electric energy metering system substation |
-
2013
- 2013-04-19 CN CN201310138224.4A patent/CN103246497B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1630160A (en) * | 2004-08-27 | 2005-06-22 | 江苏省电力公司南京供电公司 | Method for establishing electric network monitoring disaster back-up system |
| CN201518418U (en) * | 2009-10-16 | 2010-06-30 | 巩义市供电公司 | On-line distributed voltage reactive control system |
| CN103034534A (en) * | 2011-09-29 | 2013-04-10 | 阿尔斯通电网公司 | Electric power system analysis parallel computing method and system based on grid computation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103246497A (en) | 2013-08-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2017036244A1 (en) | Digital simulation system of power distribution network | |
| CN112104751B (en) | Method, device and system for processing regulation and control cloud data | |
| WO2016004733A1 (en) | Integrated system and method for figure/model/data of main distributed grid based on remote retrieval and information interaction | |
| CN109327323B (en) | New energy grid-connected power communication network planning and optimizing method and system | |
| CN109325008A (en) | The cross-domain data synchronous method and system of Message Oriented Middleware | |
| CN103390932B (en) | A kind of Distributed Autonomous disaster tolerance standby mode ground county integral system | |
| CN102685237A (en) | Method for requesting session maintaining and dispatching in cluster environment | |
| CN103595140B (en) | A kind of generating based on cloud, distribution and the system and method for intelligent optimization | |
| CN103246497B (en) | A kind of real time data method for parallel processing based on Data Placement | |
| CN104363278A (en) | Mass terminal communication access system | |
| CN113054743A (en) | Internet of things terminal data access system and method suitable for power distribution cloud master station | |
| CN110380512A (en) | The intelligent distribution network coordinated control system and method for the information containing synchronized phasor | |
| CN104836846A (en) | Energy interconnection communication network architecture system based on SDN technology | |
| CN103414784A (en) | Cloud computing resource scheduling method allowing emergency mode | |
| CN104378239A (en) | Rapid reliability index statistical system and method based on cluster frame | |
| CN104298536A (en) | Dynamic frequency modulation and pressure adjustment technology based data center energy-saving dispatching method | |
| CN115543582A (en) | Method, system and equipment for unified scheduling of super computing power network | |
| CN105991366B (en) | A kind of business monitoring method and system | |
| CN115981802A (en) | Computing system and related performance adjusting method | |
| CN103200225A (en) | Method and system of centralized management of services in server cluster | |
| CN103593416B (en) | A kind of Distributed Experiment runs the automatic generation method of middleware isomery gateway | |
| CN209313868U (en) | A kind of distribution cloud stocking system | |
| CN105576827A (en) | High-voltage DC transmission centralized control center realization method and control system | |
| Zhang et al. | Research on distribution network status management system based on cloud platform | |
| CN208316766U (en) | A kind of distributed type assemblies synchronous control system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |