CN106708624A - Adaptive adjustment method for calculation resources in multiple working areas - Google Patents
Adaptive adjustment method for calculation resources in multiple working areas Download PDFInfo
- Publication number
- CN106708624A CN106708624A CN201611048286.6A CN201611048286A CN106708624A CN 106708624 A CN106708624 A CN 106708624A CN 201611048286 A CN201611048286 A CN 201611048286A CN 106708624 A CN106708624 A CN 106708624A
- Authority
- CN
- China
- Prior art keywords
- working field
- computing unit
- unit number
- working
- calculate node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5011—Pool
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Multi Processors (AREA)
Abstract
The invention belongs to the field of distributed calculation, and puts forward an adaptive adjustment method for calculation resources in multiple working areas. According to different calculation periods of an application function, a system needs to be divided into a plurality of working areas, each working area is taken as a unit to carry out independent organization calculation, and calculation nodes are switched among the plurality of working areas to meet the requirements of different working area calculation periods. By use of the method, the constraint conditions, including the reference working period, the calculation resource distribution priority, the shortest calculation period, the longest allowable calculation period and the like, of each working area are comprehensively considered, and a calculation unit amount distributed by each activated working area is determined on the basis of the execution time of the calculation task in the calculation unit. On the basis, the calculation resources of the calculation nodes configured by different hardware are subjected to standardized processing according to the calculation unit so as to realize the adaptive adjustment of the calculation resources among the plurality of working areas.
Description
Technical field
The invention belongs to Distributed Calculation field, more precisely, the present invention relates to a kind of many working field computing resources
Self-adapting regulation method.
Background technology
In Distributed Calculation field, in particular for the field for carrying out a large amount of calculating, such as in power system safety and stability
Analysis field, calculating speed has turned into the key factor of the practical lifting of application function.Parallel computation is extensive, complicated lifting
Electrical network analysis calculate the effective ways of performance, and the ripe application of Distributed Calculation management platform promotes power system safety and stability
Analyze the popularization of application on site.There is the requirement of real-time of calculating cycle due to online multiple application function, and it is offline multiplex
Family has random nature, and the computing resource utilization rate of its separate configurations is relatively low.Distributed Calculation management platform is by working field
Carry out calculation process tissue and control on the basis of, in the urgent need to computing resource support dynamically carried out between multiple working fields it is excellent
Change distribution and self-adaptative adjustment, the effectively reliability of lifting application on site and the efficient utilization for promoting offline computation resource.
Document one《The fault-tolerance approach mutually backed up using cluster nodes》(the patent No.：CN02159479.1 one kind) is proposed
The fault-tolerance approach mutually backed up using cluster nodes.The method between cluster nodes by heartbeat ring connect, be in communication with each other and
Backup；Host node distributes position of the newly added node in a group of planes, and returns to the information on services that newly added node is undertaken；Work as machine
When group's interior joint finds that adjacent node is abnormal, confirmed to the adjacent node；Host node takes over to the failed services.Should
Method mainly solves the problems, such as the node stand-by heat in cluster management, but the method does not consider that computing resource is calculated in multiple
Dynamically distributes problem between a group of planes.
Document two《A kind of group of planes AMS and its application management method》(the patent No.：CN
201010286186.3) a kind of group of planes AMS for being applied to large-scale cluster management is proposed.The system includes holding
Row engine modules and DBM, DBM are used to store in real time the result of each application, and set up monitoring table
The change information of the result of all associated applications of the multiple applications of record；During enforcement engine module is used to perform NOWs
Each application, and by the result of each application write into Databasce module in real time, while periodically in reading database module
Monitoring table and the results according to the multiple applications read change information, the trigger condition of each application is judged respectively is
It is no to have met, the corresponding application of triggering if trigger condition meets.
Document three《A kind of many application function asynchronism and concurrency dispatching methods of Distributed Calculation》(the patent No.：
CN201110005759.5 the method for) proposing a kind of asynchronism and concurrency scheduling of multiple application functions, is applied to Distributed Calculation
In management platform.The time consuming nature and quantity and meter of the calculating task of comprehensive each application function of Distributed Calculation management platform
The scale and performance information of cluster nodes are calculated, suitable computing job scheduling granularity are independently arranged by for each application function,
The calculating operation of each application function is formed, is added in the schedule sequences of Distributed Calculation management platform, realize multiple applications
The calculating task asynchronism and concurrency of function is submitted to, the United Dispatching of the calculating operation of multiple application functions is calculated and multiple application functions
The asynchronous recovery of result of calculation.
In three of the above method, document one solve only the node stand-by heat problem in single working field, not account for many
The intermediate node stand-by heat and computing resource dynamically distributes problem of individual working field；Document two is based on database realizing and is applied to big rule
The group of planes AMS of mould cluster management, does not account for different pieces of information source (the online and offline applied field of power system
Scape) application function run calculating cycle otherness demand.Document three solve only multiple application work(in single computer cluster
The method of the asynchronism and concurrency scheduling of energy, and the corresponding computing resource of group of planes fixed allocation, it is impossible to flat according to Distributed Calculation management
Platform running emergency case (such as running statuses of the change of working field state of activation, cluster nodes) to computing resource
Enter Mobile state adjustment, to meet the demand in system-computed cycle.Therefore above-mentioned three kinds of methods all do not solve power train well
Statistics calculates resource dynamic management problem, and the calculating cycle of on-line system cannot be to be fully ensured, the computing resource of off-line system
Cannot be utilized.
The content of the invention
The present invention seeks to：For calculation features, the work week of multiple application functions under online, offline different application scene
Phase and the resulting dynamic need to computing resource, there is provided a kind of many work suitable for Distributed Calculation management platform
Domain computing resource self-adapting regulation method.
Specifically, the present invention is realized using following technical scheme, is comprised the following steps：
1) different demands of all kinds of computing resources will be completed by a calculating task and calculates each of needs according to calculating task
Class computing resource minimum combination is defined as a computing unit, the computing resource that each calculate node is included according to computing unit
It is standardized, its computing capability is measured again by the nonnegative integer of computing unit, into step 2)；
2) for the calculating demand of each application function under the online and offline application scenarios of power system, computer cluster is drawn
It is divided into multiple working fields, each working field independently carries out calculating task tissue, scheduling and manages based on Distributed Calculation management platform,
Do not carry out calculating the interaction of data and control information in calculating process between each working field directly；
Important level and its requirement to calculating the time according to each working field, are that its unification sets reference work week
Phase, computational resource allocation priority, most short calculating cycle and permission calculating cycle parameter most long, and exist according to single calculating task
Expected time on one computing unit, will most short calculating cycle and most it is long permission calculating cycle be converted to maximum allocated meter
Calculate unit number and minimum reservation computing unit number；
When the state of activation in any operative domain or the running status of any calculate node change, go to step 3)；
3) reference work cycle, computational resource allocation priority, the most short calculating set based on each working field for activating
Cycle and permission calculating cycle parameter most long and the working field for activating at present and the normal calculate node run, ask for each and swash
The computing unit quantity of working field predistribution living；If currently available computing unit sum is less than or equal to current all activated work
Make the minimum of domain and retain computing unit number sum, then the computational resource allocation priority according to each activation working field is suitable from high to low
Sequence, retain computing unit number according to its minimum carries out computing unit predistribution successively, until all computing units are assigned；
4) for all working domain for completing predistribution, computing unit number and adjustment based on the predistribution of each working field
The preceding allocated normal operation computing unit number of each working field calculates the normal operation calculating that each working field participates in adjustment
Number of unit, and principle is lacked based on the calculate node quantity for participating in adjustment as far as possible, it is preferential with reference to the switching of each calculate node
Level and its computing unit number for including determine the affiliated working field of each calculate node after adjustment；And protected to minimum to unallocated
The activation working field of computing unit number computing resource is stayed to provide prompting；
5) the Distributed Calculation management platform in each calculate node perceives working field change information, from the switching of former working field
Working field after to adjustment, completes self-adaptative adjustment of the calculate node between multiple working fields.
Above-mentioned technical proposal is further characterized by, the step 1) in the calculating that is had according to it of each calculate node
Resource carries out computing unit standardization according to formula (1), determines effective computing unit number that each calculate node is included,
So as to be measured to its computing capability：
Wherein, n is cluster computing interstitial content, u in systemiIt is the computing unit number that i-th calculate node is included, its
It is nonnegative integer；
Rcpu(i)、Rmem(i) and RioI () is respectively CPU, internal memory and the IO computing resources that i-th calculate node has；
Rcpu 0、Rmem 0And Rio 0It is respectively CPU, internal memory and IO computing resources that metric calculation unit has.
Above-mentioned technical proposal is further characterized by, the step 2) in by formula (2) and (3) respectively by each work
Make the most short calculating cycle in domain and permission calculating cycle most long is converted to maximum allocated computing unit number and minimum reservation calculates single
First number：
Wherein, m is working field number quantity, t in systemminJ () is j-th most short calculating cycle of working field, tmax(j)
It is j-th permission calculating cycle most long of working field, fjIt is the j-th application function quantity of working field needs operation, sjkIt is jth
The calculating task quantity of k-th application function of individual working field operation,It is k-th single calculating task of application function one
Expected time on individual computing unit,It is the oeprator that rounds up；
cmaxJ () is nonnegative integer, represent j-th maximum allocated computing unit number of working field；cminJ () is that non-negative is whole
Number, represents that j-th the minimum of working field retains computing unit number.
Above-mentioned technical proposal is further characterized by, the step 3) specifically include following steps：
3.1) using all normal computing unit quantity sums for being included of operation calculate nodes as computing unit to be allocated
Quantity cuInitial value, the predistribution computing unit number of each working field is initialized as 0；The current all activated work of statistics
The minimum of domain retains computing unit number sumWherein m is working field number quantity, d in systemjIt is j-th
The state of activation of working field, its value is 1 expression working field activation, is worth for 0 represents the working field un-activation；
IfGo to step 3.2)；Otherwise go to step 3.8)；
3.2) to each working field being active, dropped from high to low according to its computational resource allocation priority
Sequence arrange, the most preceding working field of selected and sorted as working field to be pre-allocated, into step 3.3)；
3.3) the predistribution computing unit number of working field to be pre-allocated is asked for according to formula (4)：
Wherein, rjIt is j-th predistribution computing unit number of working field；ciIt is i-th running status of calculate node,
ciValue for 1 expression the calculate node normally run, be worth for 0 represent the calculate node failure；ηjIt is j-th working field setting
In the reference work cycle, be the computational resource allocation coefficient between different operating domain；It is downward rounding operation symbol；
If 3.4) the predistribution computing unit number of working field to be pre-allocated calculates single more than the working field maximum allocated
First number, then be updated to the working field maximum allocated meter by the value of the predistribution computing unit number of the working field to be pre-allocated
Calculate number of unit；
3.5) judge whether the predistribution computing unit number of working field to be pre-allocated calculates single less than or equal to be allocated
First quantity cuIf being less than or equal to cuThen by cuValue be updated to cuSubtract the predistribution computing unit of the working field to be pre-allocated
Number, is otherwise updated to c by the value of the predistribution computing unit number of working field to be pre-allocateduAnd update cuValue be 0；
3.6) c is judgeduWhether it is 0, step 4 is entered if equal to 0), will otherwise come this working field to be pre-allocated
3.3) working field of latter goes to step as new working field to be pre-allocated carries out the computing resource of next activation working field
Predistribution, until completing the computing resource predistribution of all activated working field；
If 3.7) computing unit quantity c to be allocated after the completion of the computing resource predistribution of all activated working fielduStill greater than
0, then to being not up to all activated working field that maximum allocated computing unit number is constrained, according to the computational resource allocation of each working field
Order is sequentially allocated a computing unit to priority to each activation working field from high to low, until all computing units all divide
Dispensing working field or the computing unit of all activated working field distribution all reach maximum allocated computing unit number；Into step 4)；
3.8) for all activated working field, the computational resource allocation priority by each activation working field is suitable from high to low
Sequence, retains computing unit number predistribution computing unit, until all computing units to each activation working field according to its minimum successively
It is assigned；Into step 4).
Above-mentioned technical proposal is further characterized by, the step 4) specifically include following steps：
4.1) for all working domain, each working field before the computing unit number pre-allocated according to each working field and adjustment
Allocated computing unit number, the computing unit number that each working field participates in adjustment is calculated based on formula (7)：
Δsj=pj-rj(1≤j≤m) (7)
Wherein, pjIt is j-th working field preceding allocated normal operation computing unit number of this adjustment；rjIt is j-th work
Make the predistribution computing unit number in domain；ΔsjTo participate in the calculating list of switching adjustment after j-th working field this suboptimization distribution
First number, its value represents that j-th working field switching computing unit is used to other working fields more than 0, and its value represents other less than 0
Working field switching computing unit is used to j-th working field.
4.2) each working field that 0 is more than to the computing unit number that adjustment is participated in working field set, according to the work
The switching priority of each calculate node chooses a calculating section in normal operating condition successively in descending order in domain
Point, is designated as k, and the value for updating the computing unit number that the working field participates in adjustment is the computing unit number and u that it participates in adjustmentk
Difference；If the working field participates in the computing unit number of adjustment more than or equal to 0 after updating, calculate node k is added to be cut
Calculate node set is changed, continues to select next calculate node in normal operating condition in the working field, if should after updating
Working field participates in the computing unit number of adjustment, and less than 0 or in the working field, all calculate nodes are complete treatment, then turn step
It is rapid 4.3)；
4.3) 0 is less than to the computing unit number that adjustment is participated in working field set and this adjustment is preceding allocated normal
Each working field that operation computing unit number is equal to 0, according to each calculate node to be switched calculate node set
Switching priority chooses a calculate node from low to high, is designated as k, is switched to these working fields and uses, and updates these
The value that working field participates in the computing unit number of adjustment is the computing unit number and u that each working field participates in adjustmentkSum, while
The affiliated working field for updating calculate node k is these working fields；If to be switched calculate node collection is combined into sky, go to step 4.5)；
If 4.4) to be switched calculate node set non-NULL, go to step；
4.4) to participating in working field of the computing unit number of adjustment less than 0 in working field set according to its computing resource point
Descending arrangement is sequentially carried out from high to low with priority, for each of which working field, to be switched calculate node set
Switching priority according to each calculate node chooses a calculate node successively from low to high, is designated as k, is switched to these
Working field is used, and it is the calculating that each working field participates in adjustment to update the value of the computing unit number that these working fields participate in adjustment
Number of unit and ukSum, while the affiliated working field for updating calculate node k is these working fields, until these working fields are participated in
The computing unit number of adjustment is combined into sky more than or equal to 0 or to be switched calculate node collection, goes to step 4.5)；
4.5) for all activated working field, according to belonging to each calculate node work domain information and comprising meter
Unit number is calculated, the computing unit number of each activation actual distribution of working field is counted, and judges each actual point of activation working field
The computing unit number matched somebody with somebody retains the size of computing unit number with its minimum, to the unallocated meter for retaining computing unit number to minimum
The activation working field for calculating resource provides prompting, into step 5).
Beneficial effects of the present invention are as follows：Reference work cycle of the present invention based on each working field, with reference to each work
The real-time state of activation in domain and the real-time running state of each calculate node, then consider the most short calculating week of each working field
The constraint such as phase and permission calculating cycle most long, realizes the computing resource dynamically distributes of working field；On this basis by each
Calculate node is standardized according to computing unit, and further combined with before adjustment each working field and calculate node point
With relation, Adjustable calculation number of nodes calculate node optimization handover scheme as few as possible is finally given.The present invention is not only realized
The running status of state of activation and calculate node based on working field carries out the online dynamic optimization of many working field computing resources
Distribution, and make the calculate node quantity that participation is adjusted as few as possible.Therefore, the present invention can realize computing resource self adaptation pipe
On the basis of reason, it is also contemplated that calculate node switching to activating the influence of working field, effectively improve computing resource using effect
Rate and working field reliability of operation.
Brief description of the drawings
Fig. 1 is the inventive method multiple working field data interaction schematic diagram.
Fig. 2 is the schematic flow sheet 1 of the inventive method one embodiment.
Fig. 3 is the schematic flow sheet 2 of the inventive method one embodiment.
Specific embodiment
The present invention is described in further detail in conjunction with the embodiments with reference to the accompanying drawings.
General principle of the invention is：Corresponding reference work is set to the demand of computing resource based on each working field
Cycle, and as the computational resource allocation coefficient between different operating domain；Based on all kinds of computing resources pair that computing unit is included
Each calculate node is standardized, and determines effective computing unit number that each calculate node is included；According to each work
Make the real-time state of activation in domain and the real-time running state of each calculate node, consider the computing resource point of each working field
With priority and most short calculating cycle and the execution for allowing the constraint such as calculating cycle and calculating task in computing unit most long
Time, determine the computing unit quantity of each activation working field predistribution；On this basis according to calculate node include it is effective
Computing unit number, further combines the relations of distribution of each activation working field and calculate node before adjustment, is adjusted based on participation
Whole calculate node quantity principle as few as possible provides calculate node optimization handover scheme, realizes that computing resource works in multiple
Self-adaptative adjustment between domain.
Therefore, calculating demand of the present invention for each application function under the online and offline application scenarios of power system, will
Computer cluster is divided into multiple working fields, as shown in Figure 1.Each working field is independently calculated based on Distributed Calculation management platform
Taskings, scheduling and management, do not carry out calculating the interaction of data and control information directly between each working field in calculating process.
Flow of the invention is as shown in Figures 2 and 3.Step 1 is described in Fig. 2, according to calculating task to all kinds of calculating
The different demands of resource, all kinds of computing resource minimum combinations that will complete a calculating task calculating needs are defined as a calculating
Unit, is standardized based on the computing resource that computing unit is included to each calculate node according to formula (1), is counted
Calculation ability is measured again by the nonnegative integer of computing unit：
Wherein, n is cluster computing interstitial content, u in systemiIt is the computing unit number that i-th calculate node is included, its
It is nonnegative integer；Rcpu(i)、Rmem(i) and RioI () is respectively CPU, internal memory and the IO computing resources that i-th calculate node has；
Rcpu 0、Rmem 0And Rio 0It is respectively CPU, internal memory and IO computing resources that metric calculation unit has.
Step 2 is described in Fig. 2, and corresponding ginseng is set by hand for it to the demand of computing resource based on each working field
Work period, computational resource allocation priority, most short calculating cycle and permission calculating cycle parameter most long are examined, and according to single meter
Expected time of the calculation task on a computing unit, respectively will most short calculating cycle and most long according to formula (2) and (3)
Calculating cycle is allowed to be converted to maximum allocated computing unit number and minimum reservation computing unit number：
Wherein, m is working field number quantity, t in systemminJ () is j-th most short calculating cycle of working field, tmax(j)
It is j-th permission calculating cycle most long of working field, fjIt is the j-th application function quantity of working field needs operation, sjkIt is jth
The calculating task quantity of k-th application function of individual working field operation,It is k-th single calculating task of application function one
Expected time on individual computing unit,It is the oeprator that rounds up.
cmaxJ () is nonnegative integer, represent j-th maximum allocated computing unit number of working field；cminJ () is that non-negative is whole
Number, represents that j-th the minimum of working field retains computing unit number.
Step 3 is described in Fig. 2, when the state of activation in any operative domain or the running status of any calculate node occur
During change, following treatment is carried out：
3.1) all computing unit quantity sums included in normal operating condition calculate node are counted as treating point
With computing unit quantity cuInitial value, the predistribution computing unit number r of each working fieldjIt is initialized as 0；Count current institute
There is the minimum of activation working field to retain computing unit number sumIfGo to step
3.2)；Otherwise go to step 3.8)；Wherein m is working field number quantity, d in systemjIt is j-th state of activation of working field, its
It is worth for 1 represents working field activation, is worth for 0 represents the working field un-activation.
3.2) to each working field being active, dropped from high to low according to its computational resource allocation priority
Sequence arrange, the most preceding working field of selected and sorted as working field to be pre-allocated, into step 3.3)；
3.3) the predistribution computing unit number of working field to be pre-allocated is asked for according to formula (4)：
Wherein, rjIt is j-th predistribution computing unit number of working field；ciIt is i-th running status of calculate node,
ciValue for 1 expression the calculate node normally run, be worth for 0 represent the calculate node failure；ηjIt is j-th working field setting
In the reference work cycle, be the computational resource allocation coefficient between different operating domain；It is downward rounding operation symbol；
rjValue be nonnegative integer, used because a computing unit can only distribute to a working field in synchronization,
Round asking for by the result of calculation to formula (4).
3.4) r is judgedjWhether the working field maximum allocated computing unit number constraint of formula (5) definition is met, if rjIt is more than
Working field maximum allocated computing unit number cmax(j), then rjValue is cmax(j)；
rj≤cmax(j)(1≤j≤m) (5)
3.5) r is judgedjWhether computing unit quantity c to be allocated is less than or equal touIf, rjLess than or equal to cu, then c is updateduIt is cu-
rj；If rjMore than cu, then r is updatedjIt is cu, update cuIt is 0；
3.6) c is judgeduWhether it is 0, step 4 is entered if equal to 0, will otherwise comes this wait after the working field for pre-allocating
The working field of one gone to step as new working field to be pre-allocated 3.3) carry out it is next activation working field computing resource it is pre-
Distribution, until completing the computing resource predistribution of all activated working field；
If 3.7) computing unit quantity c to be allocated after the completion of the computing resource predistribution of all activated working fielduStill greater than
0, then to being not up to all activated working field that maximum allocated computing unit number is constrained, according to the computational resource allocation of each working field
Order is sequentially allocated a computing unit to priority to each activation working field from high to low, until all computing units all divide
Dispensing working field or the computing unit of all activated working field distribution all reach maximum allocated computing unit number；Into step 4；
3.8) for all activated working field, the computational resource allocation priority by each activation working field is suitable from high to low
Sequence, retains computing unit number predistribution computing unit, until all computing units to each activation working field according to its minimum successively
It is assigned；Into step 4.
Step 4 is described in Fig. 3, has been divided before the computing unit number of the predistribution according to each working field and its adjustment
The computing unit number matched somebody with somebody, the institute of each calculate node after adjustment is determined based on calculate node adjustment quantity principle as few as possible
Category working field, specific treatment is as follows：
4.1) for all working domain, according to the computing unit number r that each working field is pre-allocatedjWith each work before adjustment
The allocated computing unit number p in domainj, the computing unit number Δ s that each working field participates in adjustment is calculated based on formula (7)j
4.2) (1≤j≤m), go to step；
Δsj=pj-rj(1≤j≤m) (7)
In formula, pjIt is the working field j preceding allocated normal operation computing unit numbers of this adjustment；ΔsjIt is working field j sheets
The computing unit number of switching adjustment is participated in after suboptimization distribution, its value represents working field j switching computing units to other more than 0
Working field is used, and its value is used less than 0 expression other working fields switching computing unit to working field j.
4.2) to Δ s in working field setjEach working field j more than 0, according to each calculate node in the working field
Switching priority chooses a calculate node in normal operating condition successively in descending order, is designated as k, updates Δ sjFor
Δsj-ukIf, Δ sjMore than or equal to 0, then to be switched calculate node set, continuation is added to select in the working field calculate node k
Next calculate node in normal operating condition；If Δ sjLess than 0 or in the working field, all calculate nodes are complete
4.3) treatment, then go to step；
4.3) to Δ s in working field setjLess than 0 and pjEach working field j equal to 0, to be switched calculate node collection
Switching priority in conjunction according to each calculate node chooses a calculate node from low to high, is designated as k, is switched to work
Domain j is used, and updates Δ sjIt is Δ sj+uk, while the affiliated working field for updating calculate node k is j.If to be switched calculate node
Collection is combined into sky, then go to step 4.5)；If 4.4) to be switched calculate node set non-NULL, go to step；
4.4) to Δ s in working field setjWorking field less than 0 is suitable from high to low according to its computational resource allocation priority
Sequence carries out descending arrangement, for each of which working field j, according to each calculate node to be switched calculate node set
Switching priority chooses a calculate node successively from low to high, is designated as k, is switched to working field j and uses, and updates Δ sj
It is Δ sj+uk, while it is j to update these affiliated working fields for choosing calculate node, until Δ sjMore than or equal to 0 or to be switched meter
Operator node collection is combined into sky.Go to step 4.5)；
4.5) for all activated working field, according to belonging to each calculate node work domain information and comprising meter
Unit number is calculated, the computing unit number of each activation actual distribution of working field is counted, and updated to rj, enter according still further to formula (6)
The minimum of each activation working field of going retains computing unit number constraint checking, is calculated to minimum reservation computing unit number unallocated
The activation working field of resource provides prompting, into step 5.
rj≥cmin(j)(1≤j≤m) (6)
Step 5 is described in Fig. 3, and for all calculate nodes, the Distributed Calculation management in each calculate node is flat
Platform perceives its affiliated working field change information, and working field after adjusting is switched to from former working field, completes computing resource in multiple works
Make the self-adaptative adjustment between domain.
Although the present invention is disclosed as above with preferred embodiment, embodiment is not of the invention for limiting.Not
Depart from the spirit and scope of the present invention, any equivalence changes done or retouching also belong to the protection domain of the present invention.Cause
The content that this protection scope of the present invention should be defined with claims hereof is as standard.
Claims (5)
1. a kind of self-adapting regulation method of many working field computing resources, it is characterised in that comprise the following steps：
1) different demands of all kinds of computing resources will be completed by a calculating task and calculates all kinds of meters for needing according to calculating task
Calculate resource minimum combination and be defined as a computing unit, each calculate node is carried out according to the computing resource that computing unit is included
Standardization, its computing capability is measured again by the nonnegative integer of computing unit, into step 2)；
2) for the calculating demand of each application function under the online and offline application scenarios of power system, computer cluster is divided into
Multiple working fields, each working field independently carries out calculating task tissue, scheduling and manages based on Distributed Calculation management platform, calculates
During do not carry out calculating the interaction of data and control information between each working field directly；
Important level according to each working field and its requirement to calculating the time, be its unification set reference work cycle,
Computational resource allocation priority, most short calculating cycle and permission calculating cycle parameter most long, and according to single calculating task one
Expected time on individual computing unit, will most short calculating cycle and most it is long permission calculating cycle be converted to maximum allocated calculate
Unit number and minimum reservation computing unit number；
When the state of activation in any operative domain or the running status of any calculate node change, go to step 3)；
3) reference work cycle, computational resource allocation priority, the most short calculating cycle set based on each working field for activating
It is most long to allow calculating cycle parameter and the at present working field of activation and the normally calculate node of operation, ask for each activation
The computing unit quantity of working field predistribution；If currently available computing unit sum is less than or equal to current all activated working field
It is minimum retain computing unit number sum, then the order from high to low of the computational resource allocation priority according to each activation working field,
Retain computing unit number according to its minimum carries out computing unit predistribution successively, until all computing units are assigned；
4) it is each before computing unit number and adjustment based on the predistribution of each working field for all working domain for completing predistribution
The allocated normal operation computing unit number of working field calculates the normal operation computing unit that each working field participates in adjustment
Number, and based on participate in adjustment calculate node quantity as far as possible lack principle, with reference to each calculate node switching priority and
The computing unit number that it is included determines the affiliated working field of each calculate node after adjustment；And retain meter to minimum to unallocated
The activation working field for calculating unit number computing resource provides prompting；
5) the Distributed Calculation management platform in each calculate node perceives working field change information, and tune is switched to from former working field
Whole rear working field, completes self-adaptative adjustment of the calculate node between multiple working fields.
2. the self-adapting regulation method of many working field computing resources according to claim 1, it is characterised in that the step
1) computing resource that each calculate node has according to it in carries out computing unit standardization according to formula (1), it is determined that often
Effective computing unit number that individual calculate node is included, so as to be measured to its computing capability：
Wherein, n is cluster computing interstitial content, u in systemiFor the computing unit number that i-th calculate node is included, it is non-
Negative integer；
Rcpu(i)、Rmem(i) and RioI () is respectively CPU, internal memory and the IO computing resources that i-th calculate node has；
Rcpu 0、Rmem 0And Rio 0It is respectively CPU, internal memory and IO computing resources that metric calculation unit has.
3. the self-adapting regulation method of many working field computing resources according to claim 1, it is characterised in that the step
2) the most short calculating cycle of each working field is converted to most with permission calculating cycle most long respectively by formula (2) and (3) in
Big distribution computing unit number and minimum reservation computing unit number：
Wherein, m is working field number quantity, t in systemminJ () is j-th most short calculating cycle of working field, tmaxJ () is the
The j permission calculating cycle most long of working field, fjIt is the j-th application function quantity of working field needs operation, sjkIt is j-th work
Make the calculating task quantity of k-th application function of domain operation,It is that k-th single calculating task of application function is counted at one
The expected time on unit is calculated,It is the oeprator that rounds up；
cmaxJ () is nonnegative integer, represent j-th maximum allocated computing unit number of working field；cminJ () is nonnegative integer, table
Show that j-th the minimum of working field retains computing unit number.
4. the self-adapting regulation method of many working field computing resources according to claim 1, it is characterised in that the step
3) following steps are specifically included：
3.1) using all normal computing unit quantity sums for being included of operation calculate nodes as computing unit quantity c to be allocatedu
Initial value, the predistribution computing unit number of each working field is initialized as 0；The current all activated working field of statistics is most
Small reservation computing unit number sumWherein m is working field number quantity, d in systemjIt is j-th working field
State of activation, its value be 1 represent the working field activation, be worth for 0 represent the working field un-activation；
IfGo to step 3.2)；Otherwise go to step 3.8)；
3.2) to each working field being active, descending row is carried out from high to low according to its computational resource allocation priority
Row, the most preceding working field of selected and sorted as working field to be pre-allocated, into step 3.3)；
3.3) the predistribution computing unit number of working field to be pre-allocated is asked for according to formula (4)：
Wherein, rjIt is j-th predistribution computing unit number of working field；ciIt is i-th running status of calculate node, ci's
It is worth for 1 represents that the calculate node is normally run, is worth for 0 represents the calculate node failure；ηjIt is j-th reference of working field setting
Work period, is the computational resource allocation coefficient between different operating domain；It is downward rounding operation symbol；
If 3.4) the predistribution computing unit number of working field to be pre-allocated is more than the working field maximum allocated computing unit number
Mesh, then be updated to the value of the predistribution computing unit number of the working field to be pre-allocated the working field maximum allocated and calculate single
First number；
3.5) judge the predistribution computing unit number of working field to be pre-allocated whether less than or equal to computing unit number to be allocated
Amount cuIf being less than or equal to cuThen by cuValue be updated to cuThe predistribution computing unit number of the working field to be pre-allocated is subtracted,
The value of the predistribution computing unit number of working field to be pre-allocated otherwise is updated to cuAnd update cuValue be 0；
3.6) c is judgeduWhether it is 0, step 4 is entered if equal to 0), will otherwise come latter of this working field to be pre-allocated
Working field gone to step as new working field to be pre-allocated 3.3) carry out it is next activation working field computing resource predistribution,
Until completing the computing resource predistribution of all activated working field；
If 3.7) computing unit quantity c to be allocated after the completion of the computing resource predistribution of all activated working fielduIt is still greater than 0, then right
The all activated working field that not up to maximum allocated computing unit number is constrained, according to the computational resource allocation priority of each working field
Order is sequentially allocated a computing unit to each activation working field from high to low, until all computing units are fully allocated to work
The computing unit for making domain or the distribution of all activated working field all reaches maximum allocated computing unit number；Into step 4)；
3.8) for all activated working field, by it is each activation working field computational resource allocation priority from high to low order, according to
It is secondary to retain computing unit number predistribution computing unit according to its minimum to each activation working field, until all computing units are distributed
Finish；Into step 4).
5. the self-adapting regulation method of many working field computing resources according to claim 1, it is characterised in that the step
4) following steps are specifically included：
4.1) for all working domain, each working field has divided before the computing unit number pre-allocated according to each working field and adjustment
The computing unit number matched somebody with somebody, the computing unit number that each working field participates in adjustment is calculated based on formula (7)：
Δsj=pj-rj(1≤j≤m) (7)
Wherein, pjIt is j-th working field preceding allocated normal operation computing unit number of this adjustment；rjIt is j-th working field
Predistribution computing unit number；ΔsjTo participate in the computing unit number of switching adjustment after j-th working field this suboptimization distribution
Mesh, its value represents that j-th working field switching computing unit is used to other working fields more than 0, and its value represents other work less than 0
Domain switching computing unit is used to j-th working field.
4.2) each working field that 0 is more than to the computing unit number that adjustment is participated in working field set, according in the working field
The switching priority of each calculate node chooses a calculate node in normal operating condition, note successively in descending order
It is k, the value for updating the computing unit number that the working field participates in adjustment is the computing unit number and u that it participates in adjustmentkDifference
Value；If the working field participates in the computing unit number of adjustment more than or equal to 0 after updating, calculate node k is added into be switched meter
Operator node set, continues to select next calculate node in normal operating condition in the working field, if the work after updating
Domain participates in the computing unit number of adjustment, and less than 0 or in the working field, all calculate nodes are complete treatment, then go to step
4.3)；
4.3) allocated normal operation before 0 and this adjustment is less than to the computing unit number that adjustment is participated in working field set
Each working field that computing unit number is equal to 0, according to the switching of each calculate node to be switched calculate node set
Priority chooses a calculate node from low to high, is designated as k, is switched to these working fields and uses, and updates these work
The value that domain participates in the computing unit number of adjustment is the computing unit number and u that each working field participates in adjustmentkSum, while update
The affiliated working field of calculate node k is these working fields；If to be switched calculate node collection is combined into sky, go to step 4.5)；If treating
4.4) switching calculate node set non-NULL, go to step；
4.4) it is excellent according to its computational resource allocation to participating in the working field of the computing unit number less than 0 of adjustment in working field set
First level sequentially carries out descending arrangement from high to low, for each of which working field, the basis to be switched calculate node set
The switching priority of each calculate node chooses a calculate node successively from low to high, is designated as k, is switched to these work
Domain uses, and it is the computing unit that each working field participates in adjustment to update the value of the computing unit number that these working fields participate in adjustment
Number and ukSum, while the affiliated working field for updating calculate node k is these working fields, until these working fields participate in adjusting
Computing unit number be combined into sky more than or equal to 0 or to be switched calculate node collection, go to step 4.5)；
4.5) for all activated working field, according to belonging to each calculate node work domain information and comprising calculating list
First number, counts the computing unit number of each activation actual distribution of working field, and judges each actual distribution of activation working field
Computing unit number retains the size of computing unit number with its minimum, to the unallocated calculating money for retaining computing unit number to minimum
The activation working field in source provides prompting, into step 5).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611048286.6A CN106708624B (en) | 2016-11-25 | 2016-11-25 | Self-adaptive adjustment method for multi-working-domain computing resources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611048286.6A CN106708624B (en) | 2016-11-25 | 2016-11-25 | Self-adaptive adjustment method for multi-working-domain computing resources |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106708624A true CN106708624A (en) | 2017-05-24 |
| CN106708624B CN106708624B (en) | 2020-08-11 |
Family
ID=58934953
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611048286.6A Active CN106708624B (en) | 2016-11-25 | 2016-11-25 | Self-adaptive adjustment method for multi-working-domain computing resources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106708624B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110738322A (en) * | 2018-07-03 | 2020-01-31 | 杭州海康威视数字技术股份有限公司 | distributed training method, device, equipment and system |
| CN111753997A (en) * | 2020-06-28 | 2020-10-09 | 北京百度网讯科技有限公司 | Distributed training method, system, device and storage medium |
| CN112988372A (en) * | 2019-12-16 | 2021-06-18 | 杭州海康威视数字技术股份有限公司 | Method and device for determining distribution mode of hardware operation platform |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060195578A1 (en) * | 2005-02-28 | 2006-08-31 | Fujitsu Limited | Resource allocation method for network area and allocation program therefor, and network system |
| US7900206B1 (en) * | 2004-03-31 | 2011-03-01 | Symantec Operating Corporation | Information technology process workflow for data centers |
| CN102325054A (en) * | 2011-10-18 | 2012-01-18 | 国网电力科学研究院 | Self-adaptive adjusting method for hierarchy management of distributed type calculation management platform cluster |
| CN102063336B (en) * | 2011-01-12 | 2013-02-27 | 国网电力科学研究院 | Distributed computing multiple application function asynchronous concurrent scheduling method |
| CN104598318A (en) * | 2014-12-30 | 2015-05-06 | 北京奇艺世纪科技有限公司 | Node calculating capability reporting method and calculating node |
-
2016
- 2016-11-25 CN CN201611048286.6A patent/CN106708624B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7900206B1 (en) * | 2004-03-31 | 2011-03-01 | Symantec Operating Corporation | Information technology process workflow for data centers |
| US20060195578A1 (en) * | 2005-02-28 | 2006-08-31 | Fujitsu Limited | Resource allocation method for network area and allocation program therefor, and network system |
| CN102063336B (en) * | 2011-01-12 | 2013-02-27 | 国网电力科学研究院 | Distributed computing multiple application function asynchronous concurrent scheduling method |
| CN102325054A (en) * | 2011-10-18 | 2012-01-18 | 国网电力科学研究院 | Self-adaptive adjusting method for hierarchy management of distributed type calculation management platform cluster |
| CN104598318A (en) * | 2014-12-30 | 2015-05-06 | 北京奇艺世纪科技有限公司 | Node calculating capability reporting method and calculating node |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110738322A (en) * | 2018-07-03 | 2020-01-31 | 杭州海康威视数字技术股份有限公司 | distributed training method, device, equipment and system |
| CN110738322B (en) * | 2018-07-03 | 2023-06-02 | 杭州海康威视数字技术股份有限公司 | Distributed training method, device, equipment and system |
| CN112988372A (en) * | 2019-12-16 | 2021-06-18 | 杭州海康威视数字技术股份有限公司 | Method and device for determining distribution mode of hardware operation platform |
| CN112988372B (en) * | 2019-12-16 | 2023-10-24 | 杭州海康威视数字技术股份有限公司 | Method and device for determining allocation mode of hardware operation platform |
| CN111753997A (en) * | 2020-06-28 | 2020-10-09 | 北京百度网讯科技有限公司 | Distributed training method, system, device and storage medium |
| CN111753997B (en) * | 2020-06-28 | 2021-08-27 | 北京百度网讯科技有限公司 | Distributed training method, system, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106708624B (en) | 2020-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102063336B (en) | Distributed computing multiple application function asynchronous concurrent scheduling method | |
| Stankovic | An application of bayesian decision theory to decentralized control of job scheduling | |
| CN103164261B (en) | Multicenter data task disposal route, Apparatus and system | |
| CN107580023A (en) | A kind of the stream process job scheduling method and system of dynamic adjustment task distribution | |
| CN110555785B (en) | Monthly plan safety and stability checking method and system | |
| CN103927231B (en) | The energy optimization data set distribution method that a kind of data-oriented processes | |
| CN110597639B (en) | CPU distribution control method, device, server and storage medium | |
| CN104298550A (en) | Hadoop-oriented dynamic scheduling method | |
| CN104808770A (en) | Data center energy consumption management method and system based on dynamic frequency modulation | |
| CN106708624A (en) | Adaptive adjustment method for calculation resources in multiple working areas | |
| CN108270805A (en) | For the resource allocation methods and device of data processing | |
| CN108536539A (en) | A kind of method for scheduling task in industrial allocation formula data collecting system | |
| CN113485801A (en) | Real-time DNN scheduling system and method based on neural network similarity modeling | |
| CN103248622B (en) | A kind of Online Video QoS guarantee method of automatic telescopic and system | |
| CN108123987A (en) | The method and device of master scheduler is determined from cloud computing system | |
| CN109165045A (en) | A kind of method and apparatus for the hardware configuration adjusting server | |
| CN110196773B (en) | Multi-time-scale security check system and method for unified scheduling computing resources | |
| Li et al. | MapReduce task scheduling in heterogeneous geo-distributed data centers | |
| CN103325012A (en) | Parallel computing dynamic task distribution method applicable to grid security correction | |
| CN104869154A (en) | Distributed resource scheduling method for balancing resource credibility and user satisfaction | |
| CN107729218A (en) | A kind of system and method for monitoring processing computing resource equipment | |
| CN110262880A (en) | A kind of job scheduling method of Based on Distributed consumption of data center expense optimization | |
| Xiang et al. | Consensus mechanism of virtual power plant transaction blockchain based on credit value | |
| CN115622087A (en) | Power regulation and control method, device and equipment for power distribution network | |
| CN102254245A (en) | Method for checking safety and stability of scheduling daily plans of power system in time-interval sharing parallel mode |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |