CN108712305A - A kind of Coflow dispatching methods based on subflow flow value method of estimation - Google Patents
A kind of Coflow dispatching methods based on subflow flow value method of estimation Download PDFInfo
- Publication number
- CN108712305A CN108712305A CN201810420239.2A CN201810420239A CN108712305A CN 108712305 A CN108712305 A CN 108712305A CN 201810420239 A CN201810420239 A CN 201810420239A CN 108712305 A CN108712305 A CN 108712305A
- Authority
- CN
- China
- Prior art keywords
- coflow
- priority
- message
- subflows
- local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0882—Utilisation of link capacity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
- H04L47/52—Queue scheduling by attributing bandwidth to queues
- H04L47/527—Quantum based scheduling, e.g. credit or deficit based scheduling or token bank
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
- H04L47/62—Queue scheduling characterised by scheduling criteria
- H04L47/625—Queue scheduling characterised by scheduling criteria for service slots or service orders
- H04L47/6275—Queue scheduling characterised by scheduling criteria for service slots or service orders based on priority
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of Coflow dispatching methods based on subflow flow value method of estimation, include the following steps：S1, the estimated value of Coflow subflow flows is periodically reported to controller；S2：It by controller side Coflow priority formulating methods, calculates Coflow and corresponds to priority, and downward message is to each host；S3, host computer side update Coflow priority map and each Coflow priority correspond to the virtual transmission deadline, generate data message according to Coflow precedence informations；S4, host computer side carry out processing of joining the team to data message, obtain the storage queue of Coflow messages；S5, team's processing being carried out out to data message, obtaining schedules message, realization is scheduled Coflow.The present invention solves the problems, such as that shortage accuracy, the effect of optimization for influencing scheduling strategy, scheme shortage reasonability and optimization property of the existing technology and treatment effeciency are low.
Description
Technical field
The invention belongs to Internet technical fields, and in particular to a kind of Coflow based on subflow flow value method of estimation
Dispatching method.
Background technology
Data center refers to the large-scale special purpose computer cluster possessed by professional institution, usually has different scales, with full
The various use demand of foot.Wherein number of servers crosses ten thousand data center usually by some large-scale online service providers
(such as Google, Microsoft and Amazon) is for running some large-scale data intensive tasks, such as webpage rope
Draw or large data set analysis.These data-intensive tasks are usually by MapReduce patterns (or various mutation) in number
According to carrying out parallel calculation processing between the cluster at center, and parallel synergistic data stream, such association are generated in processing procedure
It is usually described with Coflow models with data flow.Coflow is defined as one group of collaboration number with common performance objective
According to the set of flow, typically refers to same item parallel computation and apply the set for generating synergistic data stream in stage of communication.In set
Every independent flow is referred to as a subflow of this Coflow.The deadline of Coflow transfers depending on its neutron current
At the maximum value of time.Since being transmitted on the time for Coflow has concertedness, traditional network flow dispatches correlation means
It is poorly suited for use in Coflow, therefore, it is necessary to study the flow scheduling scheme suitable for Coflow, so as to excellent
That changes Coflow is transmitted the time, to improve the processing capacity that data center applies parallel computation.In fact,
Coflow scheduling schemes mainly need to solve two problems：(1) each flow in same Coflow how bandwidth allocation.(2) different
Flow in Coflow how bandwidth allocation.
In recent years, the optimization problem of data center Coflow deadlines has attracted numerous researchers to pay close attention to.It is based on
The research of the knowability of Coflow relevant informations, this respect can be further classified as two classes --- and priori is known to be adjusted
The unknown scheduling mechanism of degree mechanism, priori.Scheduling mechanism known to priori can be known complete when referring to decision
Coflow features include the quantity of Coflow neutron currents, the size of each subflow and duration etc..Priori is unknown
Scheduling mechanism can not know complete Coflow features when referring to decision, or not can know that any feature of Coflow completely
Information.
In the different research scene of above-mentioned two, information of the scheduling mechanism known to priori due to possessing abundance,
Thinking is typically to minimize Coflow average completion times as object function, according to network topology, network capacity and business
Demand establishes constraints, establishes optimization problem model.Scheduling scheme under this scene generally can all obtain more in detail
Scheduling strategy, including every subflow of Coflow routing is specified and the bandwidth allocation in respective path, effect of optimization compared with
It is good.But the Coflow scheduling mechanisms under the scene usually require to assume that host function is powerful enough, are capable of providing complete
Coflow information, this is typically unpractiaca, is not suitable for real data central site network environment.
Problem of the existing technology is as follows：
(1) the message accumulated size of the prior art being only sent completely as Coflow size estimations method excessively
It is coarse, Given information is not made full use of, the accumulative situations of Coflow currently in a network is had ignored, causes big to Coflow
Small estimation lacks accuracy, influences the effect of optimization of scheduling strategy；
(2) prior art and only in accordance with Coflow sizes formulate Coflow priority, scheme lack reasonability；
(3) prior art is that Coflow formulations priority is unfavorable for reducing under some scenes according to Coflow sizes
Coflow average completion times, optimization property and treatment effeciency are low.
Invention content
For above-mentioned deficiency in the prior art, that the present invention provides a kind of accuracys is high, optimization property is high, reasonability is high,
High efficiency and the high Coflow dispatching methods based on subflow flow value method of estimation for the treatment of effeciency, solve the prior art and deposit
Shortage accuracy, influence the effect of optimization of scheduling strategy, scheme lacks reasonability and optimization property and treatment effeciency it is low
Problem.
In order to reach foregoing invention purpose, the technical solution adopted by the present invention is：
A kind of Coflow dispatching methods based on subflow flow value method of estimation, include the following steps：
S1：Using Coflow subflow flow value methods of estimation, Coflow subflows flow value periodically each to host computer side carries out
Estimation, and the estimated value of Coflow subflow flows is periodically reported to controller；
S2：It is calculated by controller side Coflow priority formulating methods according to the estimated value of Coflow subflow flows
Coflow corresponds to priority, and issues the message for carrying Coflow precedence informations to each host；
S3：The Coflow precedence informations carried in the message that host computer side is issued according to controller, use controller report
Literary processing method adjusts local Coflow priority corresponding relations, updates Coflow priority map and each Coflow priority
The corresponding virtual transmission deadline, generate data message；
S4：Processing of joining the team is carried out to the data message that step S3 is generated in host computer side, obtains the storage team of Coflow messages
Row；
S5：Virtual transmission deadline, logarithm are corresponded to according to the obtained storage queues of step S4 and each Coflow priority
Team's processing is carried out out according to message, obtains schedules message, realization is scheduled Coflow.
Further, in step S1, Coflow subflow flow value methods of estimation include the following steps：
S1-1：Judge whether that reach the default report cycle of host computer side executes host computer side if then entering step S1-2
It counts and reports, otherwise enter step S1-4；
S1-2：Calculate the estimated value of local Coflow subflows；
S1-3：The aggregate-value zero of message will be reached in the Coflow subflow periods, and reset and preset report cycle, enter
Step S1-1；
S1-4：The aggregate-value of arrival message flow, Coflow subflows in the Coflow subflow periods of host computer side is obtained to pass
Message flow aggregate-value and Coflow subflows in defeated queue are transmitted message flow aggregate-value；
S1-5：Judge whether to reach to preset and report emulation duration, if then terminating reporting for host computer side, otherwise enters step
Rapid S1-1.
Further, in step S1-2, the calculation formula of the estimated value of local Coflow subflows is：
In formula,It is flow estimation value of the local Coflow subflows in transmission end main frame statistics；For Coflow
Flow the accumulative message flow magnitude in sending end main frame transmission queue；Sending what end main frame counted on for Coflow subflows
It is transmitted message integrated flow value；The tired of message is reached within the transmission end main frame period for Coflow subflows
Evaluation.
Further, in step S2, controller side Coflow priority formulating methods include the following steps：
S2-1：Judge whether otherwise reach the controller algorithm default execution period enters if then entering step S2-2
Step S2-4；
S2-2：According to the estimated value of local Coflow subflows flow, the corresponding bottleneck values of Coflow are calculated, and combine bottle
Neck threshold value carries out each Coflow the division of priority, obtains the correspondence priority of each Coflow；
S2-3：It in the correspondence precedence level code to message of each Coflow, will be issued to each host, and reset default execute
Period, and enter step S2-1；
S2-4：The estimated value for counting the local Coflow subflows flow that each host reports, obtains local Coflow subflows
Send the reception total estimates of total estimates and local Coflow subflows；
S2-5：Judge whether to reach the default emulation duration of algorithm, if then finishing control device side control algolithm, otherwise into
Enter step S2-1.
Further, in step S2-2, the calculation formula of the corresponding bottleneck values of Coflow is：
In formula, bottleneck (c) is the corresponding bottleneck values of Coflow；For the transmission of local Coflow subflows
Total estimates；For the reception total estimates of local Coflow subflows；
The calculation formula of the reception total estimates of local Coflow subflows is：
In formula,For the reception total estimates of local Coflow subflows；For local Coflow
The reception estimated value of stream；I ∈ I_r (n, c) indicate to receive the set for the stream for belonging to Coflow c in end main frame n；
The calculation formula of the transmission total estimates of local Coflow subflows is：
In formula,For the transmission total estimates of local Coflow subflows；For the hair of local Coflow subflows
Send estimated value；I (n, c) is the set for sending the stream for belonging to Coflow c in end main frame n.
Further, in step S3, controller message processing method includes the following steps：
S3-1：The content of message in host computer side analyzing step S2 obtains the corresponding priority of each Coflow；
S3-2：According to the correspondence of Coflow and its priority, the Coflow ID that each priority includes are obtained, and
According to the sequential update Coflow priority map of Coflow arrival times；
S3-3：Judge whether Coflow priority map changes, if then entering step S3-4, does not otherwise update Coflow
Priority corresponding virtual transmission deadline, and ending method；
S3-4：According to WFQ dispatching methods, each Coflow priority corresponding virtual transmission deadline is calculated.
Further, in step S4, data message is joined the team the method for processing, is included the following steps：
S4-1：Judge whether host locally has the record of the affiliated Coflow of the data message, if then entering step S4-
5, otherwise enter step S4-2；
S4-2：Judge that highest priority corresponds to whether queue is empty in Coflow priority map, if then entering step
Otherwise S4-3 does not update Coflow highest priorities and corresponds to the virtual transmission deadline, and enters step S4-4；
S4-3：According to WFQ dispatching methods, calculates Coflow highest priorities and correspond to the virtual transmission deadline；
S4-4：The priority of the Coflow is set as highest priority, and updates the Coflow priority of host local
map；
S4-5：The arrival time of data message and the Coflow data messages is stored in deposit corresponding with Coflow numbers
It stores up in queue, and exports storage queue.
8. the Coflow dispatching methods according to claim 7 based on subflow flow value method of estimation, feature exist
In in the step S4-3, the calculation formula that Coflow highest priorities correspond to the virtual transmission deadline is：
In formula, V_finish[i]The virtual transmission deadline is corresponded to for Coflow priority；A[i]For positioned at first place
Coflow corresponds to the arrival time of storage queue squadron first message；P[i]To be corresponded in storage queue positioned at the first Coflow
The message size value of head of the queue message；w[i]For Coflow priority i respective weights；w[l]For the corresponding power of Coflow priority l
Weight；R is physical link rate.
Further, in step S5, the method that data message goes out team's processing includes the following steps：
S5-1：The virtual transmission deadline is corresponded to according to WFQ dispatching methods and each Coflow priority, is selected corresponding empty
The quasi- priority for being transmitted time minimum is as epicycle Coflow priority to be dispatched；
S5-2：The Coflow arrival times in the Coflow that dispatching priority includes are waited for according to epicycle, obtain arrival time
Earliest Coflow is used as Coflow to be dispatched, number to be used as TRAN_ID；
S5-3：It selects number for the Coflow storage queues of TRAN_ID, therefrom selects the pop-up of head of the queue message, output scheduling
Message.
S5-4：Corresponding Coflow messages are in the epicycle Coflow priority to be dispatched obtained in judgment step S5-1
No is sky, if do not update the virtual transmission deadline of epicycle Coflow priority to be dispatched then, and ending method, otherwise
Enter step S5-5；
S5-5：According to WFQ dispatching methods, the virtual transmission deadline of epicycle Coflow priority to be dispatched is calculated.
Further, the calculation formula of the virtual transmission deadline of Coflow priority is：
In formula, V_finish[i]The virtual transmission deadline is corresponded to for Coflow priority；V_start[i]For Coflow
Priority corresponds to the virtual transmission time started；P[i]To correspond to the report of storage queue squadron first message positioned at the first Coflow
Literary sizes values；w[i]For Coflow priority i respective weights；w[l]For the respective weights of Coflow priority l；R is practical chain
Road rate；
V_start[i]=max { A[i],V_pre[i]}
In formula, V_start[i]The virtual transmission time started is corresponded to for Coflow priority；A[i]For positioned at first place
Coflow corresponds to the arrival time of storage queue squadron first message；V_pre[i]For on priority i in Coflow priority map
The virtual transmission deadline once calculated.
This programme has the beneficial effect that：
(1) present invention formulates Coflow priority according to Coflow bottleneck sizes, makes full use of Given information, improves
Accuracy；
(2) method of estimation for improving Coflow subflow sizes improves the reasonability of scheme；
(3) the average Coflow deadlines are effectively reduced, the treatment effeciency of parallel computation application is improved；
(4) Coflow scheduling mechanisms proposed by the invention, deployment is simple, and algorithm complexity is low, can be applied to big business
Scene is measured, realizes the efficient Coflow scheduling of data center.
Description of the drawings
Fig. 1 is the Coflow dispatching method flow charts based on subflow flow value method of estimation；
Fig. 2 is Coflow subflow flow value method of estimation flow charts；
Fig. 3 device side Coflow priority formulating method flow charts in order to control；
Fig. 4 device message processing method flow charts in order to control；
Fig. 5 is that data message is joined the team the method flow diagram of processing；
Fig. 6 is the method flow diagram that data message goes out team's processing.
Specific implementation mode
The specific implementation mode of the present invention is described below, in order to facilitate understanding by those skilled in the art originally
Invention carrys out those skilled in the art it should be apparent that the present invention is not limited to the range of specific implementation mode
It says, as long as various change is in the spirit and scope of the present invention that the attached claims limit and determine, these variations are aobvious
And be clear to, all are using the innovation and creation of present inventive concept in the row of protection.
In the embodiment of the present invention, a kind of Coflow dispatching methods based on subflow flow value method of estimation, as shown in Fig. 1,
Include the following steps：
S1：Using Coflow subflow flow value methods of estimation, Coflow subflows flow value periodically each to host computer side carries out
Estimation, and the estimated value of Coflow subflow flows is periodically reported to controller；
Coflow subflow flow value methods of estimation, as shown in Fig. 2, including the following steps：
S1-1：Judge whether that reach the default report cycle of host computer side executes host computer side if then entering step S1-2
It counts and reports, otherwise enter step S1-4；
S1-2：Calculate the estimated value of local Coflow subflows；
The calculation formula of the estimated value of local Coflow subflows is：
In formula,It is flow estimation value of the local Coflow subflows in transmission end main frame statistics；For Coflow
Flow the accumulative message flow magnitude in sending end main frame transmission queue；Sending what end main frame counted on for Coflow subflows
It is transmitted message integrated flow value；The tired of message is reached within the transmission end main frame period for Coflow subflows
Evaluation；
S1-3：The aggregate-value zero of message will be reached in the Coflow subflow periods, and reset and preset report cycle, enter
Step S1-1；
S1-4：The aggregate-value of arrival message flow, Coflow subflows in the Coflow subflow periods of host computer side is obtained to pass
Message flow aggregate-value and Coflow subflows in defeated queue are transmitted message flow aggregate-value；
S1-5：Judge whether to reach to preset and report emulation duration, if then terminating reporting for host computer side, otherwise enters step
Rapid S1-1；
When Coflow subflow flow value methods of estimation are run, interior reach will not be recycled at first and presets report cycle,
To which method is directly entered step S1-4 by step S1-1, obtain reaching message flow in the Coflow subflow periods of host computer side
Message flow aggregate-value and Coflow subflows in transmitting queue of aggregate-value, Coflow subflows be transmitted message flow
Aggregate-value, default report cycle, which is less than to preset, reports emulation duration, reenters step S1-1, is preset when reaching host computer side
When reporting the period, the calculating of the estimated value of local Coflow subflows is carried out；
S2：It is calculated by controller side Coflow priority formulating methods according to the estimated value of Coflow subflow flows
Coflow corresponds to priority, and issues the message for carrying Coflow precedence informations to each host；
Controller side Coflow priority formulating methods, as shown in figure 3, including the following steps：
S2-1：Judge whether otherwise reach the controller algorithm default execution period enters if then entering step S2-2
Step S2-4；
S2-2：According to the estimated value of local Coflow subflows flow, the corresponding bottleneck values of Coflow are calculated, and combine bottle
Neck threshold value carries out each Coflow the division of priority, obtains the correspondence priority of each Coflow；
The calculation formula of the corresponding bottleneck values of Coflow is：
In formula, bottleneck (c) is the corresponding bottleneck values of Coflow；For the transmission of local Coflow subflows
Total estimates；For the reception total estimates of local Coflow subflows；
The calculation formula of the reception total estimates of local Coflow subflows is：
In formula,For the reception total estimates of local Coflow subflows；For local Coflow
The reception estimated value of stream；I ∈ I_r (n, c) indicate to receive the set for the stream for belonging to Coflow c in end main frame n；
The calculation formula of the transmission total estimates of local Coflow subflows is：
In formula,For the transmission total estimates of local Coflow subflows；For the hair of local Coflow subflows
Send estimated value；I (n, c) is the set for sending the stream for belonging to Coflow c in end main frame n；
S2-3：It in the correspondence precedence level code to message of each Coflow, will be issued to each host, and reset default execute
Period, and enter step S2-1；
S2-4：The estimated value for counting the local Coflow subflows flow that each host reports, obtains local Coflow subflows
Send the reception total estimates of total estimates and local Coflow subflows；
S2-5：Judge whether to reach the default emulation duration of algorithm, if then finishing control device side control algolithm, otherwise into
Enter step S2-1；
When controller side Coflow priority formulating methods are run, the interior controller that reaches will not be recycled at first and is calculated
Method is default to execute the period, to which method is directly entered step S2-4 by step S2-1, counts the local Coflow that each host reports
The estimated value of subflow flow, the reception for obtaining the transmission total estimates and local Coflow subflows of local Coflow subflows are always estimated
Evaluation is preset and executes the period less than the default emulation duration of algorithm, reenters step S2-1, default when reaching controller algorithm
When executing the period, according to the estimated value of local Coflow subflows flow, the corresponding bottleneck values of Coflow are calculated, and combine bottleneck
Threshold value carries out each Coflow the division of priority, obtains the correspondence priority of each Coflow；
S3：The Coflow precedence informations carried in the message that host computer side is issued according to controller, use controller report
Literary processing method adjusts local Coflow priority corresponding relations, updates Coflow priority map and each Coflow priority
The corresponding virtual transmission deadline, generate data message；
Controller message processing method, as shown in figure 4, including the following steps：
S3-1：The content of message in host computer side analyzing step S2 obtains the corresponding priority of each Coflow；
S3-2：According to the correspondence of Coflow and its priority, the Coflow ID that each priority includes are obtained, and
According to the sequential update Coflow priority map of Coflow arrival times；
S3-3：Judge whether Coflow priority map changes, if then entering step S3-4, does not otherwise update Coflow
Priority corresponding virtual transmission deadline, and ending method；
S3-4：According to WFQ dispatching methods, each Coflow priority corresponding virtual transmission deadline is calculated；
The calculation formula of the virtual transmission deadline of Coflow priority is：
In formula, V_finish[i]The virtual transmission deadline is corresponded to for Coflow priority；V_start[i]For Coflow
Priority corresponds to the virtual transmission time started；P[i]To correspond to the report of storage queue squadron first message positioned at the first Coflow
Literary sizes values；w[i]For Coflow priority i respective weights；w[l]For the respective weights of Coflow priority l；R is practical chain
Road rate；
V_start[i]=max { A[i],V_pre[i]}
In formula, V_start[i]The virtual transmission time started is corresponded to for Coflow priority；A[i]For positioned at first place
Coflow corresponds to the arrival time of storage queue squadron first message；V_pre[i]For on priority i in Coflow priority map
The virtual transmission deadline once calculated；
S4：Processing of joining the team is carried out to the data message that step S3 is generated in host computer side, obtains the storage team of Coflow messages
Row；
Data message is joined the team the method for processing, as shown in figure 5, including the following steps：
S4-1：Judge whether host locally has the record of the affiliated Coflow of the data message, if then entering step S4-
5, otherwise enter step S4-2；
S4-2：Judge that highest priority corresponds to whether queue is empty in Coflow priority map, if then entering step
Otherwise S4-3 does not update Coflow highest priorities and corresponds to the virtual transmission deadline, and enters step S4-4；
S4-3：According to WFQ dispatching methods, calculates Coflow highest priorities and correspond to the virtual transmission deadline；
The calculation formula that Coflow highest priorities correspond to the virtual transmission deadline is：
In formula, V_finish[i]The virtual transmission deadline is corresponded to for Coflow priority；A[i]For positioned at first place
Coflow corresponds to the arrival time of storage queue squadron first message；P[i]To be corresponded in storage queue positioned at the first Coflow
The message size value of head of the queue message；w[i]For Coflow priority i respective weights；w[l]For the corresponding power of Coflow priority l
Weight；R is physical link rate；
S4-4：The priority of the Coflow is set as highest priority, and updates the Coflow priority of host local
map；
S4-5：The arrival time of data message and the Coflow data messages is stored in deposit corresponding with Coflow numbers
It stores up in queue, and exports storage queue；
S5：Virtual transmission deadline, logarithm are corresponded to according to the obtained storage queues of step S4 and each Coflow priority
Team's processing is carried out out according to message, obtains schedules message, realization is scheduled Coflow；
The method that data message goes out team's processing, as shown in fig. 6, including the following steps：
S5-1：The virtual transmission deadline is corresponded to according to WFQ dispatching methods and each Coflow priority, is selected corresponding empty
The quasi- priority for being transmitted time minimum is as epicycle Coflow priority to be dispatched；
S5-2：The Coflow arrival times in the Coflow that dispatching priority includes are waited for according to epicycle, obtain arrival time
Earliest Coflow is used as Coflow to be dispatched, number to be used as TRAN_ID；
S5-3：It selects number for the Coflow storage queues of TRAN_ID, therefrom selects the pop-up of head of the queue message, output scheduling
Message.
S5-4：Corresponding Coflow messages are in the epicycle Coflow priority to be dispatched obtained in judgment step S5-1
No is sky, if do not update the virtual transmission deadline of epicycle Coflow priority to be dispatched then, and ending method, otherwise
Enter step S5-5；
S5-5：According to WFQ dispatching methods, the virtual transmission deadline of epicycle Coflow priority to be dispatched is calculated；
The calculation formula of the virtual transmission deadline of Coflow priority is：
In formula, V_finish[i]The virtual transmission deadline is corresponded to for Coflow priority；V_start[i]For Coflow
Priority corresponds to the virtual transmission time started；P[i]To correspond to the report of storage queue squadron first message positioned at the first Coflow
Literary sizes values；w[i]For Coflow priority i respective weights；w[l]For the respective weights of Coflow priority l；R is practical chain
Road rate；
V_start[i]=max { A[i],V_pre[i]}
In formula, V_start[i]The virtual transmission time started is corresponded to for Coflow priority；A[i]For positioned at first place
Coflow corresponds to the arrival time of storage queue squadron first message；V_pre[i]For on priority i in Coflow priority map
The virtual transmission deadline once calculated.
That the present invention provides a kind of accuracys is high, optimization property is high, reasonability is high, high efficiency and treatment effeciency it is high based on
The Coflow dispatching methods of subflow flow value method of estimation solve shortage accuracy of the existing technology, influence scheduling plan
Effect of optimization slightly, scheme lack reasonability and optimization property and the low problem for the treatment of effeciency.
Claims (10)
1. a kind of Coflow dispatching methods based on subflow flow value method of estimation, which is characterized in that include the following steps：
S1：Using Coflow subflow flow value methods of estimation, Coflow subflows flow value periodically each to host computer side is estimated,
And the estimated value of Coflow subflow flows is periodically reported to controller；
S2：Coflow is calculated by controller side Coflow priority formulating methods according to the estimated value of Coflow subflow flows
Corresponding priority, and issue and carry the messages of Coflow precedence informations to each host；
S3：The Coflow precedence informations carried in the message that host computer side is issued according to controller, at controller message
Reason method adjusts local Coflow priority corresponding relations, updates Coflow priority map and each Coflow priority corresponds to void
It is quasi- to be transmitted the time, generate data message；
S4：Processing of joining the team is carried out to the data message that step S3 is generated in host computer side, obtains the storage queue of Coflow messages；
S5：The virtual transmission deadline is corresponded to according to the obtained storage queues of step S4 and each Coflow priority, to datagram
Text carries out out team's processing, obtains schedules message, and realization is scheduled Coflow.
2. the Coflow dispatching methods according to claim 1 based on subflow flow value method of estimation, which is characterized in that institute
It states in step S1, Coflow subflow flow value methods of estimation include the following steps：
S1-1：Judge whether reach host computer side preset report cycle, if then entering step S1-2, execute host computer side statistics and
It reports, otherwise enters step S1-4；
S1-2：Calculate the estimated value of local Coflow subflows；
S1-3：The aggregate-value zero of message will be reached in the Coflow subflow periods, and reset and preset report cycle, enter step
S1-1；
S1-4：The aggregate-value of arrival message flow in the Coflow subflow periods of host computer side, Coflow subflows are obtained in transmission team
Message flow aggregate-value and Coflow subflows in row are transmitted message flow aggregate-value；
S1-5：Judge whether to reach to preset and report emulation duration, if then terminating reporting for host computer side, otherwise enters step S1-
1。
3. the Coflow dispatching methods according to claim 2 based on subflow flow value method of estimation, which is characterized in that institute
It states in step S1-2, the calculation formula of the estimated value of local Coflow subflows is：
In formula,It is flow estimation value of the local Coflow subflows in transmission end main frame statistics；Exist for Coflow subflows
Send the accumulative message flow magnitude in end main frame transmission queue；The transmission counted in transmission end main frame for Coflow subflows
Complete message integrated flow value；The aggregate-value that message is reached in the end main frame period is being sent for Coflow subflows.
4. the Coflow dispatching methods according to claim 1 based on subflow flow value method of estimation, which is characterized in that institute
It states in step S2, controller side Coflow priority formulating methods include the following steps：
S2-1：Judge whether otherwise reach the controller algorithm default execution period enters step if then entering step S2-2
S2-4；
S2-2：According to the estimated value of local Coflow subflows flow, the corresponding bottleneck values of Coflow are calculated, and combine bottleneck threshold
Value carries out each Coflow the division of priority, obtains the correspondence priority of each Coflow；
S2-3：It in the correspondence precedence level code to message of each Coflow, will be issued to each host, and reset the default execution period,
And enter step S2-1；
S2-4：The estimated value for counting the local Coflow subflows flow that each host reports, obtains the transmission of local Coflow subflows
The reception total estimates of total estimates and local Coflow subflows；
S2-5：Judge whether to reach the default emulation duration of algorithm, if then finishing control device side control algolithm, otherwise enters step
S2-1。
5. the Coflow dispatching methods according to claim 4 based on subflow flow value method of estimation, which is characterized in that institute
It states in step S2-2, the calculation formula of the corresponding bottleneck values of Coflow is：
In formula, bottleneck (c) is the corresponding bottleneck values of Coflow；For the transmission overall estimate of local Coflow subflows
Value；For the reception total estimates of local Coflow subflows；
The calculation formula of the reception total estimates of local Coflow subflows is：
In formula,For the reception total estimates of local Coflow subflows；For connecing for local Coflow subflows
Receive estimated value；I ∈ I_r (n, c) indicate to receive the set for the stream for belonging to Coflow c in end main frame n；
The calculation formula of the transmission total estimates of local Coflow subflows is：
In formula,For the transmission total estimates of local Coflow subflows；Estimate for the transmission of local Coflow subflows
Value；I (n, c) is the set for sending the stream for belonging to Coflow c in end main frame n.
6. the Coflow dispatching methods according to claim 1 based on subflow flow value method of estimation, which is characterized in that institute
It states in step S3, controller message processing method includes the following steps：
S3-1：The content of message in host computer side analyzing step S2 obtains the corresponding priority of each Coflow；
S3-2：According to the correspondence of Coflow and its priority, the CoflowID that each priority includes is obtained, and according to
The sequential update Coflow priority map of Coflow arrival times；
S3-3：Judge whether Coflow priority map changes, if then entering step S3-4, it is preferential otherwise not update Coflow
Grade corresponding virtual transmission deadline, and ending method；
S3-4：According to WFQ dispatching methods, each Coflow priority corresponding virtual transmission deadline is calculated.
7. the Coflow dispatching methods according to claim 1 based on subflow flow value method of estimation, which is characterized in that institute
It states in step S4, data message is joined the team the method for processing, is included the following steps：
S4-1：Judge whether host locally has the record of the affiliated Coflow of the data message, if then entering step S4-5, otherwise
Enter step S4-2；
S4-2：Judge that highest priority corresponds to whether queue is empty in Coflow priority map, if then entering step S4-3,
Otherwise Coflow highest priorities are not updated and correspond to the virtual transmission deadline, and enter step S4-4；
S4-3：According to WFQ dispatching methods, calculates Coflow highest priorities and correspond to the virtual transmission deadline；
S4-4：The priority of the Coflow is set as highest priority, and updates the Coflow priority map of host local；
S4-5：The arrival time of data message and the Coflow data messages is stored in storage queue corresponding with Coflow numbers
In, and export storage queue.
8. the Coflow dispatching methods according to claim 7 based on subflow flow value method of estimation, which is characterized in that institute
It states in step S4-3, the calculation formula that Coflow highest priorities correspond to the virtual transmission deadline is：
In formula, V_finish[i]The virtual transmission deadline is corresponded to for Coflow priority；A[i]For positioned at the first Coflow
The arrival time of corresponding storage queue squadron first message；P[i]To correspond to storage queue squadron first message positioned at the first Coflow
Message size value；w[i]For Coflow priority i respective weights；w[l]For the respective weights of Coflow priority l；R is real
Border link rate.
9. the Coflow dispatching methods according to claim 1 based on subflow flow value method of estimation, which is characterized in that institute
It states in step S5, the method that data message goes out team's processing includes the following steps：
S5-1：The virtual transmission deadline is corresponded to according to WFQ dispatching methods and each Coflow priority, selects corresponding virtual transmission
The priority of deadline minimum is as epicycle Coflow priority to be dispatched；
S5-2：The Coflow arrival times in the Coflow that dispatching priority includes are waited for according to epicycle, and it is earliest to obtain arrival time
Coflow be used as Coflow dispatch, number is as TRAN_ID；
S5-3：It selects number for the Coflow storage queues of TRAN_ID, therefrom selects the pop-up of head of the queue message, output scheduling message.
S5-4：In the epicycle Coflow priority to be dispatched obtained in judgment step S5-1 corresponding Coflow messages whether be
Sky, if do not update the virtual transmission deadline of epicycle Coflow priority to be dispatched then, and ending method, otherwise enter step
Rapid S5-5；
S5-5：According to WFQ dispatching methods, the virtual transmission deadline of epicycle Coflow priority to be dispatched is calculated.
10. the Coflow dispatching methods based on subflow flow value method of estimation according to claim 6 or 9, feature exist
In the calculation formula of the virtual transmission deadline of the Coflow priority is：
In formula, V_finish[i]The virtual transmission deadline is corresponded to for Coflow priority；V_start[i]It is preferential for Coflow
The grade corresponding virtual transmission time started；P[i]To correspond to the message size of storage queue squadron first message positioned at the first Coflow
Value；w[i]For Coflow priority i respective weights；w[l]For the respective weights of Coflow priority l；R is physical link rate；
V_start[i]=max { A[i],V_pre[i]}
In formula, V_start[i]The virtual transmission time started is corresponded to for Coflow priority；A[i]For positioned at the first Coflow
The arrival time of corresponding storage queue squadron first message；V_pre[i]It is calculated for the priority i last times in Coflow priority map
The virtual transmission deadline.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810420239.2A CN108712305B (en) | 2018-05-04 | 2018-05-04 | Flow scheduling method based on substream flow value estimation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810420239.2A CN108712305B (en) | 2018-05-04 | 2018-05-04 | Flow scheduling method based on substream flow value estimation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108712305A true CN108712305A (en) | 2018-10-26 |
| CN108712305B CN108712305B (en) | 2020-03-31 |
Family
ID=63867849
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810420239.2A Active CN108712305B (en) | 2018-05-04 | 2018-05-04 | Flow scheduling method based on substream flow value estimation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108712305B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110708259A (en) * | 2019-09-25 | 2020-01-17 | 江苏省未来网络创新研究院 | Information-agnostic Coflow scheduling system capable of automatically adjusting queue threshold and scheduling method thereof |
| CN111756653A (en) * | 2020-06-04 | 2020-10-09 | 北京理工大学 | Multi-coflow scheduling method based on deep reinforcement learning of graph neural network |
| WO2022074415A1 (en) * | 2020-10-06 | 2022-04-14 | Huawei Technologies Co., Ltd. | Device and method for coflow tracking and scheduling |
| CN117221126A (en) * | 2023-11-09 | 2023-12-12 | 之江实验室 | Network collaboration flow-oriented route scheduling method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105227488A (en) * | 2015-08-25 | 2016-01-06 | 上海交通大学 | A kind of network flow group scheduling method for distributed computer platforms |
| CN105827545A (en) * | 2016-04-21 | 2016-08-03 | 中国科学院信息工程研究所 | Scheduling method and device of TCP co-flows in data center network |
-
2018
- 2018-05-04 CN CN201810420239.2A patent/CN108712305B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105227488A (en) * | 2015-08-25 | 2016-01-06 | 上海交通大学 | A kind of network flow group scheduling method for distributed computer platforms |
| CN105827545A (en) * | 2016-04-21 | 2016-08-03 | 中国科学院信息工程研究所 | Scheduling method and device of TCP co-flows in data center network |
Non-Patent Citations (1)
| Title |
|---|
| SHUO WANG 等: "Leveraging multiple coflow attributes for information-agnostic coflow scheduling", 《2017 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS (ICC)》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110708259A (en) * | 2019-09-25 | 2020-01-17 | 江苏省未来网络创新研究院 | Information-agnostic Coflow scheduling system capable of automatically adjusting queue threshold and scheduling method thereof |
| CN111756653A (en) * | 2020-06-04 | 2020-10-09 | 北京理工大学 | Multi-coflow scheduling method based on deep reinforcement learning of graph neural network |
| WO2022074415A1 (en) * | 2020-10-06 | 2022-04-14 | Huawei Technologies Co., Ltd. | Device and method for coflow tracking and scheduling |
| CN117221126A (en) * | 2023-11-09 | 2023-12-12 | 之江实验室 | Network collaboration flow-oriented route scheduling method and system |
| CN117221126B (en) * | 2023-11-09 | 2024-02-13 | 之江实验室 | Network collaboration flow-oriented route scheduling method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108712305B (en) | 2020-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108712305A (en) | A kind of Coflow dispatching methods based on subflow flow value method of estimation | |
| Samanta et al. | Dyme: Dynamic microservice scheduling in edge computing enabled IoT | |
| CN103605567B (en) | Cloud computing task scheduling method facing real-time demand change | |
| Salot | A survey of various scheduling algorithm in cloud computing environment | |
| Huang et al. | An optimistic job scheduling strategy based on QoS for cloud computing | |
| CN108170530B (en) | Hadoop load balancing task scheduling method based on mixed element heuristic algorithm | |
| CN109684083A (en) | A kind of multilevel transaction schedule allocation strategy towards under edge-cloud isomery | |
| Gao et al. | Resource provisioning and profit maximization for transcoding in clouds: A two-timescale approach | |
| CN103685309B (en) | The method for building up of Asynchronous Request queue towards map visualization tile service access | |
| CN104657221A (en) | Multi-queue peak-alternation scheduling model and multi-queue peak-alteration scheduling method based on task classification in cloud computing | |
| CN109324875A (en) | A kind of data center server power managed and optimization method based on intensified learning | |
| CN104065745A (en) | Cloud computing dynamic resource scheduling system and method | |
| CN112332404B (en) | Intelligent management system and method for heating service | |
| CN104317646B (en) | Based on cloud data center dispatching method of virtual machine under OpenFlow frameworks | |
| CN105426241A (en) | Cloud computing data center based unified resource scheduling energy-saving method | |
| CN108170531B (en) | Cloud data center request flow scheduling method based on deep belief network | |
| Guo et al. | On-demand resource provision based on load estimation and service expenditure in edge cloud environment | |
| CN109740870B (en) | Resource dynamic scheduling method for Web application in cloud computing environment | |
| CN108108225A (en) | A kind of method for scheduling task towards cloud computing platform | |
| CN108711007A (en) | A kind of multitask real-time scheduling method of energy integration collection system | |
| CN109445921A (en) | A kind of distributed data task processing method and device | |
| CN109542608A (en) | A kind of cloud artificial tasks dispatching method based on mixing queuing network | |
| CN108536528A (en) | Using the extensive network job scheduling method of perception | |
| CN109934416A (en) | Research-on-research flows down the time optimization scheduling method of expense budget constraint in a kind of cloud | |
| Tang et al. | Nanily: A qos-aware scheduling for dnn inference workload in clouds |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |