CN110389843A - A kind of business scheduling method, device, equipment and readable storage medium storing program for executing - Google Patents
A kind of business scheduling method, device, equipment and readable storage medium storing program for executing Download PDFInfo
- Publication number
- CN110389843A CN110389843A CN201910689973.3A CN201910689973A CN110389843A CN 110389843 A CN110389843 A CN 110389843A CN 201910689973 A CN201910689973 A CN 201910689973A CN 110389843 A CN110389843 A CN 110389843A
- Authority
- CN
- China
- Prior art keywords
- goal task
- host
- gpu
- numa
- numa group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000003860 storage Methods 0.000 title claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000004590 computer program Methods 0.000 claims description 9
- 238000004891 communication Methods 0.000 abstract description 26
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 101000598160 Homo sapiens Nuclear mitotic apparatus protein 1 Proteins 0.000 description 100
- 102100036961 Nuclear mitotic apparatus protein 1 Human genes 0.000 description 100
- 238000012549 training Methods 0.000 description 10
- 238000013135 deep learning Methods 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 101710176296 Switch 2 Proteins 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of business scheduling methods, including obtain goal task；Obtain the available host that goal task can be performed in kubernets cluster；Goal task is successively corresponded to the distribution of NUMA group according to vacant GPU quantity in NUMA group in available host from more to few sequence to the goal task of vacant GPU quantity；The goal task corresponding with the CPU of same NUMA group processing NUMA group of the GPU in NUMA group is called according to the host topology information obtained in advance.When distributing goal task, it specifically can be preferentially using the GPU and CPU processing target task in same NUMA group, communication distance is most short between the GPU and CPU, and kubernets cluster resource utilization rate and GPU resource utilization rate can be improved, so that reducing business executes the time.The present invention also provides a kind of device, equipment and readable storage medium storing program for executing, equally have above-mentioned beneficial effect.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of business scheduling method, a kind of traffic scheduling device,
A kind of traffic scheduling equipment and a kind of computer readable storage medium.
Background technique
Container is with the characteristic of its operating system independence, by numerous developers, the happiness of operation maintenance personnel, major cloud manufacturer
Love, in artificial intelligence field, more and more algorithm personnel gradually begin to use container to carry out deep learning task run environment
Packing, deployment and debugging, to solve the problems, such as that deep learning environmental structure is complicated.For being dedicated to providing deep learning
The platform vendor of training environment, it will usually (be one to increase income, for managing multiple main frames in cloud platform using Kubernetes
On containerization application, referred to as: K8s) be trained the management of environment, main includes being based on holding for deep learning task creation
The training environment of device, and the resources such as reasonable GPU card, CPU, memory are distributed, the training of platform offer can be used in algorithm personnel
The training mission of environment operation deep learning.From the angle of resource utilization, platform operation maintenance personnel wishes the resource dispensed
It can be fully utilized, improve the resource utilization of cluster, wish that training mission can using the algorithm personnel of the training environment
It is completed using the time as few as possible, accelerates the iteration speed of model.
In the prior art, when distributing task resource by Kubernetes, GPU card quantity has normally only been used
Selection, i.e., under distributed scene, current technology only judge whether the quantity of GPU card sufficient, if GPU card quantity reach
Carry out task.But in the prior art, the speed of performing task distributed by above-mentioned Task Assigned Policy is usually relatively slow,
So how to improve the speed of performing task in kubernets cluster is those skilled in the art's urgent problem.
Summary of the invention
The object of the present invention is to provide a kind of business scheduling method, the task in kubernets cluster that can effectively improve is held
Scanning frequency degree；Another object of the present invention is to provide a kind of traffic scheduling device, a kind of traffic scheduling equipment and a kind of calculating
Machine readable storage medium storing program for executing can effectively improve the speed of performing task in kubernets cluster.
In order to solve the above technical problems, the present invention provides a kind of business scheduling method, comprising:
Obtain goal task；
Obtain the available host that the goal task can be performed in kubernets cluster；
By goal task according to vacant GPU quantity in NUMA group in the available host from more to few sequence successively to institute
State the goal task of the corresponding vacant GPU quantity of NUMA group distribution；
The GPU in the NUMA group and the CPU of same NUMA group is called to handle according to the host topology information obtained in advance
The corresponding goal task of the NUMA group.
Optionally, described by goal task according to vacant GPU quantity in NUMA group in the available host from more to few
Sequence successively to before the goal task of the corresponding vacant GPU quantity of NUMA group distribution, the method also includes:
When the goal task needs multiple NUMA groups to run the goal task, believed according to the host topology
The goal task is distributed to same available host by breath.
Optionally, described by goal task according to vacant GPU quantity in NUMA group in the available host from more to few
Sequence successively to before the goal task of the corresponding vacant GPU quantity of NUMA group distribution, the method also includes:
When the goal task needs multiple available hosts to run the goal task, according to the collection obtained in advance
The goal task is distributed to multiple available hosts that same interchanger is connected by group's topology information.
Optionally, described by goal task according to vacant GPU quantity in NUMA group in the available host from more to few
Sequence successively to before the goal task of the corresponding vacant GPU quantity of NUMA group distribution, the method also includes:
When the goal task needs the corresponding available host of multiple interchangers to run the goal task,
The goal task is distributed to the available host connecting with same father's interchanger according to the cluster topology information.
Optionally, the host topology information that the basis obtains in advance call GPU in the NUMA group with it is same
Before the CPU of NUMA group handles the corresponding goal task of the NUMA group, the method also includes:
Agency by the way that whole hosts in the kubernets cluster are arranged in obtains the host topology letter of the host
Breath.
Optionally, described obtain can be performed the available host of the goal task in kubernets cluster and include:
The stand-by host in the kubernets cluster is determined according to the default scheduling strategy of kubernets；
The available host is determined according to the mission bit stream of the goal task in the stand-by host.
The present invention also provides a kind of traffic scheduling devices, comprising:
Task acquisition module: for obtaining goal task；
Available host obtains module: for obtaining the available host that the goal task can be performed in kubernets cluster；
Distribution module: for by goal task according to the interior vacant GPU quantity of NUMA group in the available host from more to few
Sequence successively to the goal task of the corresponding vacant GPU quantity of NUMA group distribution；
Scheduler module: GPU and the same NUMA group in the NUMA group are called according to the host topology information obtained in advance
CPU handle the goal task.
Optionally, further includes:
First host assignment module: for needing multiple NUMA groups to run the goal task when the goal task
When, the goal task is distributed to by same available host according to the host topology information.
The present invention also provides a kind of traffic scheduling equipment, comprising:
Memory: for storing computer program；
Processor: the step of the business scheduling method as described in any of the above-described is realized when for executing the computer program
Suddenly.
The present invention also provides a kind of computer readable storage medium, meter is stored on the computer readable storage medium
Calculation machine program is realized as described in any of the above-described when the computer program is executed by processor the step of business scheduling method.
A kind of business scheduling method provided by the present invention, including obtain goal task；Obtaining can in kubernets cluster
The available host of performance objective task；By goal task according to vacant GPU quantity in NUMA group in available host from more to few
Sequence is successively to the goal task of the corresponding vacant GPU quantity of NUMA group distribution；It is called according to the host topology information obtained in advance
GPU goal task corresponding with the CPU of same NUMA group processing NUMA group in NUMA group.It is not single when distributing goal task
It is singly the available host that selection meets goal task demand, it also specifically can be preferentially using the GPU performance objective in same NUMA group
Task, and select and the CPU processing target task in the same NUMA group of the GPU.Since GPU and CPU belong to a NUMA group,
Communication distance is most short between the GPU and CPU, and communication time is most short, and communication loss is minimum, and kubernets cluster money can be improved
Source utilization rate and GPU resource utilization rate, so that reducing business executes time, such as the execution time of deep learning training mission.
The present invention also provides a kind of traffic scheduling device, a kind of traffic scheduling equipment and a kind of computer-readable storages
Medium equally has above-mentioned beneficial effect, is no longer repeated herein.
Detailed description of the invention
It, below will be to embodiment or existing for the clearer technical solution for illustrating the embodiment of the present invention or the prior art
Attached drawing needed in technical description is briefly described, it should be apparent that, the accompanying drawings in the following description is only this hair
Bright some embodiments for those of ordinary skill in the art without creative efforts, can be with root
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is a kind of flow chart of business scheduling method provided by the embodiment of the present invention；
Fig. 2 is a kind of flow chart of specific business scheduling method provided by the embodiment of the present invention；
Fig. 3 is a kind of structural topology schematic diagram of host provided by the embodiment of the present invention；
Fig. 4 is the flow chart of the specific business scheduling method of another kind provided by the embodiment of the present invention；
Fig. 5 is a kind of structural topology schematic diagram of kubernets cluster provided by the embodiment of the present invention；
Fig. 6 is the flow chart of another specific business scheduling method provided by the embodiment of the present invention；
Fig. 7 is a kind of structural block diagram of traffic scheduling device provided by the embodiment of the present invention；
Fig. 8 is a kind of structural block diagram of traffic scheduling equipment provided by the embodiment of the present invention.
Specific embodiment
Core of the invention is to provide a kind of business scheduling method.In the prior art, it is distributed by Kubernetes
When task resource, the selection of GPU card quantity is normally only used, i.e., under distributed scene, current technology is only sentenced
Whether the quantity of disconnected GPU card is sufficient, if GPU card quantity enough carries out task.Due to do not account for different GPU and with
Communication loss between CPU, often across NUMA group, across host or switch-spanning distribute task, this will make task execution
In information exchanging process, communication loss with higher needs a large amount of time so as to cause task execution in the prior art.
And a kind of business scheduling method provided by the present invention, including obtain goal task；It obtains in kubernets cluster
The available host of executable goal task；By goal task according to vacant GPU quantity in NUMA group in available host from more to few
Sequence successively to the goal task of the corresponding vacant GPU quantity of NUMA group distribution；According to the host topology information tune obtained in advance
With the GPU goal task corresponding with the CPU of same NUMA group processing NUMA group in NUMA group.Since GPU and CPU belong to one
A NUMA group, communication distance is most short between the GPU and CPU, and communication time is most short, and communication loss is minimum, can be improved
Kubernets cluster resource utilization rate and GPU resource utilization rate, so that reducing business executes time, such as deep learning training
The execution time of task.
In order to enable those skilled in the art to better understand the solution of the present invention, with reference to the accompanying drawings and detailed description
The present invention is described in further detail.Obviously, described embodiments are only a part of the embodiments of the present invention, rather than
Whole embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creative work premise
Under every other embodiment obtained, shall fall within the protection scope of the present invention.
Referring to FIG. 1, Fig. 1 is a kind of flow chart of business scheduling method provided by the embodiment of the present invention.
Referring to Fig. 1, in embodiments of the present invention, the business scheduling method includes:
S101: goal task is obtained.
In this step, need to obtain the task that kubernets cluster needs to be implemented, i.e. goal task.The goal task
Usually the training mission etc. of deep learning needs GPU to do largely computing repeatedly for task in embodiments of the present invention, accordingly
The meeting corresponding GPU of preferred allocation is for executing the goal task in the subsequent step of the embodiment of the present invention.Certainly, in the present invention
Above-mentioned goal task can also be other tasks in embodiment, in embodiments of the present invention and be not specifically limited.It is related to obtain
The specific steps of goal task can refer to the prior art, no longer be repeated herein.It should be noted that in this step,
Other than it can obtain goal task, the corresponding goal task information of goal task, such as goal task fortune are usually also obtained
Required GPU quantity, required CPU quantity, required memory headroom and required disk space etc. when row.
S102: the available host that goal task can be performed in kubernets cluster is obtained.
In embodiments of the present invention, kubernets cluster is the cluster that kubernets is deployed in distributed type assemblies,
Each host is usually provided with the correlation module of kubernets in the cluster.Kubernets can usually be called
Resource in kubernets cluster, including GPU, CPU, memory etc.；It can also distribute such as goal task received in S101
Deng.
In this step, can be according to goal task, specifically being determined according to goal task information can in kubernets cluster
With the available host for performance objective task, i.e. free host, which can run the host of the goal task.
It should be noted that the host of above-mentioned available host and non-required isolated operation goal task, it only can be with execution part target
The host of task can also be referred to as available host.Specifically, in this step, first choice needs to select from kubernets cluster
Taking can be with the host of operational objective task, i.e., selection GPU, CPU, memory are met the requirements from kubernets cluster, are not had simultaneously
The host of memory pressure and disk pressure is as available host.The Algorithms of Selecting of available host can refer to following formula:
Wherein, host.gpuFree >=job.gpu is vacant GPU quantity in host not less than GPU number needed for target service
Amount；Host.memoryFree >=job.memory is free memory space in host not less than memory headroom needed for target service；
Host.cpuFree >=job.cpu is vacant CPU quantity in host not less than CPU quantity needed for target service；
Host.diskPressure < diskPressureThread is that disk pressure is less than disk pressure line in host；
Host.memoryPressure < memoryPressureThread is that memory pressure is less than memory pressure line in host.According to
The host that above-mentioned formula is chosen can be used as available host for performance objective task.
S103: by goal task according to vacant GPU quantity in NUMA group in available host from more to few sequence successively to
The goal task of the corresponding vacant GPU quantity of NUMA group distribution.
In embodiments of the present invention, host is normally based on NUMA (Non Uniform Memory in kubernets cluster
Access Architecture, nonuniform memory access framework) building host, in host divide have multiple NUMA groups, often
A certain number of GPU and CPU, the CPU communication connection in GPU and same NUMA group, different NUMA are provided in a NUMA group
CPU communication connection between group.Specific topology information in relation to host in the embodiment of the present invention will be in following inventive embodiments
It is described in detail, is no longer repeated herein.
It in embodiments of the present invention, can be first by NUMA group each in available host according to vacant GPU in each NUMA group
Quantity is ranked up, and is usually ranked up according to vacant GPU quantity from more to few sequence, later can according to this sequentially to
The goal task of the corresponding respective vacant GPU quantity of NUMA group of NUMA group distribution.The respective NUMA group vacant GPU quantity of so-called correspondence
Goal task is equal to the goal task that the vacant GPU of NUMA group can handle traffic peak, to allow same NUMA as far as possible
GPU in group handles goal task assigned by the available host.Specifically, when the most NUMA group of vacant GPU quantity can
When with independent performance objective task completely, goal task can be fully allocated to the NUMA group.
S104: the GPU in NUMA group and the CPU of same NUMA group is called to handle according to the host topology information obtained in advance
Goal task.
Before this step, need to obtain the host topology information of above-mentioned available host in advance, the host topology information mark
The physical layout for interconnecting various equipment in host with transmission medium is known.Specifically, usually identify the affiliated NUMA group of above-mentioned GPU,
The information such as the affiliated NUMA group of CPU.Particular content in relation to host topology information will be described in detail in following inventive embodiments,
It is no longer repeated herein.In this step, it needs to call the GPU and CPU in same NUMA group according to host topology information
Cooperate the goal task for executing and being assigned to, and to reduce the communication loss between GPU as far as possible, and reduces GPU as far as possible
Communication loss between CPU realizes traffic scheduling.
A kind of business scheduling method provided by the embodiment of the present invention, including obtain goal task；Obtain kubernets collection
The available host of goal task can be performed in group；By goal task according to vacant GPU quantity in NUMA group in available host from more
To few sequence successively to the goal task of the corresponding vacant GPU quantity of NUMA group distribution；According to the host topology letter obtained in advance
Breath calls the goal task corresponding with the CPU of same NUMA group processing NUMA group of the GPU in NUMA group.In distribution goal task
When, it is not merely the available host that selection meets goal task demand, also specifically can be preferentially held using the GPU in same NUMA group
Row goal task, and select and the CPU processing target task in the same NUMA group of the GPU.Since GPU and CPU belong to one
NUMA group, communication distance is most short between the GPU and CPU, and communication time is most short, and communication loss is minimum, and kubernets can be improved
Cluster resource utilization rate and GPU resource utilization rate, so that reducing business executes time, such as the execution of deep learning training mission
Time.
It will be done in following inventive embodiments in detail in relation to a kind of particular content of business scheduling method provided by the present invention
It is thin to introduce.
Fig. 2 and Fig. 3 is please referred to, Fig. 2 is a kind of stream of specific business scheduling method provided by the embodiment of the present invention
Cheng Tu；Fig. 3 is a kind of structural topology schematic diagram of host provided by the embodiment of the present invention.
Referring to fig. 2, in embodiments of the present invention, the business scheduling method includes:
S201: the agency by the way that whole hosts in kubernets cluster are arranged in obtains the host topology information of host.
A kind of business scheduling method provided by foregoing invention embodiment and the embodiment of the present invention is substantially with one kind
The specific execution step of scheduler in kubernets cluster, in embodiments of the present invention it is generally necessary in kubernets cluster
Each host deployments one agency, i.e. agent, the agent are used to report the host topology informations of current hosts, while will adjust
Degree device containerization is simultaneously disposed in the form of kubernets Pod.
Referring to Fig. 3, it is divided into multiple NUMA groups, i.e. NUMA0 and NUMA1 in embodiments of the present invention, in host, when
So, more or fewer NUMA groups can be divided in a practical situation, depending on the circumstances, all parts in related host
Particular number in embodiments of the present invention and is not specifically limited.In embodiments of the present invention, usually pass through QPI between NUMA group
(Quick Path Interconnect also known as CSI, Common System Interface public system interface) connection.In
In same NUMA group, it is usually provided with multiple GPU, such as GPU0, GPU1 or GPU2, GPU3；And multiple CPU, such as
CPU0, CPU1, CPU2, CPU3 or CPU4, CPU5, CPU6, CPU7.Between the interior GPU of same NUMA group and CPU and GPU
Usually pass through PCIe connector, i.e. PCIe-Switch communication connection.
In this step, the agency being arranged in every host can send the host topology letter of host with preset format
Breath, i.e., in this step, the agency by the way that whole hosts in kubernets cluster are arranged in obtains the host topology information of host.
Above-mentioned host topology information usually requires to include that host ip, GPU ID, GPU video memory, GPU have used video memory, GPU card corresponding
CPU ID, usually following json data format are reported:
{"hostip":"192.168.159.128","gpu":[{"memoryTotal":"32G","
memoryUsed":"10G","no":0,"numa":"0","cpu":["cpu0","cpu1","cpu2","cpu3"]},{"
memoryTotal":"32G","memoryUsed":"10G","no":1,"numa":"1","cpu":["cpu4","
cpu5","cpu6","cpu7"]}]}
It should be noted that this step can most start to hold with a kind of business scheduling method provided by the embodiment of the present invention
Row, as long as being executed before S207 in the next steps, the usually execution parallel with following step.
S202: goal task is obtained.
This step and S101 in foregoing invention embodiment are almost the same, and detailed content please refers to foregoing invention embodiment, In
This is no longer repeated.
S203: the stand-by host in kubernets cluster is determined according to the default scheduling strategy of kubernets.
In this step, it will usually call the default scheduling strategy of kubernets, first to judge kubernets cluster
Whether the non-GPU relevant information of middle host, such as the memory information that task needs meet standard, for example, if the memory of the host
Deficiency, this host will not be selected performance objective task.Default scheduling strategy in relation to kubernets may refer to the following table 1:
The default scheduling strategy of table 1.kubernets
In this step, can according to kubernets default scheduling strategy, according in kubernets cluster host it is non-
GPU relevant information filters out stand-by host, and the exploitation of scheduler can be effectively reduced using the default scheduling strategy of kubernets
Difficulty.
S204: available host is determined according to the mission bit stream of goal task in stand-by host.
In this step, can appoint according to the Algorithms of Selecting of foregoing invention available host as described in the examples according to target
The mission bit stream of business filters out available host from stand-by host, to call the resource of available host to run mesh in subsequent step
Mark task.The particular content of Algorithms of Selecting in relation to available host is described in detail in foregoing invention embodiment, herein not
It is repeated again.
S205: when goal task needs multiple NUMA group operational objective tasks, target is appointed according to host topology information
Same available host is distributed in business.
In this step, logical between NUMA group in order to reduce when goal task can not be run in same NUMA group
News loss, needs to be distributed to goal task according to above-mentioned host topology information in this step multiple in same available host
NUMA group reduces the communication loss between NUMA group to use same available host operational objective task as far as possible.Judging
When whether needing multiple NUMA group operational objective tasks, money will can be usually used in goal task used resource and multiple NUMA groups
The maximum NUMA group in source is compared, to draw a conclusion.
S206: by goal task according to vacant GPU quantity in NUMA group in available host from more to few sequence successively to
The goal task of the corresponding vacant GPU quantity of NUMA group distribution.
This step and S103 in foregoing invention embodiment are almost the same, and detailed content please refers to foregoing invention embodiment, In
This is no longer repeated.
S207: the GPU in NUMA group and the CPU of same NUMA group is called to handle according to the host topology information obtained in advance
Goal task.
This step and S104 in foregoing invention embodiment are almost the same, and detailed content has been done in foregoing invention embodiment in detail
It is thin to introduce, it is no longer repeated herein.In this step, it due to being only assigned to NUMA group partial target task, i.e., divides in advance
Target subtask, the GPU of NUMA group is only used for executing the corresponding goal task of NUMA group, the NUMA in this step accordingly
CPU in group is only used for handling the corresponding goal task of NUMA group.
A kind of business scheduling method provided by the embodiment of the present invention can reduce as far as possible and complete the available of goal task
Communication loss in host, improves kubernets cluster resource utilization rate and GPU resource utilization rate, reduces business and executes the time.
It will be done in following inventive embodiments in detail in relation to a kind of particular content of business scheduling method provided by the present invention
It is thin to introduce.
Fig. 4 and Fig. 5 is please referred to, Fig. 4 is the specific business scheduling method of another kind provided by the embodiment of the present invention
Flow chart；Fig. 5 is a kind of structural topology schematic diagram of kubernets cluster provided by the embodiment of the present invention.
Referring to fig. 4, in embodiments of the present invention, the business scheduling method includes:
S301: goal task is obtained.
S302: the available host that goal task is corresponded in kubernets cluster is obtained.
Above-mentioned S301 to S302 and S101 to S102 in foregoing invention embodiment are almost the same, and detailed content please refers to above-mentioned
Inventive embodiments are no longer repeated herein.
S303: when goal task needs multiple available host operational objective tasks, according to the cluster topology obtained in advance
Goal task is distributed to multiple available hosts that same interchanger is connected by information.
Referring to Fig. 5, in embodiments of the present invention, host is usually by the interchanger mutually company of communicating in kubernets cluster
It connects, and is usually to be connected in tree construction between interchanger, for example, switch1 is father's interchanger of switch3, host in Fig. 4
Host0 passes sequentially through interchanger switch3, switch1, switch4 and host host2 communication connection.Certainly, of the invention real
Interchanger and host that kubernets cluster in example there can also be other quantity are applied, Fig. 4 is only illustrated.In this step
Before, need to obtain the cluster topology information of kubernets cluster, cluster topology message identification kubernets collection in advance
The equipment interconnected in group with transmission medium, i.e. physical layout between interchanger and host.Above-mentioned cluster topology information usually needs
It to include interchanger number, switch concatenation information, exchanger host relation information, the data format of the cluster topology information can
With as follows:
[{"switchname":"switch0","info":{"switch":["switch1","switch2"],"
hosts":[]}},{"switchname":"switch1","info":{"switch":["switch3","switch4"],"
hosts":[]}},{"switchname":"switch2","info":{"switch":["switch5","switch6"],"
Hosts ": [] } },
{"switchname":"switch3","info":{"switch":[],"hosts":["host0","
host1"]}},{"switchname":"switch4","info":{"switch":[],"hosts":["host2","
host3"]}},{"switchname":"switch5","info":{"switch":[],"hosts":["host4","
host5"]}},{"switchname":"switch6","info":{"switch":[],"hosts":["host6","
host7"]}}]
In this step, it when goal task can not be run in same available host, needs to distribute to goal task
Multiple available hosts.In order to reduce the communication loss between available host, needed in this step according to above-mentioned cluster topology information
Goal task is distributed to the available host connecting with same interchanger, so that same interchanger operational objective be made to appoint as far as possible
Business reduces the communication loss between available host.When judging whether to need multiple available host operational objective tasks, usually
Goal task used resource is compared with the maximum available host of available resources in multiple available hosts, to obtain knot
By.
S304: by goal task according to vacant GPU quantity in NUMA group in available host from more to few sequence successively to
The goal task of the corresponding vacant GPU quantity of NUMA group distribution.
S305: the GPU in NUMA group and the CPU of same NUMA group is called to handle according to the host topology information obtained in advance
Goal task.
Above-mentioned S304 to S305 and S206 to S207 in foregoing invention embodiment are almost the same, and detailed content please refers to above-mentioned
Inventive embodiments are no longer repeated herein.
A kind of business scheduling method provided by the embodiment of the present invention can reduce as far as possible and complete the available of goal task
Communication loss inside and outside host, improves kubernets cluster resource utilization rate and GPU resource utilization rate, when reducing business execution
Between.
It will be done in following inventive embodiments in detail in relation to a kind of particular content of business scheduling method provided by the present invention
It is thin to introduce.
Referring to FIG. 6, Fig. 6 is the flow chart of another specific business scheduling method provided by the embodiment of the present invention.
Referring to Fig. 6, in embodiments of the present invention, the business scheduling method includes:
S401: goal task is obtained.
S402: the available host that goal task is corresponded in kubernets cluster is obtained.
Above-mentioned S401 to S402 and S101 to S102 in foregoing invention embodiment are almost the same, and detailed content please refers to above-mentioned
Inventive embodiments are no longer repeated herein.
S403: it when goal task needs the corresponding available host operational objective task of multiple switch, is opened up according to cluster
It flutters information and goal task is distributed to the available host connecting with same father's interchanger.
In this step, it when goal task can not be run in same interchanger, needs to pass through the goal task more
A interchanger distributes to multiple available hosts.In order to reduce the communication loss between interchanger, needed in this step according to upper
It states cluster topology information and goal task is distributed into multiple available hosts by the multiple switch connecting with same father's interchanger,
Operational objective task in available host to be connected as far as possible using same father's interchanger reduces the communication damage between interchanger
It loses.It, usually will be used in goal task in the available host operational objective task for judging whether that multiple switch is needed to connect
Resource is compared with the maximum interchanger of available resources in multiple switch, to draw a conclusion.
S404: by goal task according to vacant GPU quantity in NUMA group in available host from more to few sequence successively to
The goal task of the corresponding vacant GPU quantity of NUMA group distribution.
S405: the GPU in NUMA group and the CPU of same NUMA group is called to handle according to the host topology information obtained in advance
Goal task.
Above-mentioned S404 to S405 and S304 to S305 in foregoing invention embodiment are almost the same, and detailed content please refers to above-mentioned
Inventive embodiments are no longer repeated herein.
A kind of business scheduling method provided by the embodiment of the present invention can reduce as far as possible and complete the available of goal task
Communication loss inside and outside host, improves kubernets cluster resource utilization rate and GPU resource utilization rate, when reducing business execution
Between.
It is provided for the embodiments of the invention a kind of traffic scheduling device below to be introduced, traffic scheduling described below
Device can correspond to each other reference with above-described business scheduling method.
Fig. 7 is a kind of structural block diagram of traffic scheduling device provided by the embodiment of the present invention, is filled referring to Fig. 7 traffic scheduling
It sets and may include:
Task acquisition module 100: for obtaining goal task.
Available host obtains module 200: for obtaining the available master that the goal task can be performed in kubernets cluster
Machine.
Distribution module 300: it is used for goal task according to vacant GPU quantity in NUMA group in the available host from more to
Few sequence is successively to the goal task of the corresponding vacant GPU quantity of NUMA group distribution.
Scheduler module 400: it is called in the available host in same NUMA group according to the host topology information obtained in advance
GPU execute the goal task, and call the CPU processing goal task for be in same NUMA group with the GPU.
Preferably, in embodiments of the present invention, traffic scheduling device can also include:
First host assignment module: for needing multiple NUMA groups to run the goal task when the goal task
When, the goal task is distributed to by same available host according to the host topology information.
Preferably, in embodiments of the present invention, traffic scheduling device can also include:
Second host assignment module: for appointing when the goal task needs multiple available hosts to run the target
When business, according to the cluster topology information obtained in advance by the goal task distribute to that same interchanger connected it is multiple described
Available host.Preferably, in embodiments of the present invention, traffic scheduling device can also include:
Third host assignment module: it is used to need corresponding described can use of multiple interchangers to lead when the goal task
When machine runs the goal task, the goal task is distributed to according to the cluster topology information and is connected with same father's interchanger
The available host connect.Preferably, in embodiments of the present invention, traffic scheduling device can also include:
Host topology information obtains module: for the agency by the way that whole hosts in the kubernets cluster are arranged in
Obtain the host topology information of the host.
Preferably, in embodiments of the present invention, available host obtains module 200 and may include:
Stand-by host acquiring unit: the kubernets cluster is determined for the default scheduling strategy according to kubernets
In stand-by host；
Available host acquiring unit: for determining institute according to the mission bit stream of the goal task in the stand-by host
State available host.
The traffic scheduling device of the present embodiment is for realizing business scheduling method above-mentioned, therefore in traffic scheduling device
The embodiment part of the visible hereinbefore business scheduling method of specific embodiment, for example, task acquisition module 100, available host
Module 200, distribution module 300 are obtained, scheduler module 400 is respectively used to realize step S101 in above-mentioned business scheduling method,
S102, S103 and S104, so, specific embodiment is referred to the description of corresponding various pieces embodiment, herein
It repeats no more.
A kind of traffic scheduling equipment provided in an embodiment of the present invention is introduced below, traffic scheduling described below is set
It is standby to correspond to each other reference with above-described business scheduling method and traffic scheduling device.
Referring to FIG. 8, Fig. 8 is a kind of structural block diagram of traffic scheduling equipment provided by the embodiment of the present invention.
Referring to Fig. 8, which may include processor 11 and memory 12.
The memory 12 is for storing computer program；It is real when the processor 11 is for executing the computer program
Existing foregoing invention business scheduling method as described in the examples.
Processor 11 is for installing business tune described in above-mentioned inventive embodiments in the traffic scheduling equipment of the present embodiment
Device is spent, simultaneous processor 11 is combined with memory 12 may be implemented traffic scheduling described in any of the above-described inventive embodiments
Method.Therefore the embodiment part of the visible book recommendation method hereinbefore of specific embodiment in traffic scheduling equipment,
Specific embodiment is referred to the description of corresponding various pieces embodiment, and details are not described herein.
The present invention also provides a kind of computer readable storage medium, meter is stored on the computer readable storage medium
Calculation machine program, the computer program realize a kind of business described in any of the above-described inventive embodiments when being executed by processor
Dispatching method.Remaining content is referred to the prior art, no longer carries out expansion description herein.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with it is other
The difference of embodiment, same or similar part may refer to each other between each embodiment.For being filled disclosed in embodiment
For setting, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is referring to method part
Explanation.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond the scope of this invention.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that the process, method, article or equipment for including a series of elements not only includes that
A little elements, but also including other elements that are not explicitly listed, or further include for this process, method, article or
The intrinsic element of equipment.In the absence of more restrictions, the element limited by sentence "including a ...", is not arranged
Except there is also other identical elements in the process, method, article or apparatus that includes the element.
A kind of business scheduling method provided by the present invention, a kind of traffic scheduling device, a kind of traffic scheduling are set above
Standby and a kind of computer readable storage medium is described in detail.Specific case used herein is to the principle of the present invention
And embodiment is expounded, the above embodiments are only used to help understand, and method and its core of the invention is thought
Think.It should be pointed out that for those skilled in the art, without departing from the principle of the present invention, may be used also
With several improvements and modifications are made to the present invention, these improvements and modifications also fall within the scope of protection of the claims of the present invention.
Claims (10)
1. a kind of business scheduling method characterized by comprising
Obtain goal task；
Obtain the available host that the goal task can be performed in kubernets cluster；
By goal task according to vacant GPU quantity in NUMA group in the available host from more to few sequence successively to described
The goal task of the corresponding vacant GPU quantity of NUMA group distribution；
According to the host topology information obtained in advance calls the GPU in the NUMA group and the CPU of same NUMA group is handled
The corresponding goal task of NUMA group.
2. the method according to claim 1, wherein it is described by goal task according in the available host
Vacant GPU quantity is successively appointed to the target of the corresponding vacant GPU quantity of NUMA group distribution from more to few sequence in NUMA group
Before business, the method also includes:
It, will according to the host topology information when the goal task needs multiple NUMA groups to run the goal task
The goal task distributes to same available host.
3. according to the method described in claim 2, it is characterized in that, it is described by goal task according in the available host
Vacant GPU quantity is successively appointed to the target of the corresponding vacant GPU quantity of NUMA group distribution from more to few sequence in NUMA group
Before business, the method also includes:
When the goal task needs multiple available hosts to run the goal task, opened up according to the cluster obtained in advance
It flutters information and the goal task is distributed into multiple available hosts that same interchanger is connected.
4. according to the method described in claim 3, it is characterized in that, it is described by goal task according in the available host
Vacant GPU quantity is successively appointed to the target of the corresponding vacant GPU quantity of NUMA group distribution from more to few sequence in NUMA group
Before business, the method also includes:
When the goal task needs the corresponding available host of multiple interchangers to run the goal task, according to
The goal task is distributed to the available host connecting with same father's interchanger by the cluster topology information.
5. the method according to claim 1, wherein being called in the host topology information that the basis obtains in advance
Before GPU goal task corresponding with the CPU of the same NUMA group processing NUMA group in the NUMA group, the method is also
Include:
Agency by the way that whole hosts in the kubernets cluster are arranged in obtains the host topology information of the host.
6. the method according to claim 1, wherein the mesh can be performed in the acquisition kubernets cluster
The available host of mark task includes:
The stand-by host in the kubernets cluster is determined according to the default scheduling strategy of kubernets；
The available host is determined according to the mission bit stream of the goal task in the stand-by host.
7. a kind of traffic scheduling device characterized by comprising
Task acquisition module: for obtaining goal task；
Available host obtains module: for obtaining the available host that the goal task can be performed in kubernets cluster；
Distribution module: for by goal task according to vacant GPU quantity in NUMA group in the available host from more to few suitable
Sequence is successively to the goal task of the corresponding vacant GPU quantity of NUMA group distribution；
Scheduler module: the CPU of the GPU and same NUMA group in the NUMA group are called according to the host topology information obtained in advance
Handle the goal task.
8. device according to claim 7, which is characterized in that further include:
First host assignment module: being used for when the goal task needs multiple NUMA groups to run the goal task,
The goal task is distributed into same available host according to the host topology information.
9. a kind of traffic scheduling equipment characterized by comprising
Memory: for storing computer program；
Processor: realizing the business scheduling method as described in any one of claim 1 to 6 when for executing the computer program
Step.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program realizes the step of the business scheduling method as described in any one of claim 1 to 6 when the computer program is executed by processor
Suddenly.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910689973.3A CN110389843B (en) | 2019-07-29 | 2019-07-29 | Service scheduling method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910689973.3A CN110389843B (en) | 2019-07-29 | 2019-07-29 | Service scheduling method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110389843A true CN110389843A (en) | 2019-10-29 |
| CN110389843B CN110389843B (en) | 2022-04-15 |
Family
ID=68287878
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910689973.3A Active CN110389843B (en) | 2019-07-29 | 2019-07-29 | Service scheduling method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110389843B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113014611A (en) * | 2019-12-19 | 2021-06-22 | 华为技术有限公司 | Load balancing method and related equipment |
| CN113377520A (en) * | 2021-07-07 | 2021-09-10 | 北京百度网讯科技有限公司 | Resource scheduling method, device, equipment and storage medium |
| CN113722085A (en) * | 2020-05-26 | 2021-11-30 | 安图斯科技股份有限公司 | Method and system for distributing graphic resources |
| CN114422419A (en) * | 2021-12-17 | 2022-04-29 | 阿里巴巴（中国）有限公司 | Network equipment connection method and device |
| CN116610372A (en) * | 2023-07-14 | 2023-08-18 | 腾讯科技（深圳）有限公司 | CPU configuration method, device and related products |
| CN117311990A (en) * | 2023-11-28 | 2023-12-29 | 苏州元脑智能科技有限公司 | Resource adjustment method and device, electronic equipment, storage medium and training platform |
| US11868805B2 (en) | 2021-04-13 | 2024-01-09 | Red Hat, Inc. | Scheduling workloads on partitioned resources of a host system in a container-orchestration system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107436798A (en) * | 2017-08-15 | 2017-12-05 | 深信服科技股份有限公司 | A kind of process access method and device based on NUMA node |
| CN107967180A (en) * | 2017-12-19 | 2018-04-27 | 上海交通大学 | Based on resource overall situation affinity network optimized approach and system under NUMA virtualized environments |
| CN108363623A (en) * | 2018-02-27 | 2018-08-03 | 郑州云海信息技术有限公司 | GPU resource dispatching method, device, equipment and computer readable storage medium |
| US20180307972A1 (en) * | 2017-04-24 | 2018-10-25 | International Business Machines Corporation | Local multicast in single-host multi-gpu machine for distributed deep learning systems |
| CN109167835A (en) * | 2018-09-13 | 2019-01-08 | 重庆邮电大学 | A kind of physics resource scheduling method and system based on kubernetes |
| US20190146850A1 (en) * | 2017-11-10 | 2019-05-16 | Amazon Technologies, Inc. | Capacity management in provider networks using dynamic host device instance model reconfigurations |
-
2019
- 2019-07-29 CN CN201910689973.3A patent/CN110389843B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180307972A1 (en) * | 2017-04-24 | 2018-10-25 | International Business Machines Corporation | Local multicast in single-host multi-gpu machine for distributed deep learning systems |
| CN107436798A (en) * | 2017-08-15 | 2017-12-05 | 深信服科技股份有限公司 | A kind of process access method and device based on NUMA node |
| US20190146850A1 (en) * | 2017-11-10 | 2019-05-16 | Amazon Technologies, Inc. | Capacity management in provider networks using dynamic host device instance model reconfigurations |
| CN107967180A (en) * | 2017-12-19 | 2018-04-27 | 上海交通大学 | Based on resource overall situation affinity network optimized approach and system under NUMA virtualized environments |
| CN108363623A (en) * | 2018-02-27 | 2018-08-03 | 郑州云海信息技术有限公司 | GPU resource dispatching method, device, equipment and computer readable storage medium |
| CN109167835A (en) * | 2018-09-13 | 2019-01-08 | 重庆邮电大学 | A kind of physics resource scheduling method and system based on kubernetes |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113014611A (en) * | 2019-12-19 | 2021-06-22 | 华为技术有限公司 | Load balancing method and related equipment |
| CN113014611B (en) * | 2019-12-19 | 2024-05-14 | 华为技术有限公司 | Load balancing method and related equipment |
| CN113722085A (en) * | 2020-05-26 | 2021-11-30 | 安图斯科技股份有限公司 | Method and system for distributing graphic resources |
| CN113722085B (en) * | 2020-05-26 | 2024-04-30 | 安图斯科技股份有限公司 | Distribution method and distribution system of graphic resources |
| US11868805B2 (en) | 2021-04-13 | 2024-01-09 | Red Hat, Inc. | Scheduling workloads on partitioned resources of a host system in a container-orchestration system |
| CN113377520A (en) * | 2021-07-07 | 2021-09-10 | 北京百度网讯科技有限公司 | Resource scheduling method, device, equipment and storage medium |
| CN114422419A (en) * | 2021-12-17 | 2022-04-29 | 阿里巴巴（中国）有限公司 | Network equipment connection method and device |
| CN114422419B (en) * | 2021-12-17 | 2024-05-28 | 阿里巴巴（中国）有限公司 | Network equipment connection method and device |
| CN116610372A (en) * | 2023-07-14 | 2023-08-18 | 腾讯科技（深圳）有限公司 | CPU configuration method, device and related products |
| CN116610372B (en) * | 2023-07-14 | 2024-02-09 | 腾讯科技（深圳）有限公司 | CPU configuration method, device and related products |
| CN117311990A (en) * | 2023-11-28 | 2023-12-29 | 苏州元脑智能科技有限公司 | Resource adjustment method and device, electronic equipment, storage medium and training platform |
| CN117311990B (en) * | 2023-11-28 | 2024-02-23 | 苏州元脑智能科技有限公司 | Resource adjustment method and device, electronic equipment, storage medium and training platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110389843B (en) | 2022-04-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110389843A (en) | A kind of business scheduling method, device, equipment and readable storage medium storing program for executing | |
| US10572290B2 (en) | Method and apparatus for allocating a physical resource to a virtual machine | |
| CN107222531B (en) | Container cloud resource scheduling method | |
| CN106385329B (en) | Processing method, device and the equipment of resource pool | |
| CN112000463B (en) | GPU resource allocation method, system, terminal and storage medium based on CUDA | |
| CN107968802A (en) | The method, apparatus and filtering type scheduler of a kind of scheduling of resource | |
| CN103399781B (en) | Cloud Server and virtual machine management method thereof | |
| CN113641457A (en) | Container creation method, device, apparatus, medium, and program product | |
| CN109471725A (en) | Resource allocation methods, device and server | |
| CN104793996A (en) | Task scheduling method and device of parallel computing equipment | |
| CN110287022A (en) | A kind of scheduling node selection method, device, storage medium and server | |
| CN110166507A (en) | More resource regulating methods and device | |
| CN109976907A (en) | Method for allocating tasks and system, electronic equipment, computer-readable medium | |
| CN108304044A (en) | A kind of setting method and system of NVME hard disk hot-plugs | |
| CN106970841A (en) | A kind of resource constraint and dynamic adjusting method based on cloud computing | |
| CN110275760A (en) | Process based on fictitious host computer processor hangs up method and its relevant device | |
| CN111464331B (en) | Control method and system for thread creation and terminal equipment | |
| CN111324424A (en) | Virtual machine deployment method, device, server and storage medium | |
| CN110795202B (en) | Resource allocation method and device of virtualized cluster resource management system | |
| CN113626173A (en) | Scheduling method, device and storage medium | |
| CN105933136B (en) | A kind of resource regulating method and system | |
| CN109788061B (en) | Computing task deployment method and device | |
| CN108667750A (en) | virtual resource management method and device | |
| CN102929693A (en) | Performance evaluation method and device for servers of whole equipment cabinet | |
| CN115080242A (en) | Method, device and medium for unified scheduling of PCI equipment resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |