CN113867973B - Resource allocation method and device - Google Patents

Resource allocation method and device Download PDF

Info

Publication number
CN113867973B
CN113867973B CN202111472656.XA CN202111472656A CN113867973B CN 113867973 B CN113867973 B CN 113867973B CN 202111472656 A CN202111472656 A CN 202111472656A CN 113867973 B CN113867973 B CN 113867973B
Authority
CN
China
Prior art keywords
computing
degree
node
task
dispersion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111472656.XA
Other languages
Chinese (zh)
Other versions
CN113867973A (en
Inventor
徐士立
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202111472656.XA priority Critical patent/CN113867973B/en
Publication of CN113867973A publication Critical patent/CN113867973A/en
Application granted granted Critical
Publication of CN113867973B publication Critical patent/CN113867973B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues

Abstract

The application provides a method and a device for resource allocation. According to the embodiment of the application, the calculation node for processing the calculation task can be determined in the at least two calculation nodes according to the discrete degree of the calculation force values of the at least two calculation resources of the calculation node under the condition that the calculation task is not distributed to the calculation node and under the condition that the calculation task is distributed to the calculation node. Therefore, the resource allocation is performed on the computing tasks according to the discrete degrees of the computing force values of the computing resources of the computing nodes before and after the computing tasks are allocated, and the method and the device can help to ensure the balanced occupation of the computing resources of the computing nodes. The embodiment of the application can be applied to the technical field of cloud, such as cloud computing or cloud storage.

Description

Resource allocation method and device
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for resource allocation.
Background
With the rapid development of internet services, the demand for reliable and efficient cluster computing (cluster computing) technology is more and more strong. Cluster computing refers to the joining together of a loosely integrated set of computer software or hardware to perform computing tasks in close cooperation. The cluster computing system comprises a management server and a computing cluster, wherein the management server is responsible for scheduling a cluster computing task, the computing cluster governs a plurality of computing nodes, and each computing node can be responsible for bearing the computing task.
Currently, a compute node management server allocates resources to compute tasks based on the computing power of a Central Processing Unit (CPU) of each compute node. However, as the complexity of the computing task is higher and higher, the computing task depends on the computing resources in a diversification manner, and the resource allocation is performed based on the traditional CPU computing power, on one hand, the requirements of the computing task on other computing resources cannot be met, on the other hand, the CPU of some computing nodes is fully loaded, but other resources are still idle, or the CPU of other nodes is idle but other resources are fully loaded, so that the overall bearing capacity of the system cannot be maximized.
Disclosure of Invention
The embodiment of the application provides a resource allocation method and device, which can help to ensure balanced occupation of each computing resource of a computing node, thereby being beneficial to ensuring that computing power of a cluster is fully utilized.
In a first aspect, a method for resource allocation is provided, including:
acquiring a calculation task;
under the condition that the computing tasks are not distributed to at least two computing nodes, determining a first discrete degree of a first computation force value according to the first computation force value respectively corresponding to at least two computing resources of each computing node;
under the condition that a first computing node of the at least two computing nodes distributes the computing task, determining second computing force values respectively corresponding to at least two computing resources of the first computing node according to computing resource requirements of the computing task;
determining a second discrete degree of a second computation force value of at least two computation resources of the first computation node according to the second computation force value;
determining, among the at least two compute nodes, a compute node for processing the compute task according to the first degree of dispersion and the second degree of dispersion.
In a second aspect, an apparatus for resource allocation is provided, including:
an acquisition unit configured to acquire a calculation task;
the processing unit is used for determining a first discrete degree of a first calculation force value according to the first calculation force value corresponding to at least two calculation resources of each calculation node under the condition that the calculation tasks are not distributed to at least two calculation nodes;
the processing unit is further configured to determine, when a first computing node of the at least two computing nodes allocates the computing task, second computing force values corresponding to at least two computing resources of the first computing node according to computing resource requirements of the computing task;
the processing unit is further configured to determine a second degree of dispersion of a second computation force value of at least two computation resources of the first computation node according to the second computation force value;
the processing unit is further configured to determine, among the at least two compute nodes, a compute node for processing the compute task according to the first degree of dispersion and the second degree of dispersion.
In a third aspect, an electronic device is provided, which includes: a processor and a memory; the memory for storing a computer program; the processor is configured to execute the computer program to implement the method of the first aspect.
In a fourth aspect, a chip is provided, comprising: a processor for calling and running the computer program from the memory so that the device on which the chip is installed performs the method according to the first aspect.
In a fifth aspect, there is provided a computer readable storage medium comprising computer instructions which, when executed by a computer, cause the computer to carry out the method of the first aspect.
In a sixth aspect, there is provided a computer program product comprising computer program instructions to, when run on a computer, cause the computer to perform the method of the first aspect.
By the scheme of the embodiment of the application, the discrete degree of the calculation force values of at least two calculation resources of the calculation node can be determined according to the condition that the calculation task is not distributed to the calculation node and the condition that the calculation task is distributed to the calculation node, and the calculation node for processing the calculation task is determined in at least two calculation nodes according to the discrete degree. Therefore, the embodiment of the application allocates resources to the computing tasks according to the discrete degrees of the computing force values of the computing resources of the computing nodes before and after the computing tasks are allocated, which can help to ensure the balanced occupation of the computing resources of the computing nodes, and is favorable for ensuring the full utilization of the computing force of the cluster. In addition, the embodiment of the application can also be helpful for meeting the requirements of the computing task on two or more computing resources.
Drawings
FIG. 1 is a schematic diagram of a system to which the teachings of the embodiments of the present application are applied;
FIG. 2 is a schematic diagram of a management server and computing clusters to which the solution of an embodiment of the present application applies;
fig. 3 is a schematic flow chart of a method for resource allocation provided by an embodiment of the present application;
FIG. 4 is a schematic flow chart diagram of another method for resource allocation provided by an embodiment of the present application;
FIG. 5 is a schematic flow chart diagram of another method for resource allocation provided by an embodiment of the present application;
FIG. 6 is a schematic flow chart diagram of another method for resource allocation provided by an embodiment of the present application;
FIG. 7 is an alternative schematic block diagram of an apparatus for resource allocation of an embodiment of the present application;
fig. 8 is another alternative schematic block diagram of an electronic device provided by an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be understood that in the embodiment of the present application, "B corresponding to a" means that B is associated with a. In one implementation, B may be determined from a. It should also be understood that determining B from a does not mean determining B from a alone, but may be determined from a and/or other information.
In the description of the present application, "at least one" means one or more, "a plurality" means two or more than two, unless otherwise specified. In addition, "and/or" describes an association relationship of associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.
It should be further understood that the descriptions of the first, second, etc. appearing in the embodiments of the present application are only for illustrating and differentiating the objects, and do not represent a particular limitation to the number of devices in the embodiments of the present application, and do not constitute any limitation to the embodiments of the present application.
It should also be appreciated that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the application. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The method and the device for resource allocation provided by the embodiment of the application can be applied to a cluster computing system, for example, a computing node management server in the cluster computing system. The computing cluster system also includes a computing cluster including at least two computing nodes. The cluster computing system can be applied to the technical fields of cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like, and the cluster computing system is not limited in the application.
In the resource allocation scheme provided in the embodiment of the application, after a computation task is obtained, a first discrete degree of computation force values of at least two computation resources of the computation node is determined under the condition that the computation task is not allocated to the computation node, a second discrete degree of computation force values of at least two computation resources of the computation node is determined after the computation node is allocated to the computation task, and then the computation node for processing the computation task is determined in at least two computation nodes of a computation cluster according to the first discrete degree and the second discrete degree. Therefore, the embodiment of the application allocates resources to the computing tasks according to the discrete degrees of the computing force values of the computing resources of the computing nodes before and after the computing tasks are allocated, which can help to ensure the balanced occupation of the computing resources of the computing nodes, and is favorable for ensuring the full utilization of the computing force of the cluster. In addition, the embodiment of the application can also be helpful for meeting the requirements of the computing task on two or more computing resources.
A network architecture to which the technical solution of the embodiment of the present application can be applied will be briefly described below. It should be noted that the network architecture described below is only used for illustrating the embodiments of the present application and is not limited. In specific implementation, the technical scheme provided by the embodiment of the application can be flexibly applied according to actual needs.
Fig. 1 shows a schematic diagram of an alternative system 100 to which the solution of the embodiments of the present application applies. The system 100 includes a management server 110 and a computing cluster 120, wherein the computing cluster 120 may include a plurality of (two or more) computing nodes, such as a computing node 121, a computing node 122, a computing node 123, and the like, without limitation. Illustratively, the management server 110 is responsible for scheduling the cluster computing tasks, and the computing cluster 120, such as the computing nodes therein, is responsible for carrying the computing tasks. In some embodiments, the management server 110 may also be referred to as a compute node management server, without limitation.
In some embodiments, the system 100 may be a server cluster, may be a backend server, or a cloud server providing cloud computing or cloud storage services, such as a game server, without limitation.
Cloud computing (cloud computing) is a computing model that distributes computing tasks over a pool of resources formed by a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the "cloud" appear to the user as being infinitely expandable and available at any time, available on demand, expandable at any time, and paid for on-demand.
As a basic capability provider of cloud computing, a cloud computing resource pool (called as an ifas (Infrastructure as a Service) platform for short is established, and multiple types of virtual resources are deployed in the resource pool and are selectively used by external clients.
According to the logic function division, a Platform as a Service (PaaS) layer can be deployed on an Infrastructure as a Service (IaaS) layer, a Software as a Service (SaaS) layer is deployed on the PaaS layer, and the SaaS can be directly deployed on the IaaS. PaaS is a platform on which software runs, such as a database, a web container, etc. SaaS is a variety of business software, such as web portal, sms, and mass texting. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.
A distributed cloud storage system (hereinafter, referred to as a storage system) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of different types in a network through application software or application interfaces to cooperatively work by using functions such as cluster application, grid technology, and a distributed storage file system, and provides a data storage function and a service access function to the outside.
At present, a storage method of a storage system is as follows: logical volumes are created, and when created, each logical volume is allocated physical storage space, which may be the disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as data Identification (ID), the file system writes each object into a physical storage space of the logical volume, and the file system records storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.
The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided in advance into stripes according to a group of capacity measures of objects stored in a logical volume (the measures often have a large margin with respect to the capacity of the actual objects to be stored) and Redundant Array of Independent Disks (RAID), and one logical volume can be understood as one stripe, thereby allocating physical storage space to the logical volume.
In some embodiments, referring to fig. 2, a capacity calculation module 201, a task scheduling module 202, and a node management module 203 may be included in the management server 110.
The capacity calculation module 201 may be configured to determine data related to computing resources of each computing node in the computing cluster 120, such as available capacity, used capacity, or total capacity. Optionally, the capacity calculation module 201 may obtain data reported by the computing nodes in each computing cluster, for example, data related to computing resources, such as used capacity, available capacity, or total capacity.
The task scheduling module 202 may assign a compute node to a received compute task. Optionally, the task scheduling module 202 may monitor a state of the computing task in a process of executing the computing task, for example, whether an exception occurs, and may reschedule a new suitable computing node for the computing task when the exception occurs in the computing task.
The node management module 203 may be configured to manage the computing nodes in the cluster, for example, determine whether a new computing task may be received according to the report status, for example, kick an abnormal computing node out of the computing cluster 120, and for example, add a new computing node to the computing cluster 120 as needed.
In some embodiments, referring to fig. 2, the computing cluster 120 may include a data collection module 204, a task management module 205, and a data reporting module 206. Optionally, the data acquisition module 204, the task management module 205, and the data reporting module 206 may be located on each computing node, or may be separately arranged outside each computing node, which is not limited.
The data collection module 204 may be disposed on a computing node, for example, and is configured to collect data related to a computing resource on the computing node, such as a maximum load capacity or a current actual load capacity. Optionally, the data acquisition module 204 may perform necessary calculation processing on the acquired data, for example, calculate available capacity according to the maximum carrying capacity, the current actual carrying capacity, and the like, without limitation.
The data reporting module 205 may report the data acquired by the data acquisition module 204 to the management server 110. Optionally, the data reporting module 205 may summarize the data acquired by the data acquisition module 204 before reporting the data, for example, report the data with larger available capacity preferentially after summarizing, which is not limited.
The task management module 206 may receive a new computing task from the management server 110 and may manage the computing task, such as, without limitation, performing exception detection, task completion, and the like during a lifecycle of the computing task.
Optionally, in this embodiment of the present application, the computing resource may include at least one of CPU power, Graphics Processing Unit (GPU) power, a memory, a network bandwidth, a disk input/output (I/O), and the like.
For example, the CPU computation power and the GPU computation power may be measured by an Operation Per Second (OPS), that is, an average number of times that an operation instruction can be executed per second, or a floating-point operation per second (FLOPS), that is, an average number of times that a floating-point operation instruction can be executed per second, respectively. For example, the CPU power requirements of different games may be approximately scaled to OPS, or FLOPS.
Furthermore, the CPU computation power or the GPU computation power may also be MIllion Operations Per Second (MOPS), billions operations Per Second (GOPS), trillion operations Per Second (terra operations Per Second, TOPS), and the like, without limitation. MOPS, GOPS and TOPS are different in unit and can be converted among the three.
The CPU power or the GPU power may also be MIllion times of floating-point Operation Per Second (MFLOPS), billion times of floating-point Operation Per Second (GFLOPS), millions of times of floating-point Operation Per Second (TFLOPS), or the like, without limitation. MFLOPS, GFLOPS and TFLOPS have different units, and the units can be converted. In addition, an appropriate calculation power unit can be selected according to different calculation scenes, and is not limited.
In addition, the FLOPS can be divided into different types such as half-precision, single-precision, and double-precision. Because the difference of the calculation forces calculated by the different calculation types is large, the bearing capacity of the GPU or the CPU can be calculated respectively according to the measurement indexes of the different types.
In some embodiments, for example for video processing class operations, the computational resources may also include throughput, such as the upper bearer limit may be determined from lower bounds on computational power and throughput.
For example, the memory may refer to a memory space that needs to be occupied by the computing task in the running process, and the unit may be, for example, MB.
Illustratively, network bandwidth may refer to the network bandwidth of a computing node. The bandwidth of a single computer or an edge computing node affects the final bearable capacity of the computing node, so that the required values of the internal network and the external outlet can be calculated, for example, the lower limit of the required values can be taken as the basis for carrying calculation.
Illustratively, the disk IO may be divided into two directions of read and write throughput, which may be, for example, the amount of data read or written from or to the disk per second (KB/S).
The technical solutions provided by the embodiments of the present application are described below with reference to the accompanying drawings.
Fig. 3 shows a schematic flow chart of a method 300 for resource allocation provided by an embodiment of the present application. The method 300 may be applied to a computing cluster system, such as the system 100 shown in fig. 1, for example, executed by the management server 110 in fig. 1 or fig. 2. As shown in fig. 3, method 300 includes steps 310 through 350. Illustratively, the computing cluster system may include a game server, which may include a management server and at least two computing nodes, wherein the management server may be configured to perform the method 300.
Step 310, obtain a computing task.
Illustratively, step 310 may be performed by the task scheduling module 202 in the management server 110. The task scheduling module 202 may obtain instructions from other servers or user clients and determine computing tasks based on the instructions. As an example, the computing task may include a computing task corresponding to a game operation instruction, such as a character change computing task corresponding to a game character movement instruction. Meanwhile, relevant information of the computing task, such as at least one of CPU computation amount, GPU computation amount, transmission bandwidth, required memory, and the like, may be determined, and a computing resource requirement of the computing task may be determined according to the relevant information of the computing task, which may be, for example, a highest value of the required computing resource.
In some alternative embodiments, the computing resource requirements of the computing task may also be determined (or predicted) based on the relevant information of the computing task and historical computing resource usage data. Illustratively, the historical computing resource usage data may be determined based on required computing resources for computing tasks that have been previously completed. Since the historical computing resource usage data can reflect the computing resource usage of the completed computing task, determining the computing resource demand of the computing task according to the historical computing resource usage data can help to predict the computing resource demand of a certain computing task in the future more accurately.
As a specific example, the CPU computation amount required for computing task a may be 100000OPS, the GPU computation amount may be 150000FLOPS, the memory may be 300000MB, and the disk IO may be 2000KB/S, which is not limited in this application.
Step 320, in a case that at least two computing nodes do not allocate the computing task, determining a first discrete degree of a first computation force value according to the first computation force value corresponding to at least two computing resources of a computing node of the at least two computing nodes, respectively.
The first discrete degree can represent the balanced occupation condition of each computing resource before the computing node is not allocated with the computing task. That is, the smaller the first degree of dispersion, the more evenly the computing nodes occupy the computing resources before the computing tasks are not allocated.
Illustratively, step 320 may be performed by the capacity calculation module 201 in the management server 110. For example, the capacity computation module 201 may determine, in real time or periodically, a discrete degree of the computation force value corresponding to each computation resource of each computation node in the computation cluster 120, that is, the first discrete degree. In this way, when the task scheduling module 202 receives a computing task, the degree of dispersion of the computation force values of the computing resources at the time when the computing task is not allocated at each computing node, which is computed by the capacity computation module 201, can be obtained.
In some optional embodiments, computing resource data of the computing node may be further obtained, where the computing resource data may include at least one of a total resource amount, an available resource amount, and an used resource amount corresponding to the at least two computing resources of the computing node, respectively.
Illustratively, the computing resource data of the computing nodes may be obtained by the capacity computation module 201 in the management server 110. Alternatively, the capacity calculation module 201 may obtain the computing resource data of the computing node at predetermined time intervals, for example, periodically. For example, the capacity computation module 201 may send a computation resource query request to the data reporting module 206 in the computation cluster 120 and receive computation resource data reported by the data reporting module 206.
In some alternative embodiments, the first computation force value is a normalized computation force value. As a possible implementation manner, when the computing nodes are not allocated with the computing tasks, the computing resources of the computing nodes may be normalized according to the maximum computing power corresponding to the computing resources of the computing nodes, so as to obtain first computing power values corresponding to the computing resources of the computing nodes.
For example, for a computing node, the maximum computation force value of its single computing resource may be defined as 1, and the actual usage of the single computing resource of the computing node may be proportionally converted according to its maximum computation force value. For example, the CPU power, the OPS for GPU power, the MB for memory, the MB for network bandwidth, and the KB/S for disk IO may all be converted by dividing the current actual usage by the maximum power value.
As a specific example, if the highest CPU computation force value of the compute node 121 is 2000000OPS and the currently actually used CPU computation force value is 1000000OPS, the currently actually used normalized CPU computation force value of the compute node 121 is (1000000 ÷ 2000000) = 0.50.
As another specific example, for the computing node 121, the highest GPU computation value is 3000000FLOPS, and the currently actually used GPU computation value is 2000000FLOPS, so that the currently actually used normalized GPU computation value of the computing node 121 is (2000000 ÷ 3000000) = 0.67.
As another specific example, for the compute node 121, the highest memory computation value (i.e., memory value) is 8000000MB, and the currently actually used memory value is 5000000MB, then the currently actually used normalized memory computation value of the compute node 121 is (5000000 ÷ 8000000) = 0.625.
In some alternative embodiments, the first degree of dispersion is a standard deviation of first computing force values of at least two computing resources of the computing node. That is, a standard deviation of a first value of the computational effort of the at least two computational resources of the compute node may be determined and determined as the first degree of dispersion.
Alternatively, at least two of the one compute nodes may be determined according to the following equation (1)Standard deviation of first computing power value corresponding to each computing resource
Figure 990582DEST_PATH_IMAGE001
Figure 132850DEST_PATH_IMAGE003
(1)
Wherein n is the number of computing resources, which can also be called computing power index number,
Figure 689733DEST_PATH_IMAGE004
for the normalized calculation force value of the ith calculation force index before the calculation task is not distributed,
Figure 35395DEST_PATH_IMAGE005
the average value of the normalized calculation force values of the n calculation force indexes before the calculation task is not distributed is obtained.
As a specific example, taking three computing resources of the computing node 121 as an example, if the normalized CPU computation force value is 0.50, the normalized GPU computation force value is 0.67, and the normalized memory value is 0.625, then the average value of the normalized computation force values of the three computing resources is (0.5 +0.67+ 0.625) ÷ 3= 0.598. Substituting the above equation (1) can obtain:
Figure 968716DEST_PATH_IMAGE006
for example, the standard deviation of the normalized computation force values of the computing resources may be calculated for all computing nodes having idle computing resources in the computing cluster 120, as the first discrete degree of the computation force values of the computing resources of the corresponding computing nodes, which is not limited in this application.
In some alternative embodiments, the first discrete degree may be a variance of the first computation force values of the at least two computation resources, which is not limited in this application.
Step 330, determining, according to the computing resource requirements of the computing task, second computing force values respectively corresponding to at least two computing resources of the first computing node, when the first computing node of the at least two computing nodes allocates the computing task.
Illustratively, the first computing node may be a computing node having free resources available to perform the computing task.
Illustratively, step 330 may be performed by the capacity calculation module 201 in the management server 110. For example, after the task scheduling module 202 receives a computing task and determines the computing resource requirements needed by the computing task, the computing resource requirements may be sent to the capacity computation module 201. The capacity calculation module 201 may determine, according to the computing resource requirement, second computing force values corresponding to at least two computing resources of a first computing node in the computing cluster, respectively, when the first computing node allocates the computing task.
In some alternative embodiments, the second computation force value is a normalized computation force value.
As a possible implementation, referring to fig. 4, the normalized second computation force value may be determined according to the following steps 331 and 332.
Step 331, according to the computing resource requirement of the computing task and the maximum computing power of the computing resource of the first computing node, performing normalization processing on the computing power value corresponding to the computing resource requirement to obtain a normalized computing resource requirement of the computing task.
Step 332, determining the second computation force values corresponding to at least two computation resources of the first computation node respectively according to the normalized computation resource requirement of the computation task and the normalized computation force values of the computation resources of the first computation node under the condition that the computation task is not allocated.
As a specific example, taking the case where the calculation task a is allocated to the calculation node 121, the CPU calculation amount required for the calculation task a is 100000OPS, and the highest CPU calculation power value of the calculation node 121 is 2000000OPS, the normalized CPU calculation power value required for the calculation task a is (100000 ÷ 2000000) = 0.05. Further, when the computing node 121 does not allocate the computing task a, the normalized CPU computation force value actually used is 0.50, and then it may be determined that the computing node 121 uses (0.50 + 0.05) =0.055 after allocating the computing task a.
The GPU computation amount required for computing task a is 150000FLOPS, the highest GPU computation value of compute node 121 is 3000000FLOPS, and the normalized GPU computation value required for computing task a is (150000 ÷ 3000000) = 0.05. Further, when the computing node 121 does not allocate the computing task a, the normalized GPU computation force value actually used is 0.67, and then it may be determined that the normalized GPU computation force value used by the computing node 121 after allocating the computing task a is (0.67 + 0.05) = 0.72.
The memory required for computing task a is 300000MB, the highest memory value of compute node 121 is 8000000MB, and the normalized memory required for computing task a is (300000 ÷ 8000000) = 0.0375. Further, when the computing node 121 does not allocate the computing task a, the actually used normalized memory value is 0.625, and then it may be determined that the computing node 121 uses the normalized memory value of (0.625 + 0.0375) =0.6625 after allocating the computing task a.
Optionally, under the condition that the computing task is allocated to a second computing node of the at least two computing nodes of the computing cluster, third computing force values corresponding to the at least two computing resources of the second computing node are determined according to computing resource requirements of the computing task. Illustratively, the second computing node may be a computing node having free resources available to perform the computing task. Specifically, reference may be made to a process of determining second computation force values corresponding to at least two computation resources of the first computation node, which is not described herein again.
Optionally, under the condition that each of the at least two computing nodes of the computing cluster is allocated with the computing task, the computing force value corresponding to each of the at least two computing resources of each computing node may be determined according to the computing resource requirement of the computing task. Illustratively, each computing node herein may be a computing node having free resources available to perform the computing task. Specifically, reference may be made to a process of determining second computation force values corresponding to at least two computation resources of the first computation node, which is not described herein again.
Step 340, determining a second discrete degree of a second computation force value of at least two computation resources of the first computation node according to the second computation force value.
The second discrete degree can represent the balanced occupation condition of each computing resource after the first computing node distributes the computing task. That is, the smaller the second degree of dispersion, the more evenly the computing resources are occupied by the first computing node after the first computing node has allocated the computing task.
Illustratively, step 320 may be performed by the capacity calculation module 201 in the management server 110. For example, after performing step 330, the capacity calculation module 201 may determine a second discrete degree of the second computation force value of each computation resource of the first computation node according to the determined second computation force values corresponding to the at least two computation resources of the first computation node, respectively.
In some alternative embodiments, the second degree of dispersion may be a standard deviation of second force values of the at least two computing resources of the first computing node. That is, a standard deviation of second computation force values of at least two computation resources of the first computation node may be determined and determined as the second degree of dispersion.
For example, the standard deviation of the second computation force values corresponding to at least two computation resources of the first computation node may be determined according to the above formula (1), for example, when the computation task a is allocated to the computation node 121, the normalized CPU computation force value 0.055, the normalized GPU computation force value 0.72, and the normalized memory value 0.6625 that are used may be substituted into the above formula (1), so as to obtain the CPU computation force value, the GPU computation force value, and the standard deviation of the memory of the computation node 121, where the standard deviation is the second degree of dispersion.
In some alternative embodiments, the second degree of dispersion may be, without limitation, a variance of the second computation force values of the at least two computation resources.
Step 350, determining a computing node for processing the computing task in the at least two computing nodes according to the first discrete degree and the second discrete degree.
Therefore, the embodiment of the application determines the discrete degrees of the computation force values of at least two computation resources of the computation node in the case that the computation task is not allocated to the computation node and in the case that the computation task is allocated to the computation node respectively, and determines the computation node for processing the computation task in at least two computation nodes according to the discrete degrees in the two cases. Therefore, the embodiment of the application allocates resources to the computing tasks according to the discrete degrees of the computing force values of the computing resources of the computing nodes before and after the computing tasks are allocated, which can help to ensure the balanced occupation of the computing resources of the computing nodes, and is favorable for ensuring the full utilization of the computing force of the cluster. In addition, the embodiment of the application can also be helpful for meeting the requirements of the computing task on two or more computing resources.
In some alternative embodiments, referring to fig. 5, the compute nodes for processing the compute task may be determined according to steps 351 and 352 as follows:
step 351, determining a first total dispersion degree of at least two computing nodes in the computing cluster under the condition that the first computing node is allocated with the computing task according to the first dispersion degree and the second dispersion degree.
In step 352, a computing node for processing the computing task is determined among the at least two computing nodes according to the first total degree of dispersion.
Wherein the first total degree of dispersion includes a sum of the second degree of dispersion and a first degree of dispersion of computing nodes other than the first computing node of the at least two computing nodes in the computing cluster. The first total discrete degree can represent the balanced occupation condition of each computing resource of all the computing nodes in the computing cluster after the first computing node distributes the computing tasks. That is, the smaller the first total degree of dispersion, the more evenly the computing resources on the computing nodes in the computing cluster are occupied by the first computing node after the computing task is distributed.
Therefore, according to the first total dispersion degree, the computing node for processing the computing task is determined in the at least two computing nodes, so that resource allocation can be performed on the computing task based on the balanced occupation situation of each computing resource of all the computing nodes in the computing cluster after the computing task is allocated to the first computing node, and the balanced occupation of each computing resource of the computing nodes can be further facilitated to be guaranteed, and the computing power of the cluster can be further facilitated to be fully utilized.
As a possible implementation manner, the first total dispersion degree may be determined according to a sum of dispersion degrees of the computation force values of at least two computation resources before the computation task is not allocated to each computation node in the computation cluster, a second dispersion degree of the computation force values of at least two computation resources when the first computation node is allocated to the computation task, and a first dispersion degree of the computation force values of at least two computation resources when the first computation node is not allocated to the computation task.
Illustratively, the first total degree of dispersion may be determined according to the following equation (2)
Figure 231070DEST_PATH_IMAGE007
Figure 693276DEST_PATH_IMAGE008
(2)
Wherein the content of the first and second substances,
Figure 290348DEST_PATH_IMAGE009
for the normalized calculation force value of the ith calculation force index after the calculation task is distributed,
Figure 496201DEST_PATH_IMAGE010
and averaging the normalized calculation force values of the n calculation force indexes after the calculation task is distributed.
Specifically, the first term in equation (2)
Figure 613062DEST_PATH_IMAGE011
A sum of discrete degrees of computation force values representing at least two computing resources of computing nodes not allocated with the computing task, a second term
Figure 511748DEST_PATH_IMAGE012
A second degree of dispersion, a third term, representing a computational force value of at least two computational resources of the first computational node after the computational task has been assigned
Figure DEST_PATH_IMAGE013
A first degree of dispersion of the computation force values of at least two computing resources of the first computing node prior to the unassigned computing task is represented.
In some alternative embodiments, referring to fig. 6, the compute node for processing the compute task may be determined according to steps 353 through 355 below:
step 353, determining a third discrete degree of a third computation force value corresponding to each of at least two computation resources of a second computation node of the at least two computation nodes, when the second computation node allocates the computation task.
For example, a third degree of dispersion of a third value of computational effort for at least two computational resources of a second computing node of the at least two computational nodes of the computing cluster may be determined in an instance in which the second computing node is assigned the computational task.
Step 354, determining a second total degree of dispersion of the at least two compute nodes when the second compute node in the compute cluster is assigned the compute task based on the first degree of dispersion and the third degree of dispersion. Specifically, the process of determining the second total dispersion degree may refer to the process of determining the first total dispersion degree, and is not described herein again.
Step 355, determining a computing node for processing the computing task from the at least two computing nodes according to the first total discrete degree and the second total discrete degree.
Wherein the second total degree of dispersion comprises a sum of the third degree of dispersion and the first degree of dispersion of the computing nodes of the at least two computing nodes other than the second computing node. The second total discrete degree can represent a balanced occupation situation of each computing resource of all the computing nodes in the computing cluster after the second computing node allocates the computing task. That is, the smaller the second total degree of dispersion, the more evenly the computing resources on the computing nodes in the computing cluster are occupied by the second computing node after the computing task is distributed.
Therefore, according to the first total dispersion degree and the second total dispersion degree, namely, by comparing the equilibrium occupation situation of each computing resource of all computing nodes in the computing cluster after the computing tasks are respectively allocated to the first computing node and the second computing node, the computing node for processing the computing tasks is determined in the at least two computing nodes, which can further contribute to ensuring the equilibrium occupation of each computing resource of the computing nodes, thereby further contributing to ensuring that the computing power of the cluster is fully utilized.
As one implementation, when the first total degree of dispersion is less than the second total degree of dispersion, a first computing node may be determined among the at least two computing nodes to process the computing task; when the first total degree of dispersion is greater than the second total degree of dispersion, a second computing node may be determined among the at least two computing nodes to process the computing task. That is, after the computing task is allocated to which computing node, the computing resources on the computing nodes in the computing cluster are occupied more evenly, and then it can be determined that the computing task is allocated to the computing node.
In some embodiments, optionally, the degree of dispersion of the computation force values of the at least two computing resources of each computing node may also be determined in a case where each computing node of the at least two computing nodes of the computing cluster is assigned the above-mentioned computation task.
Thereafter, a total degree of dispersion of at least two compute nodes in the compute cluster may be determined with each compute node in the compute cluster (i.e., each available compute node with idle resources) assigned the compute task, and the compute node corresponding to the minimum total degree of dispersion may be selected as the compute node for processing the compute task. Specifically, the process of determining the total dispersion degree corresponding to each computing node may refer to the process of determining the first total dispersion degree, which is not described herein again.
After the computing task is distributed to the computing node corresponding to the minimum total discrete degree, the computing node can ensure that the occupation of each computing resource of all the computing nodes in the computing cluster is most balanced, so that the embodiment of the application can further contribute to ensuring the balanced occupation of each computing resource of the computing nodes, and further contribute to ensuring that the computing power of the cluster is utilized most fully.
In other embodiments, as an implementation manner for determining, from among the at least two computing nodes, a computing node for processing the computing task according to the first total dispersion degree, the first total dispersion degree may be compared with a preset threshold, and if the first total dispersion degree is greater than the preset threshold, the computing task is not performed using the first computing node; if the first total degree of dispersion is less than or equal to the preset threshold, the first computing node may be used to perform the computing task.
For example, the preset threshold may be determined based on historical computing resource usage data, or may be empirically configured without limitation.
In some optional embodiments, the management server, for example, the capacity calculation module, may store data in the above calculation process, for example, at least one of the first degree of dispersion, the second degree of dispersion, the first total degree of dispersion, and the second total degree of dispersion, as historical calculation resource usage data, so that the calculation resource requirement of a new calculation task may be predicted subsequently according to the historical calculation resource usage data, or the size of the first total degree of dispersion may be determined.
After determining that a computing node (e.g., a first computing node, or a second computing node) in the computing cluster performs the computing task, the management server 110 may send an instruction to the task management module 205 in the computing cluster 120 instructing the computing node to perform the computing task. Accordingly, the task management module 205 may receive instructions from the management server 110 to perform the computing task and cause the computing node to perform the computing task. Optionally, the task management module 205 may also monitor a state of the computing node running the computing task, including whether the computing resource is sufficient, whether the computing task is ended, whether an exception occurs, and the like.
As a specific example, the technical solution provided in the embodiment of the present application may be applied in a game scene. For example, the computing cluster system may be used as a game server, wherein the management server may receive a game task corresponding to a game operation instruction sent by the game client, such as a character movement instruction, for example, a process of computing character information, game information, a small map and other related information. After receiving the game task, the management server in the computing cluster system may obtain relevant information of the game task, such as CPU computation, GPU computation, transmission bandwidth, required memory, and the like, and determine the computing resource requirements of the game task. Further, the management server may determine, in the computing nodes of the computing cluster system, a computing node for processing the game task according to a first discrete degree of the computation force values of the computing resources when the game task is not allocated to each computing node in the computing cluster system, and a second discrete degree of the computation force values of each computing resource when one computing node (or each computing node) is allocated to the game task, so as to enable balanced occupation of each computing resource of the computing nodes of the computing cluster system, thereby further facilitating to ensure that the computation force of the cluster is fully utilized.
After determining that a computing node in the computing cluster system executes the game task, the management server may send an instruction to a task management module in the computing cluster to instruct the selected computing node to execute the game task. Accordingly, the task management module may receive an instruction to perform the game task from the management server and cause the computing node to perform the game task. In the process that the computing node runs the game task, the task management module in the computing cluster can also monitor the running state of the game task, including whether the computing resources are enough, whether the game task is finished, whether an exception occurs, and the like.
Optionally, in the process of running the computing task by the computing node, the data acquisition module 204 may acquire the computing resource usage data of the computing node, and the data reporting module 206 may report the data acquired by the data acquisition module 204, which is not limited in this application.
In some alternative embodiments, the management server 110, for example, the node management module 203, may perform machine selection of the computing cluster system according to the actual situation of resource allocation, for example, add a new computing node, or adjust the computing resource of the computing node. For example, after the computing power of each computing resource of the computing node is optimally allocated, the normalized computing power value of a certain computing resource is always close to 1, that is, the computing resource reaches or approaches a bottleneck, while the normalized computing power values of other computing resources are sufficient, which indicates that the computing cluster system has an unreasonable machine selection problem. At this point, emphasis may be placed on such resources when the machine is typed. For example, when the CPU computing power always reaches the bottleneck, the maximum CPU carrying capacity of the compute node may be expanded during subsequent machine model selection to optimize the compute cluster system performance.
The present invention is not limited to the details of the above embodiments, and various simple modifications can be made to the technical solution of the present invention within the technical concept of the present invention, and the technical solution of the present invention is protected by the present invention. For example, the various features described in the foregoing detailed description may be combined in any suitable manner without contradiction, and various combinations that may be possible are not described in this application in order to avoid unnecessary repetition. For example, various embodiments of the present application may be arbitrarily combined with each other, and the same should be considered as the disclosure of the present application as long as the concept of the present application is not violated.
It should also be understood that, in the various method embodiments of the present application, the sequence numbers of the above-mentioned processes do not imply an execution sequence, and the execution sequence of the processes should be determined by their functions and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. It is to be understood that the numerical designations are interchangeable under appropriate circumstances such that the embodiments of the application described are capable of operation in sequences other than those illustrated or described herein.
Method embodiments of the present application are described in detail above with reference to fig. 1 to 6, and apparatus embodiments of the present application are described in detail below with reference to fig. 7 to 8.
Fig. 7 is a schematic block diagram of an apparatus 700 for resource allocation according to an embodiment of the present application. The device 700 may be, for example, the management server in fig. 1 or fig. 2, and is not limited thereto. As shown in fig. 7, the apparatus 700 may include an acquisition unit 710 and a processing unit 720.
An obtaining unit 710 for obtaining a calculation task.
The processing unit 720 is configured to, when the at least two computing nodes are not allocated with the computing task, determine a first discrete degree of a first computation force value according to the first computation force value corresponding to each of the at least two computing resources of each computing node.
The processing unit 720 is further configured to, when a first computing node of the at least two computing nodes allocates the computing task, determine, according to computing resource requirements of the computing task, second computing force values corresponding to at least two computing resources of the first computing node, respectively.
The processing unit 720 is further configured to determine a second degree of dispersion of a second computation force value of at least two computation resources of the first computation node according to the second computation force value.
The processing unit 720 is further configured to determine a computing node for processing the computing task from the at least two computing nodes according to the first degree of dispersion and the second degree of dispersion.
Optionally, the processing unit 720 is specifically configured to:
determining a first total degree of dispersion of the at least two computing nodes under the condition that the first computing node is allocated with the computing task according to the first degree of dispersion and the second degree of dispersion, wherein the first total degree of dispersion comprises the sum of the second degree of dispersion and the first degree of dispersion of computing nodes except the first computing node in the at least two computing nodes;
and determining a computing node for processing the computing task in the at least two computing nodes according to the first total discrete degree.
Optionally, the processing unit 720 is further configured to:
determining a third discrete degree of a third computing force value corresponding to at least two computing resources of a second computing node in the at least two computing nodes respectively under the condition that the second computing node allocates the computing task;
determining a second total degree of dispersion of the at least two computing nodes under the condition that the second computing node allocates the computing task according to the first degree of dispersion and the third degree of dispersion, wherein the second total degree of dispersion comprises the sum of the third degree of dispersion and the first degree of dispersion of computing nodes except the second computing node in the at least two computing nodes;
and determining a computing node for processing the computing task in the at least two computing nodes according to the first total discrete degree and the second total discrete degree.
Optionally, the processing unit 720 is specifically configured to:
when the first total dispersion degree is smaller than the second total dispersion degree, determining that the first computing node processes the computing task;
and when the second total dispersion degree is smaller than the first total dispersion degree, determining that the second computing node processes the computing task.
Optionally, the first degree of dispersion is a standard deviation of the first computing force values of the at least two computing resources; the second degree of dispersion is a standard deviation of the second value of the computation of the at least two computing resources.
Optionally, the first degree of dispersion is a variance of the first computation force values of the at least two computation resources; the second degree of dispersion is a variance of the second computational values of the at least two computational resources.
Optionally, the processing unit 720 is further configured to:
and under the condition that the computing tasks are not distributed to at least two computing nodes, according to the maximum computing force respectively corresponding to the at least two computing resources of the computing nodes, normalizing the computing force values respectively corresponding to the at least two computing resources of the computing nodes to obtain the first computing force values respectively corresponding to the at least two computing resources of the computing nodes.
Optionally, the processing unit 720 is further configured to:
according to the computing resource requirement of the computing task and the maximum computing power of the computing resource of the first computing node, carrying out normalization processing on the computing power value corresponding to the computing resource requirement to obtain the normalized computing resource requirement of the computing task;
and determining the second computation force values respectively corresponding to the at least two computation resources of the first computation node according to the normalized computation resource requirements of the computation tasks and the normalized computation force values of the computation resources of the first computation node under the condition that the computation tasks are not distributed.
Optionally, the processing unit 720 is further configured to:
determining the computing resource requirements of the computing task according to the information of the computing task and historical computing resource usage data, wherein the historical computing resource usage data is determined according to the computing resources required by the completed computing task.
Optionally, the obtaining unit 710 is further configured to:
and acquiring computing resource data of the computing node, wherein the computing resource data comprises at least one of total resource amount, available resource amount and used resource amount corresponding to the at least two computing resources of the computing node respectively.
Optionally, the at least two computing resources of the computing node include at least two of CPU computing power, GPU computing power, memory, network bandwidth, disk IO, and the like of the computing node.
Optionally, the calculation task includes a calculation task corresponding to the game operation instruction.
Optionally, the method is applied to a management server of a game server, where the game server further includes the at least two computing nodes.
It is to be understood that apparatus embodiments and method embodiments may correspond to one another and that similar descriptions may refer to method embodiments. To avoid repetition, further description is omitted here. Specifically, the apparatus 700 for resource allocation in this embodiment may correspond to a corresponding main body for executing the method 300 in this embodiment, and the foregoing and other operations and/or functions of the respective modules in the apparatus 700 are respectively for implementing the corresponding flows of the respective methods in fig. 3 to fig. 6, and are not described herein again for brevity.
The apparatus and system of embodiments of the present application are described above in connection with the drawings from the perspective of functional modules. It should be understood that the functional modules may be implemented by hardware, by instructions in software, or by a combination of hardware and software modules. Specifically, the steps of the method embodiments in the present application may be implemented by integrated logic circuits of hardware in a processor and/or instructions in the form of software, and the steps of the method disclosed in conjunction with the embodiments in the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. Alternatively, the software modules may be located in random access memory, flash memory, read only memory, programmable read only memory, electrically erasable programmable memory, registers, and the like, as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps in the above method embodiments in combination with hardware thereof.
Fig. 8 is a schematic block diagram of an electronic device 800 provided in an embodiment of the present application.
As shown in fig. 8, the electronic device 800 may include:
a memory 810 and a processor 820, the memory 810 being configured to store a computer program and to transfer the program code to the processor 820. In other words, the processor 820 may call and execute a computer program from the memory 810 to implement the communication method in the embodiment of the present application.
For example, the processor 820 may be configured to perform the steps of the method 400 or the steps of the method 600 according to instructions of the computer program.
In some embodiments of the present application, the processor 820 may include, but is not limited to:
general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like.
In some embodiments of the present application, the memory 810 includes, but is not limited to:
volatile memory and/or non-volatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of example, but not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic random access memory (Synchronous DRAM, SDRAM), Double Data Rate Synchronous Dynamic random access memory (DDR SDRAM), Enhanced Synchronous SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).
In some embodiments of the present application, the computer program may be partitioned into one or more modules, which are stored in the memory 810 and executed by the processor 820 to perform the encoding methods provided herein. The one or more modules may be a series of computer program instruction segments capable of performing certain functions, the instruction segments describing the execution of the computer program in the electronic device 800.
Optionally, the electronic device 800 may further include:
a transceiver 830, the transceiver 830 being connectable to the processor 820 or the memory 810.
The processor 820 may control the transceiver 830 to communicate with other devices, and specifically, may transmit information or data to the other devices or receive information or data transmitted by the other devices. The transceiver 830 may include a transmitter and a receiver. The transceiver 830 may further include one or more antennas.
It should be understood that the various components in the electronic device 800 are connected by a bus system that includes a power bus, a control bus, and a status signal bus in addition to a data bus.
According to an aspect of the present application, there is provided a communication device comprising a processor and a memory, the memory being configured to store a computer program, the processor being configured to call and execute the computer program stored in the memory, so that the encoder performs the method of the above-described method embodiment.
According to an aspect of the present application, there is provided a computer storage medium having a computer program stored thereon, which, when executed by a computer, enables the computer to perform the method of the above-described method embodiments. In other words, the present application also provides a computer program product containing instructions, which when executed by a computer, cause the computer to execute the method of the above method embodiments.
According to another aspect of the application, a computer program product or computer program is provided, comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method of the above-described method embodiment.
In other words, when implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions described in accordance with the embodiments of the present application occur, in whole or in part, when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Video Disk (DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), among others.
Those of ordinary skill in the art will appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus, device and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the module is merely a logical division, and other divisions may be realized in practice, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. For example, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (15)

1. A method of resource allocation, comprising:
acquiring a calculation task;
under the condition that the computing tasks are not distributed to at least two computing nodes, determining a first discrete degree of a first computation force value according to the first computation force value respectively corresponding to at least two computing resources of each computing node;
under the condition that a first computing node of the at least two computing nodes is allocated with the computing task, determining second computing force values respectively corresponding to at least two computing resources of the first computing node according to computing resource requirements of the computing task, wherein the first computing node is a computing node which can be used for executing the computing task and is in the at least two computing nodes;
determining a second discrete degree of a second computation force value of at least two computation resources of the first computation node according to the second computation force value;
determining a computing node for processing the computing task from the at least two computing nodes according to the first degree of dispersion and the second degree of dispersion;
wherein the determining, among the at least two compute nodes, a compute node for processing the compute task according to the first degree of dispersion and the second degree of dispersion comprises:
determining a first total degree of dispersion of the at least two computing nodes under the condition that the first computing node is allocated with the computing task according to the first degree of dispersion and the second degree of dispersion, wherein the first total degree of dispersion comprises the sum of the second degree of dispersion and the first degree of dispersion of computing nodes except the first computing node in the at least two computing nodes;
and determining a computing node for processing the computing task in the at least two computing nodes according to the first total discrete degree.
2. The method of claim 1, further comprising:
determining a third discrete degree of a third computing force value corresponding to at least two computing resources of a second computing node in the at least two computing nodes respectively under the condition that the second computing node allocates the computing task;
determining a second total degree of dispersion of the at least two computing nodes under the condition that the second computing node allocates the computing task according to the first degree of dispersion and the third degree of dispersion, wherein the second total degree of dispersion comprises the sum of the third degree of dispersion and the first degree of dispersion of computing nodes except the second computing node in the at least two computing nodes;
wherein the determining, among the at least two compute nodes, a compute node for processing the compute task according to the first total degree of dispersion comprises:
and determining a computing node for processing the computing task in the at least two computing nodes according to the first total discrete degree and the second total discrete degree.
3. The method according to claim 2, wherein determining a compute node for processing the compute task among the at least two compute nodes according to the magnitude of the first total degree of dispersion and the second total degree of dispersion comprises:
when the first total dispersion degree is smaller than the second total dispersion degree, determining that the first computing node processes the computing task;
and when the second total dispersion degree is smaller than the first total dispersion degree, determining that the second computing node processes the computing task.
4. The method of claim 1, wherein the first degree of dispersion is a standard deviation of the first computational power values of the at least two computational resources; the second degree of dispersion is a standard deviation of the second computing force values of the at least two computing resources; or
The first degree of dispersion is a variance of the first computing force values of the at least two computing resources; the second degree of dispersion is a variance of the second computational values of the at least two computational resources.
5. The method according to any one of claims 1-4, further comprising:
and under the condition that the at least two computing nodes are not allocated with the computing tasks, normalizing the computing force values respectively corresponding to the at least two computing resources of the at least two computing nodes according to the maximum computing forces respectively corresponding to the at least two computing resources of the at least two computing nodes, so as to obtain the first computing force values respectively corresponding to the at least two computing resources of the at least two computing nodes.
6. The method according to any one of claims 1-4, further comprising:
according to the computing resource requirement of the computing task and the maximum computing power of the computing resource of the first computing node, carrying out normalization processing on the computing power value corresponding to the computing resource requirement to obtain the normalized computing resource requirement of the computing task;
and determining the second computation force values respectively corresponding to the at least two computation resources of the first computation node according to the normalized computation resource requirements of the computation tasks and the normalized computation force values of the computation resources of the first computation node under the condition that the computation tasks are not distributed.
7. The method according to any one of claims 1-4, further comprising:
and acquiring computing resource data of the at least two computing nodes, wherein the computing resource data comprises at least one of total resource amount, available resource amount and used resource amount corresponding to the at least two computing resources of the at least two computing nodes respectively.
8. The method according to any one of claims 1-4, further comprising:
determining the computing resource requirements of the computing task according to the information of the computing task and historical computing resource usage data, wherein the historical computing resource usage data is determined according to the computing resources required by the completed computing task.
9. The method of any of claims 1-4, wherein the at least two computing resources of the at least two computing nodes comprise at least two of CPU computing power, GPU computing power, memory, network bandwidth, and disk IO.
10. The method of any one of claims 1-4, wherein the computing task comprises a computing task corresponding to a game play instruction.
11. The method of any of claims 1-4, applied to a management server of a game server, wherein the game server further comprises the at least two computing nodes.
12. An apparatus for resource allocation, comprising:
an acquisition unit configured to acquire a calculation task;
the processing unit is used for determining a first discrete degree of a first calculation force value according to the first calculation force value corresponding to at least two calculation resources of each calculation node under the condition that the calculation tasks are not distributed to at least two calculation nodes;
the processing unit is further configured to determine, when a first computing node of the at least two computing nodes allocates the computing task, second computing force values corresponding to at least two computing resources of the first computing node according to computing resource requirements of the computing task, where the first computing node is a computing node of the at least two computing nodes that can be used to execute the computing task;
the processing unit is further configured to determine a second degree of dispersion of a second computation force value of at least two computation resources of the first computation node according to the second computation force value;
the processing unit is further configured to determine, among the at least two computing nodes, a computing node for processing the computing task according to the first degree of dispersion and the second degree of dispersion;
wherein the processing unit is specifically configured to:
determining a first total degree of dispersion of the at least two computing nodes under the condition that the first computing node is allocated with the computing task according to the first degree of dispersion and the second degree of dispersion, wherein the first total degree of dispersion comprises the sum of the second degree of dispersion and the first degree of dispersion of computing nodes except the first computing node in the at least two computing nodes;
and determining a computing node for processing the computing task in the at least two computing nodes according to the first total discrete degree.
13. An electronic device comprising a processor and a memory, the memory having stored therein instructions that, when executed by the processor, cause the processor to perform the method of any of claims 1-11.
14. A computer storage medium for storing a computer program comprising instructions for performing the method of any one of claims 1-11.
15. A computer program product, comprising computer program code which, when run by an electronic device, causes the electronic device to perform the method of any of claims 1-11.
CN202111472656.XA 2021-12-06 2021-12-06 Resource allocation method and device Active CN113867973B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111472656.XA CN113867973B (en) 2021-12-06 2021-12-06 Resource allocation method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111472656.XA CN113867973B (en) 2021-12-06 2021-12-06 Resource allocation method and device

Publications (2)

Publication Number Publication Date
CN113867973A CN113867973A (en) 2021-12-31
CN113867973B true CN113867973B (en) 2022-02-25

Family

ID=78986008

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111472656.XA Active CN113867973B (en) 2021-12-06 2021-12-06 Resource allocation method and device

Country Status (1)

Country Link
CN (1) CN113867973B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115827185B (en) * 2022-10-31 2023-12-01 ***数智科技有限公司 Method, storage medium and equipment for combining 6G air base station with Beidou air obstacle avoidance
CN117707793B (en) * 2024-02-05 2024-05-03 太平金融科技服务(上海)有限公司 Computing task processing method, device, equipment and medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102232282A (en) * 2010-10-29 2011-11-02 华为技术有限公司 Method and apparatus for realizing load balance of resources in data center
CN109144710A (en) * 2017-06-16 2019-01-04 ***通信有限公司研究院 Resource regulating method, device and computer readable storage medium
CN109918195A (en) * 2019-01-18 2019-06-21 华南理工大学 Many-core system processor resource dispatching method based on thermal sensing dynamic task migrating
CN111651253A (en) * 2020-05-28 2020-09-11 中国联合网络通信集团有限公司 Computing resource scheduling method and device
CN111966500A (en) * 2020-09-07 2020-11-20 网易(杭州)网络有限公司 Resource scheduling method and device, electronic equipment and storage medium
CN112181665A (en) * 2020-10-21 2021-01-05 中国联合网络通信集团有限公司 Task optimization method and device for low-earth-orbit satellite
CN113448721A (en) * 2020-03-27 2021-09-28 ***通信有限公司研究院 Network system for computing power processing and computing power processing method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102232282A (en) * 2010-10-29 2011-11-02 华为技术有限公司 Method and apparatus for realizing load balance of resources in data center
CN109144710A (en) * 2017-06-16 2019-01-04 ***通信有限公司研究院 Resource regulating method, device and computer readable storage medium
CN109918195A (en) * 2019-01-18 2019-06-21 华南理工大学 Many-core system processor resource dispatching method based on thermal sensing dynamic task migrating
CN113448721A (en) * 2020-03-27 2021-09-28 ***通信有限公司研究院 Network system for computing power processing and computing power processing method
CN111651253A (en) * 2020-05-28 2020-09-11 中国联合网络通信集团有限公司 Computing resource scheduling method and device
CN111966500A (en) * 2020-09-07 2020-11-20 网易(杭州)网络有限公司 Resource scheduling method and device, electronic equipment and storage medium
CN112181665A (en) * 2020-10-21 2021-01-05 中国联合网络通信集团有限公司 Task optimization method and device for low-earth-orbit satellite

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"实时多任务异构云计算平台负载均衡算法";徐爱萍等;《中国科学技术大学学报》;20160331;第46卷(第3期);第215-221页 *

Also Published As

Publication number Publication date
CN113867973A (en) 2021-12-31

Similar Documents

Publication Publication Date Title
WO2022037337A1 (en) Distributed training method and apparatus for machine learning model, and computer device
CN113867973B (en) Resource allocation method and device
US10243872B2 (en) Management of storage cluster performance with hybrid workloads
CN102402399A (en) Virtual machine memory management in systems with asymmetric memory
CN107301093B (en) Method and device for managing resources
US11496413B2 (en) Allocating cloud computing resources in a cloud computing environment based on user predictability
CN111966496B (en) Data processing method, device, system and computer readable storage medium
US10616134B1 (en) Prioritizing resource hosts for resource placement
CN111722806A (en) Cloud disk allocation method and device, electronic equipment and storage medium
CN112068957B (en) Resource allocation method, device, computer equipment and storage medium
CN112689007B (en) Resource allocation method, device, computer equipment and storage medium
CN111666131A (en) Load balancing distribution method and device, computer equipment and storage medium
CN115033340A (en) Host selection method and related device
CN113608869A (en) Task scheduling method and device, electronic equipment and computer storage medium
CN116467082A (en) Big data-based resource allocation method and system
CN116048797A (en) Data processing method, device, electronic equipment and storage medium
US10990519B2 (en) Multi-tenant cloud elastic garbage collector
CN117311998B (en) Large model deployment method and system
US10812408B1 (en) Preventing concentrated selection of resource hosts for placing resources
US10594620B1 (en) Bit vector analysis for resource placement in a distributed system
CN112286623B (en) Information processing method and device and storage medium
US11080092B1 (en) Correlated volume placement in a distributed block storage service
CN111510479A (en) Resource allocation method and device for heterogeneous cache system
CN114884900A (en) Resource allocation method, device, equipment and storage medium
CN107977270A (en) Peers distribution method, peers distribution system and computer installation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant