WO2024055168A1 - 一种资源分配方法、处理器和计算平台 - Google Patents
一种资源分配方法、处理器和计算平台 Download PDFInfo
- Publication number
- WO2024055168A1 WO2024055168A1 PCT/CN2022/118522 CN2022118522W WO2024055168A1 WO 2024055168 A1 WO2024055168 A1 WO 2024055168A1 CN 2022118522 W CN2022118522 W CN 2022118522W WO 2024055168 A1 WO2024055168 A1 WO 2024055168A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- computing
- application
- computing unit
- units
- allocated
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 102
- 238000013468 resource allocation Methods 0.000 title claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 40
- 230000002596 correlated effect Effects 0.000 claims abstract description 7
- 230000008569 process Effects 0.000 claims description 59
- 238000004364 calculation method Methods 0.000 claims description 31
- 230000000875 corresponding effect Effects 0.000 claims description 21
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 230000003068 static effect Effects 0.000 abstract description 13
- 238000011176 pooling Methods 0.000 abstract description 4
- 238000013461 design Methods 0.000 description 31
- 238000010586 diagram Methods 0.000 description 22
- 239000011159 matrix material Substances 0.000 description 11
- 238000004590 computer program Methods 0.000 description 7
- 238000013473 artificial intelligence Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000007792 addition Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical group OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 101100317378 Mus musculus Wnt3 gene Proteins 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000002316 cosmetic surgery Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L27/00—Modulated-carrier systems
Definitions
- the present application relates to the field of computer technology, and in particular, to a resource allocation method, processor and computing platform.
- NPU graphics processing unit
- NPU neural network processor
- Figure 1 is a schematic diagram of the system on chip (SoC) of the Xavier chip, including GPU, CPU, vision accelerator, deep learning accelerator (DLA), video encoder, ( video encoder), video decoder (video decoder), camera ingest (camera ingest), Internet service provider (internet server provider, ISP), etc.
- the GPU includes multiple independent stream multiprocessors (SM), each SM contains a universal parallel computing architecture (compute unified device architecture, CUDA) core, and each CUDA core All contain a variety of computing units.
- SM system on chip
- CUDA compute unified device architecture
- FIG. 2 is a schematic diagram of the SoC of Huawei's Ascend chip, including NPU, CPU, task scheduler, network card, universal serial bus (USB) interface, external memory interface, and high-speed serial computer extension Bus (peripheral component interconnect express, PCIE) interface, general-purpose input/output (GPIO) interface, etc.
- the NPU includes multiple AI Cores, and each AI core stacks the same computing resources, such as the same type and the same number of tensor computing units, vector computing units, etc.
- the present application provides a resource allocation method, a processor, and a computing platform to improve the resource utilization of chips including GPUs, NPUs, etc., and reduce chip area and static power consumption losses.
- a processor in a first aspect, includes a computing unit pool, and the computing unit pool includes multiple computing units; each idle computing unit in the multiple computing units can be called on demand, and among the multiple computing units The number of different types of computing units is positively related to the number of times that type of computing unit is called.
- the processor may be a processing chip used to implement parallel computing, such as an NPU or a GPU or other processing chips used to implement parallel computing.
- This application pools computing units in processing chips, including NPU and GPU, which are used to implement large-scale parallel calculations, so that each computing unit can be called on demand by upper-layer applications when it is idle, which can improve resource utilization. Reduce the static power loss of the computing unit and reduce the chip area.
- all computing units in the processor are pooled, that is, all computing units in the processor are included in the computing unit pool.
- All computing units of the processor are pooled, which can minimize the static power loss of the processor and reduce the chip area.
- Some computing units of the processor are pooled, which can reduce static power loss and reduce the chip area of the processor. At the same time, retaining a small number of computing units from being pooled (for example, in the form of AI core) can make the processor have good forward compatibility.
- types correspond to algorithms executed by multiple computing units.
- the types of computing units can be divided according to the algorithms executed by the computing units.
- the type of the computing unit used to perform the addition (add) algorithm is the add computing unit
- the type of the computing unit used to perform the logarithm (log) algorithm is the type log computing unit, and so on.
- each of the multiple computing units corresponds to a low-level computing instruction.
- the add calculation unit corresponds to the addition instruction
- the log calculation unit corresponds to the log instruction. It is understandable that in actual applications, different computing units can correspond to the same underlying operation instructions. The scale of the underlying operations varies according to the capabilities of the processor. It is specifically designed according to the chip specifications of the processor. The same underlying operation instructions may be used at the bottom layer. Executed in parallel by multiple computing units.
- the computing unit pool includes one or more of a tensor computing unit pool, a vector computing unit pool, and a scalar computing unit pool.
- the processor is configured to: receive a request to execute the first application; determine the computing resource requirements of the first application, and allocate computing units to the first application from the computing unit pool according to the computing resource requirements to implement the first application,
- Computing resource requirements include one or more of tensor computing units, vector computing units, and vector computing units, as well as the quantity of each computing unit.
- resources can be dynamically scheduled for upper-layer applications according to their requests, which can improve the flexibility of resource allocation and improve resource utilization.
- the processor is also used to: determine multiple processes corresponding to the first application; determine the computing resource requirements of each of the multiple processes, and obtain the computing resource requirements from the computing unit pool according to the computing resource requirements of each process.
- Each process is allocated a computing unit to implement the first application; wherein the computing unit allocated to each process is different from the computing units allocated to other processes.
- the processor is further configured to: determine whether the computing unit allocated to the first application meets the needs of the first application; when the computing unit allocated to the first application cannot meet the needs of the first application, adjust the computing unit allocated to the first application.
- the computing resource allocation result of an application is allocated to the first application from the computing unit pool according to the adjusted computing resource allocation result.
- the processor is further configured to establish a correspondence between the first application and the computing unit allocated to the first application.
- a resource allocation method is provided, which is applied to a processor.
- the processor includes a computing unit pool, and the computing unit pool includes multiple computing units; each idle computing unit in the multiple computing units can be called on demand, And the number of different types of computing units in the plurality of computing units is positively correlated with the number of times that type of computing unit is called; the method includes: receiving a request to execute the first application; determining the computing resource requirements of the first application, and starting from Computing units are allocated to the first application in the computing unit pool to implement the first application.
- Computing resource requirements include one or more of tensor computing units, vector computing units, and vector computing units, as well as the number of each computing unit.
- determining the computing resource requirements of the first application, and allocating computing units to the first application from the computing unit pool according to the computing resource requirements to implement the first application includes: determining multiple processes corresponding to the first application; Determine the computing resource requirements of each process in the plurality of processes, and allocate computing units to each process from the computing unit pool according to the computing resource requirements of each process to implement the first application; wherein the computing unit allocated to each process is equal to Other processes are allocated different compute units.
- the method further includes: determining whether the computing unit allocated to the first application meets the needs of the first application; when the computing unit allocated to the first application cannot meet the needs of the first application, adjusting the first application the computing resource allocation result, and allocates a computing unit to the first application from the computing unit pool according to the adjusted computing resource allocation result.
- the method further includes: establishing a correspondence between the first application and the computing unit allocated to the first application.
- a third aspect provides a processing device, which includes modules/units/technical means for executing the method described in the second aspect or any possible design of the second aspect.
- the device may include: a transceiver module, configured to receive a request to execute the first application; a processing module, configured to determine the computing resource requirements of the first application, and select the computing unit pool for the first application from the computing unit pool of the processor according to the computing resource requirements.
- An application allocates computing units to implement the first application.
- the computing resource requirements include one or more of tensor computing units, vector computing units, and vector computing units, and the number of each computing unit; wherein the computing unit pool includes multiple computing units. computing units; each idle computing unit among the multiple computing units can be called on demand, and the number of different types of computing units among the multiple computing units is positively related to the number of times that type of computing unit is called.
- the processing module can also be used to: determine multiple processes corresponding to the first application; determine the computing resource requirements of each of the multiple processes, and select the computing resource from the computing unit pool according to the computing resource requirements of each process. Allocating a computing unit to each process in to implement the first application; wherein the computing unit allocated to each process is different from the computing units allocated to other processes.
- the processing module may also be used to: determine whether the computing unit allocated to the first application meets the needs of the first application; when the computing unit allocated to the first application cannot meet the needs of the first application, Adjust the computing resource allocation result of the first application, and allocate computing units to the first application from the computing unit pool according to the adjusted computing resource allocation result.
- the processing module may also be used to establish a correspondence between the first application and the computing unit allocated to the first application.
- a fourth aspect provides a computing platform, including the processor described in the first aspect and the processing device described in the third aspect.
- a computer-readable storage medium stores computer-executable instructions.
- the computer-executable instructions make it possible as in the second aspect or any one of the second aspects.
- the method described in the design is implemented.
- Figure 1 is a schematic diagram of a SoC
- FIG. 2 is a schematic diagram of another SoC
- Figure 3 is a schematic diagram of a processor provided by an embodiment of the present application.
- Figure 4 is a schematic diagram showing that all computing units in the NPU are pooled
- Figure 5 is a schematic diagram showing that some computing units in the NPU are pooled
- Figure 6 is a statistical chart of the usage frequency of operators
- Figure 7 is a flow chart of a resource allocation method provided by an embodiment of the present application.
- Figure 8A is a schematic diagram of a possible resource allocation provided by an embodiment of the present application.
- Figure 8B is a schematic diagram of another possible resource allocation provided by an embodiment of the present application.
- Figure 9 is a schematic diagram of a possible computing unit pool 11 provided by the embodiment of the present application.
- Figure 10 is a schematic diagram of a processing device provided by an embodiment of the present application.
- Figure 11 is a schematic diagram of a computing platform provided by an embodiment of the present application.
- the processor 01 includes a computing unit pool 11 , and the computing unit pool 11 includes a plurality of computing units 111 .
- the computing units in the computing unit pool 11 are pooled computing units, or computing units that can be shared, and can be called by different upper-layer applications, for example.
- each computing unit in the computing unit pool 11 when it is not called (or not running), it is an idle computing unit and can be called by upper-layer applications.
- Each idle computing unit in the plurality of computing units 111 can be called on demand. Specifically, it can be called according to the needs of upper-layer applications. That is, if any upper-layer application needs it and there are idle computing units in the computing unit pool 11, then The idle computing unit can be called by the upper-layer application.
- the processor 01 may be a processing chip used to implement parallel computing, such as an NPU or a GPU or other processing chips used to implement parallel computing.
- the multiple computing units 111 in the computing unit pool 11 may be computing units in a chip used to implement parallel computing, such as computing units in an NPU or a GPU.
- the computing unit pool 11 can be pooled by computing units in an NPU or a GPU or other processing chips used to implement parallel computing.
- the above solution pools computing units in processing chips, including NPU and GPU, which are used to implement large-scale parallel computing, so that each computing unit can be called on demand by upper-layer applications when it is idle.
- resource utilization can be improved. Under the same resource usage, the static power consumption loss of the computing unit can be reduced and the chip area of the processor 01 can be reduced.
- the NPU is mainly used as an example. But the same implementation can be applied to other chips such as GPUs.
- all computing units in the NPU are pooled, that is, all computing units in the NPU are included in the computing unit pool 11 .
- all computing units 111 in the NPU are in the computing unit pool 11 .
- every idle computing unit among all computing units of the NPU can be called on demand.
- only some of the computing units in the NPU are pooled, that is, only some of the computing units are included in the computing unit pool 11 .
- some computing units 111 in the NPU are in the computing unit pool 11, and other computing units 111 are not in the computing unit pool 11, but are configured in the AI Core.
- the computing unit pool 11 may include different types of computing units, where the computing unit type corresponds to the algorithm executed by the computing unit.
- the number of different types of computing units 111 in the plurality of computing units 111 is positively related to the number of times a type of computing unit 111 is called. In other words, the more times a type of computing unit 111 is called, the more likely it is in the computing unit pool.
- each computing unit 111 corresponds to a low-level operation instruction of the processor 01.
- each computing unit 111 among the plurality of computing units 111 corresponds to a low-level operation instruction.
- different computing units 111 can correspond to the same underlying operation instructions, and the scale of the underlying operations varies according to the capabilities of the processor 01, and is specifically designed according to the chip specifications of the processor 01.
- an add instruction when the amount of data to be processed corresponding to the instruction is large, in order to improve the processing capability of processor 01 at the same time (that is, the degree of parallelism), multiple add computing units can be used at the bottom to perform parallel processing on different data. calculation, so for the same add instruction, the underlying layer may perform multiple additions.
- the computing resource pool can be configured according to the resource situation that meets the actual requirements by analyzing and counting the proportion of various computing units used by various operators in the model (such as the neural network model) run by processor 01 in actual applications. 11 The number of various types of computing units. It can be understood that one operator can correspond to one or more computing units 111.
- Figure 6 is a statistical chart of the frequency of use of various operators in the model. The functions of each operator in Figure 6 are explained as follows:
- Slice slice
- BatchNormalization batch normalization
- ConstantOfShape generates a tensor with a given value and shape
- LeakyRelu leakyretified linear unit
- GlobalAveragePool global average pooling
- ConvTranspose transposed convolution, also known as Deconvolution
- Equal Determine whether the sequence is equal
- ReduceMean reduction layer in convolutional neural network
- each type of computing unit 111 may be slightly greater than the number corresponding to the number of calls of that type of computing unit 111 to ensure that the processor 01 has sufficient performance to cope with emergencies.
- the static performance of the processor 01 can be reduced while ensuring the running performance of the processor 01. Power loss.
- the computing unit 111 is classified into types.
- multiple computing units 111 may include a tensor computing unit (or matrix computing unit) 111A, a vector computing unit (or vector computing unit) 111B, a scalar computing unit One or more types of unit 111C, etc.
- the tensor calculation unit 111A is used to perform matrix calculations
- the vector calculation unit 111B is used to perform vector calculations
- the scalar calculation unit 111C is used to perform scalar calculations.
- the multiple computing units 111 may include an eight-bit integer (int8) computing unit and a 16-bit floating point (fp16) computing unit.
- Int8 an eight-bit integer
- fp16 16-bit floating point
- One or more types of unit 32-bit floating point (fp32) calculation unit, 4-bit integer (int4) calculation unit, etc.
- all calculation units 11 are first divided into tensor calculation units 111A, vector calculation units 111B, and scalar calculation units 111C, and then further subdivided according to mathematical operation types and/or data types in the tensor calculation unit 111A.
- the processor 01 can schedule resources for the upper-layer application from the computing unit pool 11 when receiving a request from the upper-layer application.
- FIG. 7 is a flow chart of a resource allocation method provided by an embodiment of the present application.
- the method may be executed by the processor 01 in FIG. 3 , or may be executed by other processing devices other than the processor 01 .
- the method includes:
- S702. Determine the computing resource requirements of the first application; allocate computing units to the first application from the computing unit pool according to the computing resource requirements to implement the first application.
- the computing resource requirements include tensor computing units, vector computing units and vector computing units. One or more types of computing units, and the quantity of each type of computing unit.
- the NPU in the processor 01 or other specially configured processing unit for resource allocation may be used to execute the above method, which is not limited in this application.
- the computing unit allocated to each process is different from the computing units allocated to other processes.
- different processes in the plurality of processes are allocated to different computing units.
- FIG. 8A is a schematic diagram of a possible resource allocation.
- add, sub, mul, exp, etc. each of these resources is placed in processor 01, and matrix multiplication is placed in multiple copies.
- application 0 and application n are to be executed at the same time.
- the computing resources required by application 0 include matrix multiplication, add, and sub, and the computing resources required by application n include matrix multiplication, mul, and exp.
- the resources of can be shown as resource group 0, including matrix multiplication, add, and sub.
- the resources allocated for application n can be shown as resource group n, including matrix multiplication, mul, and exp. In this way, different types of computing resources can be allocated to different applications, which can achieve the purpose of reducing chip area, cost, and static power consumption without reducing the processing performance of the chip.
- FIG. 8B is a schematic diagram of another possible resource allocation.
- add, sub, mul, exp, etc. these resources are placed in two copies each in processor 01, and matrix multiplication is placed in multiple copies.
- application 0 and application n are to be executed at the same time.
- Application 0 has a high demand for add, while application n has a low demand for add.
- the resources allocated to application 0 can be as shown in resource group 0.
- the resources allocated for application n can be as shown in resource group n, including matrix multiplication, sub, mul, and exp. In this way, different amounts of computing resources can be allocated to different applications, which can achieve the purpose of reducing chip area, cost, and static power consumption without reducing the processing performance of the chip.
- resources can be dynamically scheduled for upper-layer applications based on their requests, improving the flexibility of resource allocation.
- the computing unit allocated to the first application may also be determined whether the computing unit allocated to the first application meets the needs of the first application; when the first application When the allocated computing unit cannot meet the needs of the first application, the computing resource allocation result of the first application is adjusted, and the computing unit is allocated to the first application from the computing unit pool 11 according to the adjusted computing resource allocation result.
- the computing resource allocation result of the first application is adjusted to add more computing units, such as The computing units 111b and 111c are assigned to the first application to meet the indicators required by the first application.
- a corresponding relationship between the first application and the computing unit allocated to the first application can also be established.
- the computing unit allocated to the first application is fixed (that is, changed from an idle computing unit to a non-idle computing unit) and is only used by the first application. And during the use of the first application, other applications cannot call this part of the resource.
- the part of the computing units may be released (for example, the corresponding relationship is deleted), so that the part of the computing units may be called by other applications.
- resource utilization can be further improved.
- the first application may not release the part of the computing units (for example, still retain the corresponding relationship), so that the part of the computing units is not called by other applications, but remains
- the first application retains this part of the computing units, so that when the first application runs next time, it can directly use this part of the computing units without the need to allocate resources again.
- the performance of the first application can be guaranteed first.
- the computing unit pool 11 includes one or more of a tensor computing unit pool, a vector computing unit pool, and a scalar computing unit pool.
- the computing unit pool 11 includes a tensor computing unit pool 11A, a vector computing unit pool 11B, and a scalar computing unit pool 11C.
- the tensor computing unit pool 11A includes one or more tensor computing units 111A.
- the vector calculation unit pool 11B includes one or more vector calculation units 111B, and the scalar calculation unit pool 11C includes one or more scalar calculation units 111C.
- a corresponding computing unit pool may be found first according to its corresponding algorithm dimension, and then the computing unit may be determined from the computing unit pool.
- the embodiment of the present application also provides a processing device 100.
- the processing device 100 includes modules/units/technical means for executing the method shown in Figure 7.
- the processing device 100 may include:
- Transceiver module 1001 configured to receive a request to execute the first application
- the processing module 1002 is used to determine the computing resource requirements of the first application, and allocate computing units to the first application from the computing unit pool of the processor according to the computing resource requirements to implement the first application.
- the computing resource requirements include tensor computing units, vector One or more of computing units and vector computing units, and the number of each computing unit; wherein the computing unit pool includes multiple computing units; each idle computing unit among the multiple computing units can be called on demand, And the number of different types of computing units in multiple computing units is positively correlated with the number of times that type of computing unit is called.
- an embodiment of the present application also provides a computing platform 1100 , including the above-mentioned processor 01 and the processing device 100 .
- embodiments of the present application also provide a computer-readable storage medium.
- the computer-readable storage medium stores computer-executable instructions. When called by a computer, the computer-executable instructions enable the method shown in Figure 7 be executed.
- embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
- computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
- These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions
- the device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.
- These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device.
- Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multi Processors (AREA)
Abstract
本申请公开了一种资源分配方法、处理器和计算平台,用以提高包括GPU、NPU等在内的芯片的资源利用率,减小芯片面积和静态功耗损失。处理器包括计算单元池,计算单元池包括多个计算单元;多个计算单元中的每个空闲计算单元可以被按需调用,且多个计算单元中不同类型的计算单元的数量与该类型的计算单元被调用次数正相关。本申请通过将包括NPU、GPU等处理芯片中的计算单元池化,使得每个计算单元在其空闲时可以被上层应用按需调用,可以提高资源利用率,减少计算单元的静态功耗损失,减小芯片面积。
Description
本申请涉及计算机技术领域,尤其涉及一种资源分配方法、处理器和计算平台。
随着机器学***台。NPU、GPU等芯片中的计算资源有多种,例如张量计算单元、向量计算单元、标量计算单元等。现有芯片厂商生产的NPU、GPU中,各类计算资源比例是固定的,而且同样的计算资源会复制很多份。
例如,参见图1,为Xavier芯片的片上***(system on chip,SoC)示意图,包括GPU、CPU、视觉加速器、(vision accelerator)、深度学***方根(sqrt),加法(add),减法(sub),乘法(mul)等),分别都对应单独的物理计算单元;甚至为了满足不同的数据类型(如8位整数(int8),16位浮点数(fp16),32位浮点数(fp32),4位整数(int4)等),而放置多种规格的计算单元,这些都需要在CUDA Core中堆叠多种资源。
例如,参见图2,为华为公司的昇腾芯片的SoC示意图,包括NPU、CPU、任务调度器、网卡、通用串行总线(universal serial bus,USB)接口、外存接口、高速串行计算机扩展总线(peripheral component interconnect express,PCIE)接口、通用输入输出(general-purpose input/output,GPIO)接口等。其中,NPU中包括有多个AI Core,而且每个AI core内部都堆叠相同的计算资源,如相同种类和相同数量的张量计算单元、矢量计算单元等。
然而,实际应用中,NPU、GPU等芯片在运行时,并不是所有计算资源都同时用到的,因此现有技术中NPU、GPU等芯片设计方式,不仅造成芯片面积的浪费,还存在静态功耗损失的问题。
如何提高包括NPU、GPU等在内的芯片的资源利用率,是本申请要解决的问题。
发明内容
本申请提供一种资源分配方法、处理器和计算平台,用以提高包括GPU、NPU等在内的芯片的资源利用率,减小芯片面积和静态功耗损失。
第一方面,提供一种处理器,该处理器包括计算单元池,计算单元池包括多个计算单元;多个计算单元中的每个空闲计算单元可以被按需调用,且多个计算单元中不同类型的计算单元的数量与该类型的计算单元被调用次数正相关。
可以理解的,处理器可以是用于实现并行计算的处理芯片,例如NPU或GPU或其它用于实现并行计算的处理芯片。
本申请通过将包括NPU、GPU在内的用于实现大量并行计算的处理芯片中的计算单元池化,使得每个计算单元在其空闲时可以被上层应用按需调用,可以提高资源利用率,减少计算单元的静态功耗损失,减小芯片面积。
一种可能的设计中,处理器中的所有计算单元都被池化,即处理器中的所有计算单元都包含在该计算单元池中。
处理器的所有计算单元都被池化,可以尽可能减少处理器的静态功耗损失,减小芯片面积。
另一种可能的设计中,处理器中仅部分计算单元都被池化,即只有部分计算单元包含在该计算单元池中。
处理器的部分计算单元被池化,可以减少静态功耗损失,减小处理器的芯片面积。同时保留少部分计算单元不被池化(例如以AI core的形式存在),可以使得处理器具有良好的前向兼容性。
一种可能的设计中,类型与多个计算单元执行的算法对应。换而言之,计算单元的类型可以根据计算单元执行的算法来划分。例如,用于执行加法(add)算法的计算单元的类型为add计算单元,用于执行对数(log)算法的计算单元的类型为log计算单元,等等。
一种可能的设计中,多个计算单元中的每个计算单元对应一个底层运算指令。例如,add计算单元对应加法指令,log计算单元对应log指令。可以理解的,实际应用中,不同的计算单元可以对应相同的底层运算指令,底层运算的规模根据处理器的能力不同而不同,具体根据处理器的芯片规格来设计,同一底层运算指令在底层可能由多个计算单元并行执行。
一种可能的设计中,计算单元池包括张量计算单元池、矢量计算单元池和标量计算单元池中的一个或多个。
将张量计算单元、矢量计算单元和标量计算单元分别单独进行池化,可以提高资源管理效率,以及提高上层应用调度资源的效率。
一种可能的设计中,处理器用于:接收执行第一应用的请求;确定第一应用的计算资源需求,根据计算资源需求从计算单元池中为第一应用分配计算单元以实现第一应用,计算资源需求包括张量计算单元、矢量计算单元和矢量计算单元中的一种或多种,以及每种计算单元的数量。
通过上述方式,可以实现根据上层应用的请求为其动态调度资源,可以提高资源分配的灵活性,提高资源利用率。
一种可能的设计中,处理器还用于:确定第一应用对应的多个进程;确定多个进程中每个进程的计算资源需求,根据每个进程的计算资源需求从计算单元池中为每个进程分配计算单元以实现第一应用;其中,为每个进程分配的计算单元与为其他进程分配的计算单元不同。
通过上述方式,可以实现根据第一应用的每个进程的计算资源需求从计算单元池中为 每个进程分配计算单元,可以提高资源利用率。
一种可能的设计中,处理器还用于:确定为第一应用分配的计算单元是否满足第一应用的需要;当为第一应用分配的计算单元不能满足第一应用的需要时,调整第一应用的计算资源分配结果,并根据调整后的计算资源分配结果从计算单元池中为第一应用分配计算单元。
一种可能的设计中,处理器还用于:建立第一应用与为第一应用分配的计算单元的对应关系。
通过上述方式,可以保证为第一应用分配的资源能够满足其需求,可以提高资源分配的可靠性。
第二方面,提供一种资源分配方法,应用于处理器,该处理器包括计算单元池,计算单元池包括多个计算单元;多个计算单元中的每个空闲计算单元可以被按需调用,且多个计算单元中不同类型的计算单元的数量与该类型的计算单元被调用次数正相关;方法包括:接收执行第一应用的请求;确定第一应用的计算资源需求,根据计算资源需求从计算单元池中为第一应用分配计算单元以实现第一应用,计算资源需求包括张量计算单元、矢量计算单元和矢量计算单元中的一种或多种,以及每种计算单元的数量。
一种可能的设计中,确定第一应用的计算资源需求,根据计算资源需求从计算单元池中为第一应用分配计算单元以实现第一应用,包括:确定第一应用对应的多个进程;确定多个进程中每个进程的计算资源需求,根据每个进程的计算资源需求从计算单元池中为每个进程分配计算单元以实现第一应用;其中,为每个进程分配的计算单元与为其他进程分配的计算单元不同。
一种可能的设计中,方法还包括:确定为第一应用分配的计算单元是否满足第一应用的需要;当为第一应用分配的计算单元不能满足第一应用的需要时,调整第一应用的计算资源分配结果,并根据调整后的计算资源分配结果从计算单元池中为第一应用分配计算单元。
一种可能的设计中,方法还包括:建立第一应用与为第一应用分配的计算单元的对应关系。
第三方面,提供一种处理装置,该装置包括用于执行第二方面或第二方面任一种可能的设计中所述的方法的模块/单元/技术手段。
示例性的,该装置可以包括:收发模块,用于接收执行第一应用的请求;处理模块,用于确定第一应用的计算资源需求,根据计算资源需求从处理器的计算单元池中为第一应用分配计算单元以实现第一应用,计算资源需求包括张量计算单元、矢量计算单元和矢量计算单元中的一种或多种,以及每种计算单元的数量;其中,计算单元池包括多个计算单元;多个计算单元中的每个空闲计算单元可以被按需调用,且多个计算单元中不同类型的计算单元的数量与该类型的计算单元被调用次数正相关。
一种可能的设计中,处理模块,还可以用于:确定第一应用对应的多个进程;确定多个进程中每个进程的计算资源需求,根据每个进程的计算资源需求从计算单元池中为每个进程分配计算单元以实现第一应用;其中,为每个进程分配的计算单元与为其他进程分配的计算单元不同。
一种可能的设计中,处理模块,还可以用于:确定为第一应用分配的计算单元是否满足第一应用的需要;当为第一应用分配的计算单元不能满足第一应用的需要时,调整第一 应用的计算资源分配结果,并根据调整后的计算资源分配结果从计算单元池中为第一应用分配计算单元。
一种可能的设计中,处理模块,还可以用于:建立第一应用与为第一应用分配的计算单元的对应关系。
第四方面,提供一种计算平台,包括上述第一方面所述的处理器以及上述第三方面所述的处理装置。
第五方面,提供一种计算机可读存储介质,该计算机可读存储介质存储有计算机可执行指令,该计算机可执行指令在被计算机调用时,使得如第二方面或第二方面任一种可能的设计中所述的方法被执行。
图1为一种SoC的示意图;
图2为另一种SoC的示意图;
图3为本申请实施例提供的一种处理器的示意图;
图4为NPU中的所有计算单元都被池化的示意图;
图5为NPU中的部分计算单元都被池化的示意图;
图6为算子的使用频度的统计图;
图7为本申请实施例提供的一种资源分配方法的流程图;
图8A为本申请实施例提供的一种可能的资源分配示意图;
图8B为本申请实施例提供的另一种可能的资源分配示意图;
图9为本申请实施例提供的一种可能的计算单元池11的示意图;
图10为本申请实施例提供的一种处理装置的示意图;
图11为本申请实施例提供的一种计算平台的示意图。
下面将结合附图,对本申请实施例进行详细描述。
参见图3,为本申请实施例提供的一种处理器01的示意图,处理器01中包括计算单元池11,计算单元池11包括多个计算单元111。
可以理解的,计算单元池11中的计算单元,是被池化的计算单元,或者说是可以共享的计算单元,例如可以被不同的上层应用调用。具体实现时,对于计算单元池11中的每个计算单元,当其未被调用时(或者说未运行时),为空闲计算单元,可以被上层应用调用。多个计算单元111中的每个空闲计算单元,可以被按需调用,具体例如是按照上层应用的需要被调用,即任何上层应用若有需要,而计算单元池11也存在空闲计算单元,则该空闲的计算单元可以被该上层应用调用。
处理器01可以是用于实现并行计算的处理芯片,例如NPU或GPU或其它用于实现并行计算的处理芯片。计算单元池11中的多个计算单元111可以是用于实现并行计算的芯片中的计算单元,例如是NPU或GPU中的计算单元。换而言之,计算单元池11可以由NPU或GPU或其它用于实现并行计算的处理芯片中的计算单元进行池化得到。
上述方案,通过将包括NPU、GPU在内的用于实现大量并行计算的处理芯片中的计 算单元池化,使得每个计算单元在其空闲时可以被上层应用按需调用,相比现有技术中以SM或AI Core等设计方式堆叠多种相同的计算资源而言,可以提高资源利用率。在相同的资源使用量的情况下,可以减少计算单元的静态功耗损失,减小处理器01的芯片面积。
为了便于描述,在下文中,主要以NPU为例进行描述。但相同的实施方式,可以应用于GPU等其他芯片。
一种可能的设计中,NPU中的所有计算单元都被池化,即NPU中的所有计算单元都包含在该计算单元池11中。例如图4所示,NPU中的全部计算单元111都在计算单元池11中。换而言之,NPU的所有计算单元中的每个空闲计算单元都可以被按需调用。
通过上述方式,NPU的所有计算单元都被池化,可以尽可能减少处理器01的静态功耗损失,减小处理器01的芯片面积。
另一种可能的设计中,NPU中仅部分计算单元都被池化,即只有部分计算单元包含在该计算单元池11中。例如图5所示,NPU中的一部分计算单元111在计算单元池11中,另部分计算单元111不在计算单元池11中,而是被配置在AI Core中。
通过上述方式,NPU的部分计算单元被池化,可以减少静态功耗损失,减小处理器01的芯片面积。同时保留少部分计算单元以AI core的形式存在,可以使得处理器01具有良好的前向兼容性。
一种可能的设计中,计算单元池11中可以包括不同类型的计算单元,其中计算单元类型与计算单元执行的算法对应。多个计算单元111中不同类型的计算单元111的数量与类型的计算单元111被调用次数正相关,换而言之,哪种类型的计算单元111被调用的次数越多,其在计算单元池11中的数量越多,或者说,哪种算法被调用的次数越多,则该算法对应的计算单元111在计算单元池11中的数量越多。例如,add计算单元被调用的多,那么计算单元池11中可以有更多add计算单元,log计算单元被调用的少,那么计算单元池11中可以有更少的log计算单元。
在具体实现时,每个计算单元111都对应处理器01的一个底层运算指令。换而言之,多个计算单元111中的每个计算单元111对应一个底层运算指令。可以理解的,实际应用中,不同的计算单元111可以对应相同的底层运算指令,底层运算的规模根据处理器01的能力不同而不同,具体根据处理器01的芯片规格来设计。例如,对于一个add指令,该指令对应的待处理数据量较多时,为了提高处理器01在同一时刻的处理能力(即并行度),在底层可以由多个add计算单元针对不同的数据进行并行计算,因此针对同一add指令,底层可能会执行多次相加。
作为一种示例,可以通过分析统计实际应用中处理器01运行的模型(如神经网络模型)中各类算子用到的各类计算单元的比例,按满足实际要求的资源情况配置计算资源池11中的各类计算单元的数量。可以理解的,一个算子可以对应一个或多个计算单元111。
例如,参见图6,为模型中各类算子的使用频度的统计图,图6中各算子的功能解释如下:
Conv:卷积(convolution);
Relu:线性整流单元(rectified linear unit);
Dequant:反量化;
Mul:乘法;
Slice:切分(slice);
Gather:汇聚;
Shape:整形;
BatchNormalization:批规范化;
Resize:缩放;
Pad:数据扩充;
ConstantOfShape:用给定的值和形状(shape)生成张量;
Tanh:激活函数;
LeakyRelu:漏洞修正线性单元(leakyretified linear unit);
GlobalAveragePool:全局平均池化;
Floor:向下取整;
ConvTranspose:转置卷积,亦称Deconvolution;
Softmax:归一化;
Exp:指数;
Flatten:降维;
Equal:判断序列相等;
Expand:扩展;
MatMul:矩阵乘法运算;
And:与运算;
ReduceMean:卷积神经网络中的还原层;
ReduceMax:取最大值。
可以理解的,图6中仅示出了部分算子类型,实际应用中还可以有其它算子类型,本申请不做限制。
从图6可以看出,Conv、Relu、AscendDequant等算子的使用频度较高,因此可以在计算单元池11中为Conv、Relu、AscendDequant等算子配置较多的计算单元111,而Softmax、Exp、Flatten、Equal、Expand、MatMul、And、ReduceMean、ReduceMax等算子的使用频度较低,可以在计算单元池11中为Conv、Relu、AscendDequant等算子配置较少的计算单元111。
可以理解的,在具体实施时,每种类型的计算单元111的实际数量可以略大于与该类型的计算单元111被调用次数对应的数量,以保证处理器01具有足够的性能应对突发情况。
该设计方式,通过配置计算单元池11中不同类型的计算单元111的数量与类型的计算单元111被调用次数正相关,可以在保证处理器01的运行性能的前提下,减少处理器01的静态功耗损失。
需要说明的是,以上是以底层运算指令(或者说计算单元111执行的算法类型,或计算单元111执行的数学运算类型)来区分计算单元111的类型,实际应用时,还可以从其它方面来对计算单元111进行类型划分。
一种可能的示例中,从算法的维度上划分,多个计算单元111可以包括张量计算单元(或者称为矩阵计算单元)111A、矢量计算单元(或者称为向量计算单元)111B、标量计算单元111C等中的一种或多种类型。其中,张量计算单元111A用于执行矩阵计算,矢量计算单元111B用于执行向量计算,标量计算单元111C用于执行标量计算。
另一种可能的示例中,一种可能的示例中,从算法对应的数据类型上划分,多个计算 单元111可以包括八位的整数(int8)计算单元,16位的浮点数(fp16)计算单元,32位的浮点数(fp32)计算单元,4位的整数(int4)计算单元等中的一种或多种类型。
可以理解的,以上几种划分计算单元类型的方式仅为示例,而非限定,实际还有其它划分方式。
在实际应用时,上述几种类型划分方式可以相互结合。例如,先将所有计算单元11按照张量计算单元111A、矢量计算单元111B、标量计算单元111C进行划分,然后在张量计算单元111A中进一步按照数学运算类型和/或数据类型进行细分。
一种可能的设计中,处理器01可以在收到上层应用的请求时,从计算单元池11中为上层应用调度资源。
示例性的,参见图7,为本申请实施例提供的一种资源分配方法的流程图,该方法可以由图3中的处理器01执行,也可以由处理器01外的其他处理装置执行。该方法包括:
S701、接收执行第一应用的请求;
S702、确定第一应用的计算资源需求;根据计算资源需求从计算单元池中为第一应用分配计算单元以实现第一应用,计算资源需求包括张量计算单元、矢量计算单元和矢量计算单元中的一种或多种,以及每种计算单元的数量。
当上述方法由处理器01执行时,具体可以是处理器01中的NPU或者其他专门设置的用于实现资源分配的处理单元来执行上述方法,本申请不做限制。
可选的,可以先确定第一应用对应的多个进程;然后确定多个进程中每个进程的计算资源需求,根据每个进程的计算资源需求从计算单元池中为每个进程分配计算单元以实现第一应用;其中,为每个进程分配的计算单元与为其他进程分配的计算单元不同,换而言之,该多个进程中不同进程被分配到不同的计算单元。
作为一种示例,参见图8A,为一种可能的资源分配示意图。在芯片设计阶段,add,sub,mul,exp等,这些资源在处理器01中各放一份,矩阵乘放置多份。假设现在要同时执行应用0和应用n,其中应用0需要的计算资源有矩阵乘、add、sub,应用n需要的计算资源有矩阵乘、mul、exp,则在分配资源时,为应用0分配的资源可以如资源组0所示,包含矩阵乘、add、sub,为应用n分配的资源可以如资源组n所示,包含矩阵乘、mul、exp。这样,可以为不同应用分配不同类型的计算资源,能达到降低芯片面积,降低成本,降低静态功耗的目的,同时又不降低芯片的处理性能。
作为一种示例,参见图8B,为另一种可能的资源分配示意图。在芯片设计阶段,add,sub、mul、exp等,这些资源在处理器01中各放两份,矩阵乘放置多份。假设现在要同时执行应用0和应用n,其中应用0对add的需求高,而应用n对add的需求不高,则在分配资源时,为应用0分配的资源可以如资源组0所示,包含矩阵乘、add、sub、mul、exp,其中add有两份,为应用n分配的资源可以如资源组n所示,包含矩阵乘、sub、mul、exp。这样,可以为不同应用分配不同数量的计算资源,能达到降低芯片面积,降低成本,降低静态功耗的目的,同时又不降低芯片的处理性能。
通过该设计,可以根据上层应用的请求为其动态调度资源,提高资源分配的灵活性。
一种可能的设计中,在根据计算资源需求从计算单元池中为第一应用分配计算单元之后,还可以确定为第一应用分配的计算单元是否满足第一应用的需要;当为第一应用分配的计算单元不能满足第一应用的需要时,调整第一应用的计算资源分配结果,并根据调整后的计算资源分配结果从计算单元池11中为第一应用分配计算单元。
例如,如果为第一应用分配的计算单元111a在进行计算时,不能满足第一应用要求的指标(如时延),则调整第一应用的计算资源分配结果,将更多的计算单元,例如计算单元111b和111c分配给第一应用,以满足第一应用要求的指标。
通过上述方式,可以保证为第一应用分配的资源能够满足其需求,提高资源分配的可靠性。
一种可能的设计中,还可以建立第一应用与为第一应用分配的计算单元的对应关系。换而言之,在为第一应用分配好计算单元之后,将为第一应用分配的计算单元固定下来(即从空闲的计算单元变成非空闲的计算单元),只供第一应用使用,且在第一应用使用期间,其他应用无法调用该部分资源。
通过上述方式,可以避免将为第一应用分配的资源分配给其他资源引起第一应用中断的问题,可以保证第一应用稳定运行。
可选的,当第一应用使用完为第一应用分配的计算单元之后,可以释放该部分计算单元例如删除该对应关系),使得该部分计算单元可以被其他应用调用。通过上述方式,可以进一步提高资源利用率。
可选的,第一应用使用完为第一应用分配的计算单元之后,可以不释放该部分计算单元(例如仍然保留该对应关系),使得该部分计算单元不被其他应用调用,而是一直为第一应用保留该部分计算单元,这样第一应用下次运行时,可直接使用该部分计算单元,无需再次进行资源分配。通过上述方式,可以优先保证第一应用的性能。
一种可能的设计中,计算单元池11包括张量计算单元池、矢量计算单元池和标量计算单元池中的一个或多个。
例如,参见图9所示,计算单元池11包括张量计算单元池11A、矢量计算单元池11B和标量计算单元池11C,张量计算单元池11A中包括一个或多个张量计算单元111A,矢量计算单元池11B中包括一个或多个矢量计算单元111B,标量计算单元池11C中包括一个或多个标量计算单元111C。
在为第一应用分配计算单元时,可以先根据其对应的算法维度找到对应的计算单元池,然后再从该计算单元池中确定计算单元。
将张量计算单元、矢量计算单元和标量计算单元分别单独进行池化,可以提高资源管理效率,以及提高上层应用调度资源的效率。
可以理解的,上述各设计方式可以分别单独实施,也可以相互结合实施。
基于相同的技术构思,本申请实施例还提供一种处理装置100,处理装置100包括用于执行图7所示的方法的模块/单元/技术手段。
示例性的,参见图10,处理装置100可以包括:
收发模块1001,用于接收执行第一应用的请求;
处理模块1002,用于确定第一应用的计算资源需求,根据计算资源需求从处理器的计算单元池中为第一应用分配计算单元以实现第一应用,计算资源需求包括张量计算单元、矢量计算单元和矢量计算单元中的一种或多种,以及每种计算单元的数量;其中,计算单元池包括多个计算单元;多个计算单元中的每个空闲计算单元可以被按需调用,且多个计算单元中不同类型的计算单元的数量与该类型的计算单元被调用次数正相关。
应理解,上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述,在此不再赘述。
基于相同的技术构思,参见图11,本申请实施例还提供一种计算平台1100,包括上文所述的处理器01和处理装置100。
基于相同的技术构思,本申请实施例还提供一种计算机可读存储介质,计算机可读存储介质存储有计算机可执行指令,计算机可执行指令在被计算机调用时,使得如图7所示的方法被执行。
本领域内的技术人员应明白,本申请的实施例可提供为方法、***、或计算机程序产品。因此,本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且,本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请的方法、设备(***)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然,本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的保护范围。这样,倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内,则本申请也意图包含这些改动和变型在内。
Claims (17)
- 一种处理器,其特征在于,所述处理器包括计算单元池,所述计算单元池包括多个计算单元;所述多个计算单元中的每个空闲计算单元可以被按需调用,且所述多个计算单元中不同类型的计算单元的数量与所述类型的计算单元被调用次数正相关。
- 根据权利要求1所述的处理器,其特征在于,所述多个计算单元中的每个计算单元对应一个底层运算指令。
- 根据权利要求1或2所述的处理器,其特征在于,所述计算单元池包括张量计算单元池、矢量计算单元池和标量计算单元池中的一个或多个。
- 根据权利要求1-3中任一项所述的处理器,其特征在于,所述处理器用于:接收执行第一应用的请求;确定所述第一应用的计算资源需求,根据所述计算资源需求从所述计算单元池中为所述第一应用分配计算单元以实现所述第一应用,所述计算资源需求包括张量计算单元、矢量计算单元和矢量计算单元中的一种或多种,以及每种计算单元的数量。
- 根据权利要求4所述的处理器,其特征在于,所述处理器还用于:确定所述第一应用对应的多个进程;确定所述多个进程中每个进程的计算资源需求,根据所述每个进程的计算资源需求从所述计算单元池中为所述每个进程分配计算单元以实现所述第一应用;其中,为所述每个进程分配的计算单元与为其他进程分配的计算单元不同。
- 根据权利要求4或5所述的处理器,其特征在于,所述处理器还用于:确定为所述第一应用分配的计算单元是否满足所述第一应用的需要;当为所述第一应用分配的计算单元不能满足所述第一应用的需要时,调整所述第一应用的计算资源分配结果,并根据所述调整后的计算资源分配结果从所述计算单元池中为所述第一应用分配计算单元。
- 根据权利要求4或5所述的处理器,其特征在于,所述处理器还用于:建立所述第一应用与为所述第一应用分配的计算单元的对应关系。
- 根据权利要求1-7中任一项所述的处理器,其特征在于,所述类型与所述多个计算单元执行的算法对应。
- 一种资源分配方法,其特征在于,应用于处理器,所述处理器包括计算单元池,所述计算单元池包括多个计算单元;所述多个计算单元中的每个空闲计算单元可以被按需调用,且所述多个计算单元中不同类型的计算单元的数量与所述类型的计算单元被调用次数正相关;所述方法包括:接收执行第一应用的请求;确定所述第一应用的计算资源需求,根据所述计算资源需求从所述计算单元池中为所述第一应用分配计算单元以实现所述第一应用,所述计算资源需求包括张量计算单元、矢量计算单元和矢量计算单元中的一种或多种,以及每种计算单元的数量。
- 根据权利要求9所述的方法,其特征在于,所述确定所述第一应用的计算资源需求,根据所述计算资源需求从所述计算单元池中为所述第一应用分配计算单元以实现所述第 一应用,包括:确定所述第一应用对应的多个进程;确定所述多个进程中每个进程的计算资源需求,根据所述每个进程的计算资源需求从所述计算单元池中为所述每个进程分配计算单元以实现所述第一应用;其中,为所述每个进程分配的计算单元与为其他进程分配的计算单元不同。
- 根据权利要求9或10所述的方法,其特征在于,所述方法还包括:确定为所述第一应用分配的计算单元是否满足所述第一应用的需要;当为所述第一应用分配的计算单元不能满足所述第一应用的需要时,调整所述第一应用的计算资源分配结果,并根据所述调整后的计算资源分配结果从所述计算单元池中为所述第一应用分配计算单元。
- 根据权利要求9或10所述的方法,其特征在于,所述方法还包括:建立所述第一应用与为所述第一应用分配的计算单元的对应关系。
- 一种处理装置,其特征在于,包括:收发模块,用于接收执行第一应用的请求;处理模块,用于确定所述第一应用的计算资源需求,根据所述计算资源需求从处理器的计算单元池中为所述第一应用分配计算单元以实现所述第一应用,所述计算资源需求包括张量计算单元、矢量计算单元和矢量计算单元中的一种或多种,以及每种计算单元的数量;其中,所述计算单元池包括多个计算单元;所述多个计算单元中的每个空闲计算单元可以被按需调用,且所述多个计算单元中不同类型的计算单元的数量与所述类型的计算单元被调用次数正相关。
- 根据权利要求13所述的装置,其特征在于,所述处理模块,还用于:确定所述第一应用对应的多个进程;确定所述多个进程中每个进程的计算资源需求,根据所述每个进程的计算资源需求从所述计算单元池中为所述每个进程分配计算单元以实现所述第一应用;其中,为所述每个进程分配的计算单元与为其他进程分配的计算单元不同。
- 根据权利要求13或14所述的装置,其特征在于,所述处理模块,还用于:确定为所述第一应用分配的计算单元是否满足所述第一应用的需要;当为所述第一应用分配的计算单元不能满足所述第一应用的需要时,调整所述第一应用的计算资源分配结果,并根据所述调整后的计算资源分配结果从所述计算单元池中为所述第一应用分配计算单元。
- 根据权利要求13或14所述的装置,其特征在于,所述处理模块,还用于:建立所述第一应用与为所述第一应用分配的计算单元的对应关系。
- 一种计算平台,其特征在于,包括如权利要求1-3中任意一项所述的处理器以及如权利要求13-16中任意一项所述的装置。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2022/118522 WO2024055168A1 (zh) | 2022-09-13 | 2022-09-13 | 一种资源分配方法、处理器和计算平台 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2022/118522 WO2024055168A1 (zh) | 2022-09-13 | 2022-09-13 | 一种资源分配方法、处理器和计算平台 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2024055168A1 true WO2024055168A1 (zh) | 2024-03-21 |
Family
ID=90274061
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/118522 WO2024055168A1 (zh) | 2022-09-13 | 2022-09-13 | 一种资源分配方法、处理器和计算平台 |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2024055168A1 (zh) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111176852A (zh) * | 2020-01-15 | 2020-05-19 | 上海依图网络科技有限公司 | 资源分配方法、装置、芯片及计算机可读存储介质 |
US20200342292A1 (en) * | 2019-04-24 | 2020-10-29 | Baidu Usa Llc | Hardware-software co-design for accelerating deep learning inference |
CN113791906A (zh) * | 2021-08-09 | 2021-12-14 | 戴西(上海)软件有限公司 | 基于gpu资源在人工智能与工程领域调度***及优化算法 |
CN114661482A (zh) * | 2022-05-25 | 2022-06-24 | 成都索贝数码科技股份有限公司 | 一种gpu算力管理方法、介质、设备及*** |
-
2022
- 2022-09-13 WO PCT/CN2022/118522 patent/WO2024055168A1/zh unknown
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20200342292A1 (en) * | 2019-04-24 | 2020-10-29 | Baidu Usa Llc | Hardware-software co-design for accelerating deep learning inference |
CN111176852A (zh) * | 2020-01-15 | 2020-05-19 | 上海依图网络科技有限公司 | 资源分配方法、装置、芯片及计算机可读存储介质 |
CN113791906A (zh) * | 2021-08-09 | 2021-12-14 | 戴西(上海)软件有限公司 | 基于gpu资源在人工智能与工程领域调度***及优化算法 |
CN114661482A (zh) * | 2022-05-25 | 2022-06-24 | 成都索贝数码科技股份有限公司 | 一种gpu算力管理方法、介质、设备及*** |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110262901B (zh) | 一种数据处理方法及数据处理*** | |
US10108458B2 (en) | System and method for scheduling jobs in distributed datacenters | |
CN111488205B (zh) | 面向异构硬件架构的调度方法和调度*** | |
CN112181613B (zh) | 异构资源分布式计算平台批量任务调度方法及存储介质 | |
WO2022067531A1 (zh) | 一种计算资源感知的任务调度方法 | |
WO2021254135A1 (zh) | 任务执行方法及存储设备 | |
CN111142938B (zh) | 一种异构芯片的任务处理方法、任务处理装置及电子设备 | |
CN110471766B (zh) | 一种基于cuda的gpu资源调度***和方法 | |
WO2022247105A1 (zh) | 一种任务调度方法、装置、计算机设备和存储介质 | |
CN107656813A (zh) | 一种负载调度的方法、装置及终端 | |
WO2017185285A1 (zh) | 图形处理器任务的分配方法和装置 | |
CN111352727B (zh) | 一种应用于图像混合集群处理***的图像处理方法 | |
CN110300959B (zh) | 用于动态运行时任务管理的方法、***、设备、装置和介质 | |
CN115292016A (zh) | 基于人工智能的任务调度方法及相关设备 | |
CN105718318B (zh) | 一种基于辅助工程设计软件的集合式调度优化方法 | |
CN117058288A (zh) | 图形处理器及方法、多核图形处理***、电子装置及设备 | |
CN112433844B (zh) | 一种资源分配方法、***、设备及计算机可读存储介质 | |
CN116701001B (zh) | 目标任务分配方法、装置、电子设备及存储介质 | |
CN116431315B (zh) | 批处理任务的处理方法、装置、电子设备及存储介质 | |
CN103713953A (zh) | 一种内存数据的搬移装置及方法 | |
WO2024055168A1 (zh) | 一种资源分配方法、处理器和计算平台 | |
CN115373826B (zh) | 一种基于云计算的任务调度方法及装置 | |
CN111930485A (zh) | 一种基于性能表现的作业调度方法 | |
CN110837419A (zh) | 基于弹性批处理的推理引擎***、方法及电子设备 | |
CN112783651B (zh) | 一种云平台vGPU负载均衡调度方法、介质及装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22958362 Country of ref document: EP Kind code of ref document: A1 |