CN115421930A

CN115421930A - Task processing method, system, device, equipment and computer readable storage medium

Info

Publication number: CN115421930A
Application number: CN202211381753.2A
Authority: CN
Inventors: 张亚强; 李茹杨; 邓琪; 李雪雷; 赵雅倩; 李仁刚
Original assignee: Shandong Mass Institute Of Information Technology
Current assignee: Shandong Mass Institute Of Information Technology
Priority date: 2022-11-07
Filing date: 2022-11-07
Publication date: 2022-12-02
Anticipated expiration: 2042-11-07
Also published as: CN115421930B

Abstract

The application discloses a task processing method, a system, a device, equipment and a computer readable storage medium, which are applied to the technical field of resource scheduling, wherein the method is applied to computing nodes in an edge network, each computing node is provided with one or more heterogeneous resources, and the method comprises the steps of receiving a resource allocation strategy issued by the edge network; the resource allocation strategy comprises the processing capacity of each heterogeneous resource on the target task; inputting the resource allocation strategy and the local resource information into a preset strategy learning model for processing to obtain a resource sharing strategy; the resource sharing strategy comprises available resources of various heterogeneous resources, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target; and executing the target task by utilizing the resource sharing strategy. By applying the technical scheme provided by the application, more reasonable resource allocation can be carried out on the calculation task, and win-win of the task requester and the resource provider is realized.

Description

Task processing method, system, device, equipment and computer readable storage medium

Technical Field

The present application relates to the field of resource scheduling technologies, and in particular, to a method, a system, an apparatus, a device, and a computer-readable storage medium for task processing.

Background

Edge computing, as an emerging computing paradigm in recent years, provides various services such as computing power, storage, caching and the like for end users at the edge of a mobile network, thereby effectively reducing network delay. Because the types of the edge computing nodes are various, such as servers in operator base stations, personal computers in smart homes, and other devices, the computing resource nodes in the edge network have heterogeneous characteristics, and the performance difference of each node is large.

Heterogeneous computing is a computing system composed of computing units using different types of instruction sets and architectures, and computing resources in heterogeneous computing include multiple types of CPUs, GPUs, FPGAs, ASICs, and the like, each of which has obvious functional characteristics, has different characteristics in terms of computing power, power consumption, and the like, and can adapt to different types of computing tasks. In the edge calculation, different edge nodes have heterogeneous characteristics, and the computing resources inside the heterogeneous edge nodes also often have heterogeneous characteristics, and inside one node, multiple heterogeneous computing resources are included at the same time.

The existing research and method for distributing the resources of the edge network mainly take the resource utilization efficiency, the user experience quality and the like as main indexes for measuring the quality of a resource distribution strategy, and further determine the optimal scheme of the resource distribution. However, the main disadvantage is that the demand of computing node owners on the profit in the edge network is not considered, in an actual scenario, each edge node usually belongs to different owners, and the purpose of sharing the computing resource of the edge node is mainly to obtain the corresponding profit by using the idle computing resource, so as to realize win-win; meanwhile, the existing work has less attention to the problem of distributing edge node resources with various heterogeneous computing resources, and the research on the computing task based on the heterogeneous resource cooperative processing is not sufficient.

Therefore, how to allocate more reasonable resources for the computing task and achieve the win-win of the task requester and the resource provider is a problem to be solved urgently by those skilled in the art.

Disclosure of Invention

The task processing method can carry out more reasonable resource allocation for the calculation task, and realizes win-win of a task requester and a resource provider; another object of the present application is to provide another task processing method, a task processing apparatus and device, and a computer-readable storage medium, all of which have the above advantages.

In a first aspect, the present application provides a task processing method, applied to compute nodes in an edge network, where each compute node is provided with one or more heterogeneous resources, and the method includes:

receiving a resource allocation strategy issued by the edge network; the resource allocation strategy comprises the processing capacity of each heterogeneous resource allocated to the target task;

inputting the resource allocation strategy and the local resource information into a preset strategy learning model for processing to obtain a resource sharing strategy; the resource sharing strategy comprises available resources of each heterogeneous resource, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target;

and executing the target task by utilizing the resource sharing strategy.

Optionally, after the target task is executed by using the resource sharing policy, the method further includes:

acquiring an execution result of the target task;

and reporting the execution result to the edge network.

Optionally, the heterogeneous resources include CPU resources, GPU resources, and FPGA resources.

Optionally, the task processing method further includes:

monitoring the local resource information in real time;

and when the local resource information meets a preset condition, the local resource information is issued to the edge network.

Optionally, the issuing the local resource information to the edge network when the local resource information meets a preset condition includes:

determining available resources according to the local resource information;

and when the resource occupation ratio of the available resources reaches a preset threshold value, the local resource information is issued to the edge network.

Optionally, the local resource information includes available resources and resource provision time, and the available resources include current available resources of each of the heterogeneous resources.

In a second aspect, the present application provides another task processing method applied to an edge network, where each computing node in the edge network is provided with one or more heterogeneous resources, the method including:

determining a target computing node and a resource allocation strategy according to task information of a target task and local resource information of each computing node; the resource allocation strategy comprises the processing capacity of each heterogeneous resource allocated to the target task;

sending the resource allocation strategy to each target computing node, so that each target computing node processes the resource allocation strategy and the local resource information by using a preset strategy learning model to obtain a resource sharing strategy, and executing the target task by using the resource sharing strategy; the resource sharing strategy comprises available resources of each heterogeneous resource, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target.

Optionally, the determining a target computing node according to the task information of the target task and the local resource information of each computing node includes:

determining resource providing time and available resources according to the local resource information;

and when the resource providing time and the available resources both meet the task requirement indicated by the task information, determining the computing node as the target computing node.

Optionally, before determining the target computing node and the resource allocation policy according to the task information of the target task and the local resource information of each computing node, the method further includes:

counting the local resource information issued by each computing node; and the local resource information is issued to the edge network by the computing node when the local resource information meets a preset condition.

Optionally, the task processing method further includes:

acquiring an execution result of the target task uploaded by the target computing node;

and feeding back the execution result to the initiating end of the target task.

In a third aspect, the present application further discloses a task processing system, including an edge network and computing nodes deployed in the edge network, each of the computing nodes being provided with one or more heterogeneous resources, wherein,

the edge network is used for determining a target computing node and a resource allocation strategy according to task information of a target task and local resource information of each computing node, and sending the resource allocation strategy to each target computing node; the resource allocation strategy comprises the processing capacity of each heterogeneous resource allocated to the target task;

the target computing node is used for processing the resource allocation strategy and the local resource information by using a preset strategy learning model to obtain a resource sharing strategy and executing the target task by using the resource sharing strategy; the resource sharing strategy comprises available resources of each heterogeneous resource, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target.

In a fourth aspect, the present application further discloses a task processing apparatus applied to computing nodes in an edge network, where each computing node is provided with one or more heterogeneous resources, the apparatus including:

a receiving module, configured to receive a resource allocation policy issued by the edge network; the resource allocation strategy comprises the processing capacity of each heterogeneous resource allocated to the target task;

the processing module is used for inputting the resource allocation strategy and the local resource information into a preset strategy learning model for processing to obtain a resource sharing strategy; the resource sharing strategy comprises available resources of each heterogeneous resource, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target;

and the execution module is used for executing the target task by utilizing the resource sharing strategy.

In a fifth aspect, the present application further discloses another task processing apparatus applied to an edge network, where each compute node in the edge network is provided with one or more heterogeneous resources, the apparatus including:

the determining module is used for determining a target computing node and a resource allocation strategy according to task information of a target task and local resource information of each computing node; the resource allocation strategy comprises the processing capacity of each heterogeneous resource allocated to the target task;

the sending module is used for sending the resource allocation strategy to each target computing node so as to enable each target computing node to process the resource allocation strategy and the local resource information by using a preset strategy learning model to obtain a resource sharing strategy, and executing the target task by using the resource sharing strategy; the resource sharing strategy comprises available resources of each heterogeneous resource, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target.

In a sixth aspect, the present application further discloses a task processing device, including:

a memory for storing a computer program;

a processor for implementing the steps of any one of the task processing methods as described above when executing the computer program.

In a seventh aspect, the present application further discloses a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of any one of the task processing methods described above.

By applying the technical scheme provided by the application, the strategy learning model is deployed in each computing node of the edge network in advance, and the resource sharing strategy is determined based on the preset strategy learning model, so that the resource sharing strategy is utilized to realize task processing; in addition, a resource allocation strategy is established by the edge network, the resource allocation strategy is obtained by the computing node based on resource allocation strategy learning, and the resource allocation strategy and the resource sharing strategy take heterogeneous resources in the computing node into consideration, so that reasonable allocation of the internal heterogeneous resources of the computing node is further realized.

Drawings

In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings that are needed to be used in the description of the prior art and the embodiments of the present application will be briefly described below. Of course, the following description of the drawings related to the embodiments of the present application is only a part of the embodiments of the present application, and it will be obvious to those skilled in the art that other drawings can be obtained from the provided drawings without any creative effort, and the obtained other drawings also belong to the protection scope of the present application.

FIG. 1 is a schematic diagram of a task processing system provided in the present application;

FIG. 2 is a schematic flow chart illustrating a task processing method according to the present application;

FIG. 3 is a schematic flow chart diagram of another task processing method provided in the present application;

FIG. 4 is a schematic flow chart diagram illustrating another task processing method provided in the present application;

FIG. 5 is a schematic flowchart of a task processing device provided in the present application;

FIG. 6 is a schematic flow chart of another task processing device provided in the present application;

fig. 7 is a schematic structural diagram of a task processing device provided in the present application.

Detailed Description

The core of the application is to provide a task processing method, which can carry out more reasonable resource allocation for the calculation task and realize the win-win of a task requester and a resource provider; another core of the present application is to provide another task processing method, a task processing apparatus and device, and a computer-readable storage medium, all having the above beneficial effects.

In order to more clearly and completely describe the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the task processing method provided by the present application is applied to a task processing system, and the task processing system is an edge network system. Referring to fig. 1, fig. 1 is a schematic structural diagram of a task processing system provided in the present application, the task processing system includes an edge network 100 and computing nodes 200 deployed in the edge network, where the edge network 100 is configured to allocate resources for computing tasks, so as to allocate the resources to suitable computing nodes 100, so as to implement task processing based on the computing resources in the computing nodes 100.

It can be understood that edge computing is mainly oriented to massive end users in the world of everything interconnection, and the end users and the devices thereof generate highly concurrent computing task requests, and because the device capabilities of the end users are extremely limited, the users can reasonably unload computing tasks to edge terminals by using computing resources and network bandwidth resources in an edge computing network, and the computing nodes in the edge network allocate corresponding computing resources to process the requests of the users to complete task processing.

The embodiment of the application provides a task processing method.

Referring to fig. 2, fig. 2 is a flowchart illustrating a task processing method that can be applied to computing nodes in an edge network, where each computing node is provided with one or more heterogeneous resources, including the following steps S101 to S103.

First, it should be noted that the task processing method provided by the embodiment of the present application is applied to each computing node in the edge network, that is, the implementation flows of S101 to S103 are executed by the computing node in the edge network. In the edge network, each computing node is deployed in a geographic space, and the resource provider sources are various, in other words, the types of the computing nodes are not unique, and may be, for example, an infrastructure service provider such as a telecommunication operator, a cloud computing service provider, an individual user accessing to the internet, and the like.

Each computing node comprises one or more heterogeneous resources, the heterogeneous resources are such as the CPU, the GPU, the FPGA, the ASIC and the like, the heterogeneous resources have obvious functional characteristics, have different characteristics in the aspects of computing capacity, power consumption and the like, and can adapt to different types of computing tasks, for example, the CPU is good at processing tasks with sequential logic structures, the GPU has strong processing capacity on parallel tasks, and the FPGA has good processing performance on real-time computing tasks in certain characteristic scenes, so that the processing capacities of the heterogeneous resources are different when different computing tasks are faced. In one possible implementation, the heterogeneous resources in each compute node in the edge network may include CPU resources, GPU resources, FPGA resources.

S101: receiving a resource allocation strategy issued by an edge network; the resource allocation strategy comprises the processing capacity of each heterogeneous resource on the target task;

this step aims to achieve the acquisition of the resource allocation policy, that is, to receive the resource allocation policy issued by the edge network. The resource allocation strategy is a strategy for a target task, and can be constructed and obtained by combining a task requirement of the target task and local resource information of each computing node in the edge network through the edge network, so as to allocate reasonable computing resources for the target task, so as to facilitate the processing of the target task, where the target task is a computing task initiated by a user and needing to be processed.

The resource allocation policy is equivalent to an initial allocation policy, which includes the task amount of each heterogeneous resource allocated in the edge network with respect to the target task. As described above, each computing node includes one or more heterogeneous resources, where the processing amount of the target task allocated to each of the heterogeneous resources is the task amount of the subtask allocated to each of the heterogeneous resources by the target task, and the task amount may be a ratio of the subtask with respect to the target task or a number of the subtasks. Obviously, when the task quantity is the subtask proportion, the sum of the subtask proportions of all the heterogeneous resources is 1; and when the task quantity is the sub-task quantity, the sum of the self-task quantities of all the heterogeneous resources is the total task quantity of the target task.

It should be noted that the computing node that receives the resource allocation policy issued by the edge network is not unique, and may be a computing node selected by the edge network based on a certain screening policy, so that in the edge network, the computing node that receives the resource allocation policy may be one or multiple, may be a part of computing nodes in the edge network, and may also be all computing nodes in the edge network, and of course, when all computing nodes in the edge network are not selected, it indicates that there is no computing node that meets the processing requirement of the target task in the current edge network, and it may delay to wait for a period of time for further processing. On this basis, it is conceivable that the heterogeneous resources mentioned in the resource allocation policy refer to heterogeneous resources of the selected respective computing nodes.

S102: inputting the resource allocation strategy and the local resource information into a preset strategy learning model for processing to obtain a resource sharing strategy; the resource sharing strategy comprises available resources of various heterogeneous resources, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target;

the step aims to realize the calculation of the resource sharing strategy based on the preset strategy learning model. Firstly, a policy learning model adapted to each computing node is created in advance in each computing node, the input of the model is a resource allocation policy issued by an edge network and resource information of the computing node (i.e. the local resource information), the local resource information mainly refers to available resources expected to be provided by the computing node, and the output of the model is a resource sharing policy. The preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target, wherein the task processing value refers to the income obtained by the computing node during task processing, namely the income requirement of the computing node is taken into account, and the preset strategy learning model which can maximize the task processing value of the computing node is constructed.

Further, the preset policy learning model output resource sharing policy is a final allocation policy, where the resource sharing policy includes available resources of each heterogeneous resource inside the current computing node, where the available resources refer to resources provided for processing the current target task, and specifically may be a ratio of resources provided for processing the current target task in each heterogeneous resource to available resources actually provided by itself.

Of course, when a plurality of computing nodes receive the resource allocation policy, each computing node receiving the resource allocation policy outputs a resource sharing policy applicable to itself, and each resource sharing policy explicitly indicates that each heterogeneous resource inside its computing node is a computing resource provided by the target task, so that the processing of the target task can be implemented based on the computing resources actually provided by each heterogeneous resource in each selected computing node.

S103: and executing the target task by utilizing the resource sharing strategy.

This step is intended to achieve the execution of the target task, i.e. the target task is executed based on the resource sharing policy. Of course, executing the target task here is specifically to execute the target task by using available resources of the various heterogeneous resources indicated in the resource sharing policy. It can be understood that, because the target task may be executed by each heterogeneous resource in the multiple computing nodes together, "sharing" in the resource sharing policy means that each heterogeneous resource in each computing node executing the target task may perform information sharing during the execution of the target task, thereby ensuring that the target task may be executed completely and ensuring the accuracy of the execution result of the target task.

It can be seen that, in the task processing method provided in the embodiment of the present application, a policy learning model is pre-deployed in each computing node of an edge network, and a resource sharing policy is determined based on the preset policy learning model, so as to implement task processing by using the resource sharing policy, and since the preset policy learning model is obtained by training with the task processing value of the computing node as an optimization target, the task processing value of the computing node can be maximized by performing task processing based on the obtained resource sharing policy, and it is ensured that the task processing is completed, thereby implementing a win-win situation between a task requester and a resource provider; in addition, a resource allocation strategy is established by the edge network, the computing node learns the resource allocation strategy based on the resource allocation strategy, and the resource allocation strategy and the resource sharing strategy both take heterogeneous resources in the computing node into consideration, so that reasonable allocation of the internal heterogeneous resources of the computing node is further realized.

In an embodiment of the present application, after the target task is executed by using the resource sharing policy, the method may further include the following steps:

acquiring an execution result of a target task;

and reporting the execution result to the edge network.

The task processing method provided by the embodiment of the application can further realize the function of executing result feedback. After the target task is executed based on the resource sharing strategy, the execution result of the target task can be counted, so that the execution result is reported to the edge network, and the feedback of the execution result is realized. Further, the edge network may also feed back the execution result to the initiator of the target task.

In an embodiment of the present application, the task processing method may further include the steps of:

monitoring local resource information in real time;

and when the local resource information meets the preset condition, the local resource information is issued to the edge network.

As described above, the resource allocation policy may be constructed and obtained by combining the task requirement of the target task and the local resource information of each computing node in the edge network, and therefore, the edge network needs to acquire the local resource information of each computing node before constructing the resource allocation policy.

The local resource information is issued to the edge network by the corresponding computing node, and the issuing of the local resource information to the edge network by each computing node is executed under the condition that the local resource information meets the preset condition, and when the local resource information does not meet the preset condition of the local resource information, the issuing of the local resource information is not performed. Therefore, each computing node can monitor the local resource information of the computing node in real time and issue the local resource information to the edge network when the local resource information meets the preset conditions. The preset condition is a release condition preset by an owner of the computing node according to an actual requirement of the owner, and is not unique, and the preset condition is not limited in the application.

In an embodiment of the application, the issuing the local resource information to the edge network when the local resource information meets the preset condition includes the following steps:

determining available resources according to the local resource information;

The embodiment of the present application provides a specific type of preset condition, that is, a resource proportion of available resources in a computing node reaches a preset threshold, where the available resources refer to idle resources in the computing node, and the resource proportion refers to a proportion of the idle resources to total resources.

In the process of carrying out real-time statistics on the local resource information, available resources can be further determined according to the local resource information, the resource proportion of the available resources is counted, and when the resource proportion reaches a preset threshold value, the fact that the available resources can provide computing resources for computing tasks of other terminals is indicated, so that the local resource information can be issued; on the contrary, when the resource occupation ratio does not reach the preset threshold, the method indicates that the method cannot provide computing resources for computing tasks of other terminals, and therefore, the local resource information is not issued. Certainly, the value of the preset threshold does not affect the implementation of the technical scheme, and the owner of the computing node sets the value according to the actual requirement, which is not limited in the present application.

In one embodiment of the present application, the local resource information may include available resources including currently available resources that may be heterogeneous resources and resource provisioning times.

The embodiment of the application provides specific types of local resource information. Firstly, in order to realize task processing, available resources of a computing node need to meet the actual requirements of a target task; secondly, the sharing willingness of the owner of the computing node to the resources sometimes fluctuates along with the change of time, for example, when a computing node is idle, the owner of the computing node is more inclined to share the idle resources, service other users and obtain certain rewards or benefits, so that the resource providing time of the computing node also needs to meet the actual requirements of target tasks. Based on this, the local resource information may specifically include its available resource and resource providing time, where the available resource includes the current available resource of each type of heterogeneous resource inside itself.

The embodiment of the application provides another task processing method.

Referring to fig. 3, fig. 3 is a flowchart illustrating another task processing method provided in the present application, where the task processing method is applicable to an edge network, and each computing node in the edge network is provided with one or more heterogeneous resources, including the following S201 and S202.

First, it should be noted that the task processing method provided in the embodiment of the present application is applied to an edge network, that is, the implementation flows of S201 and S202 are executed by the edge network.

S201: determining a target computing node and a resource allocation strategy according to task information of the target task and local resource information of each computing node; the resource allocation strategy comprises the processing capacity of each heterogeneous resource on the target task;

this step is intended to enable determination of the target compute node and resource allocation policy. The target computing node is a computing node selected from the edge network and used for processing a target task, the resource allocation strategy is a strategy for the target task and used for allocating reasonable computing resources for the target task so as to facilitate the processing of the target task, and the target task is a computing task which is initiated by a user side and needs to be processed.

In the implementation process, when the edge network receives a target task to be processed, task information of the target task, including but not limited to information such as task type, task target, computing resource demand and the like, may be counted first; then, local resource information of each computing node in the edge network is counted, wherein the local resource information can include but is not limited to available resources expected to be provided by the computing node, time limit for providing the available resources and the like; and finally, screening and determining the target computing node from the plurality of computing nodes by combining the task information of the target task and the local resource information of each computing node, and constructing and obtaining a resource allocation strategy.

The resource allocation policy is equivalent to an initial allocation policy, which includes the task amount of each heterogeneous resource allocated in the edge network with respect to the target task. It can be understood that each computing node includes one or more heterogeneous resources, where a processing amount of each heterogeneous resource allocated to a target task is a task amount of a sub-task allocated to each heterogeneous resource by the target task, and the task amount may be a ratio of the sub-task to the target task or a number of the sub-tasks. Obviously, when the task quantity is the subtask proportion, the sum of the subtask proportions of all the heterogeneous resources is 1; and when the task quantity is the sub-task quantity, the sum of the self task quantities of all the heterogeneous resources is the total task quantity of the target task.

It should be noted that the number of target computing nodes is not unique, and may be one, or may be multiple, and may be part of computing nodes in the edge network, or may be all computing nodes in the edge network. On this basis, it is conceivable that the heterogeneous resources mentioned in the resource allocation policy refer to heterogeneous resources of each target computing node determined by screening.

S202: sending the resource allocation strategy to each target computing node so that each target computing node processes the resource allocation strategy and local resource information by using a preset strategy learning model to obtain a resource sharing strategy and executes a target task by using the resource sharing strategy; the resource sharing strategy comprises available resources of various heterogeneous resources, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target.

The step aims to realize the issue of the resource allocation strategy, namely, the resource allocation strategy is issued to each target computing node, and each target computing node realizes the execution of the target task based on the resource allocation strategy. The implementation process of each target computing node executing the target task is as follows:

firstly, a policy learning model adapted to each computing node of the edge network is created in advance, for a target computing node, the input of the model is a resource allocation policy issued by the edge network and resource information (i.e., the local resource information) of the target computing node, the local resource information mainly refers to available resources expected to be provided by the target computing node, and the output of the model is a resource sharing policy. The preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target, wherein the task processing value refers to the income obtained by the computing node during task processing, namely the income requirement of the computing node is taken into account, and the preset strategy learning model which can maximize the task processing value of the computing node is constructed.

Further, the preset policy learning model output resource sharing policy is a final allocation policy, where the resource sharing policy includes available resources of each heterogeneous resource inside the current target computing node, where the available resources refer to resources provided for processing the current target task, and specifically may be a ratio of resources provided for processing the current target task in each heterogeneous resource to available resources actually provided by itself.

Of course, when there are a plurality of target computing nodes, each target computing node outputs a resource sharing policy applicable to itself, and each resource sharing policy explicitly indicates the computing resources provided by the heterogeneous resources in its computing node for the target task, so that the processing of the target task can be implemented based on the computing resources actually provided by the heterogeneous resources in each target computing node.

Finally, the target task can be executed based on the resource sharing strategy. Of course, executing the target task here is specifically to execute the target task by using available resources of the various heterogeneous resources indicated in the resource sharing policy. It can be understood that, because the target task may be executed by each heterogeneous resource in the multiple target computing nodes together, "sharing" in the resource sharing policy means that each heterogeneous resource in each target computing node executing the target task may perform information sharing during the execution of the target task, thereby ensuring that the target task may be executed completely and ensuring the accuracy of the execution result of the target task.

It can be seen that, in the task processing method provided in the embodiment of the present application, a policy learning model is pre-deployed in each computing node of an edge network, and a resource sharing policy is determined based on the preset policy learning model, so as to implement task processing by using the resource sharing policy, and since the preset policy learning model is obtained by training with the task processing value of the computing node as an optimization target, the task processing value of the computing node can be maximized by performing the task processing based on the obtained resource sharing policy, and the task processing is ensured to be completed, thereby implementing a win-win situation between a task requester and a resource provider; in addition, a resource allocation strategy is established by the edge network, the computing node learns the resource allocation strategy based on the resource allocation strategy, and the resource allocation strategy and the resource sharing strategy both take heterogeneous resources in the computing node into consideration, so that reasonable allocation of the internal heterogeneous resources of the computing node is further realized.

In an embodiment of the application, the determining the target computing node according to the task information of the target task and the local resource information of each computing node may include the following steps:

and when the resource providing time and the available resources both meet the task requirement indicated by the task information, determining the computing node as a target computing node.

The embodiment of the application provides an implementation method for screening and determining target computing nodes. Firstly, in order to realize task processing, available resources of a target computing node need to meet the actual requirements of a target task; secondly, the sharing willingness of the owner of the computing node to the resources sometimes fluctuates along with the change of time, for example, when a computing node is idle, the owner of the computing node is more inclined to share the idle resources, service other users and obtain certain rewards or benefits, and therefore, the resource providing time of the target computing node also needs to meet the actual requirements of the target task.

On this basis, in the implementation process, after the local resource information of each computing node is obtained, the resource providing time and the available resources of the corresponding computing node are determined according to the local resource information, then whether the local resource providing time and the available resources meet the task requirement indicated in the task information of the target task or not is judged, and the current computing node is determined as the target computing node when the local resource providing time and the available resources meet the task requirement. And when any kind of information does not meet the task requirement, the information cannot be confirmed as the target computing node.

Of course, the selection criterion is only one implementation manner provided in the embodiment of the present application, and is not unique, and may be specifically set according to actual requirements of a resource provider and a task requester, which is not limited in the present application.

In an embodiment of the present application, before determining the target computing node and the resource allocation policy according to the task information of the target task and the local resource information of each computing node, the method may further include the following steps:

counting local resource information issued by each computing node; and the local resource information is issued to the edge network by the computing node when the local resource information meets the preset condition.

It can be understood that the resource allocation policy is constructed and obtained by the edge network based on the task information of the target task and the local resource information of each computing node, and therefore, the edge network needs to acquire the local resource information of each computing node before constructing the resource allocation policy. The local resource information is issued to the edge network by the corresponding computing node, so that the edge network can directly capture the local resource information issued by each computing node when receiving the target task.

In addition, each computing node issues the local resource information to the edge network, which is executed under the condition that the local resource information meets the preset condition, and when the local resource information does not meet the preset condition of the computing node, the local resource information is not issued. The preset condition is a distribution condition preset by the owner of the computing node according to the actual demand of the owner, and is not unique, for example, the idle resource occupation ratio of the owner of the computing node reaches a preset threshold value, and the current time meets the preset resource sharing time.

acquiring an execution result of a target task uploaded by a target computing node;

and feeding back the execution result to the initiating end of the target task.

The task processing method provided by the embodiment of the application can further realize an execution result feedback function, wherein the execution result is the execution result of the target task. For each target computing node, after the target computing node executes the target task, the execution result can be further reported to the edge network; for the edge network, after receiving the execution result uploaded by the target computing node, the edge network may feed back the final execution result to the initiator of the target task.

The embodiment of the application provides another task processing method.

Referring to fig. 4, fig. 4 is a schematic flowchart of another task processing method provided in the present application, where the task processing method includes the following steps:

1. the distributed edge network heterogeneous computing resource actively discovers the strategy. Each distributed edge computing node actively and timely issues a resource state label to an edge network according to the self running state and the available resource condition;

2. and combining and allocating the strategies based on the user task resources of the heterogeneous computing. According to the computing task request information of the current user and the computing resource state label obtained in the current edge network, counting the schedulable conditions of various heterogeneous computing resources to obtain a heterogeneous computing resource combination allocation scheme related to the computing task.

3. And learning the resource sharing strategy of the heterogeneous computing nodes. And outputting the own computing resource sharing strategy by each heterogeneous edge node according to the heterogeneous computing resource combination allocation scheme given in the last step by using a multi-agent strategy learning technology.

The specific implementation flow of the steps is as follows:

1. the distributed edge network heterogeneous computing resource actively discovers the strategy.

In an edge computing network, each edge computing node is deployed in a geographic space in a distributed manner, resource providers of the edge computing nodes are from a variety of sources, the edge computing nodes are generally heterogeneous, and the capabilities and types of computing resources that can be provided by different edge computing nodes may differ. In addition, the sharing willingness of the owner to the computing resource fluctuates with time. Therefore, heterogeneous computing resources in the edge network can be actively shared and discovered according to a certain strategy.

(1) Edge node computing resource label setting:

in the form of a label, an edge computing node owner describes the heterogeneous computing resources that it owns, and the label can be represented as the following multiple group:

；

wherein the content of the first and second substances,

representing edge compute nodes

The tags used to conduct active resource discovery,

a threshold representing that the edge computing node may publish a label, which may be a percentage of the total amount of computing resources currently available to the edge computing node that are actually owned,

indicating a period of time set by the owner during which resource sharing is possible,

，

，

respectively showing the conditions of CPU, GPU and FPGA computing resources which can be currently utilized by the edge computing node.

(2) Resource active discovery strategy:

on the edge computing node, a normal operation resource publishing module is used for monitoring the utilization condition and the current time of various self heterogeneous resources in real time, and when the publishing condition is met, the module actively publishes a label to an edge network

To indicate that the computing resources on the edge compute node may be at time at this time

The range is shared, and the sharable resource type and size are respectively

，

，

。

2. And combining and allocating the strategies based on the user task resources of the heterogeneous computing.

When a computing task (target task) is generated in the edge network, the task scheduling module performs resource combination allocation aiming at the computing task of the user based on available heterogeneous computing resources so as to improve the utilization rate of the edge network resources, meet various constraint requirements of the computing task of the user and improve the experience quality of the user.

(1) And collecting heterogeneous computing resource information. The task scheduling module firstly counts the scheduling conditions of various heterogeneous computing resources according to the labels actively issued by the current computing nodes, and screens the computing nodes which can be utilized in the time period of completing the computing tasks of the user; under the condition, whether the heterogeneous computing resources which can be scheduled can meet the computing task of the user is counted, if yes, the next step is carried out, and otherwise, the computing task cannot be met.

(2) And generating a heterogeneous resource combination allocation strategy. According to the consumption required by various computing resource processing unit computing tasks and the size of the computing task of a user, a plurality of distribution strategies based on heterogeneous computing resource combination can be obtained. The resource consumption required by CPU, GPU and FPGA for respectively processing unit calculation task amount is assumed to be real number

，

，

Something originated from

The total task amount of the calculation tasks at the moment is real number

And the calculation task can be divided into a plurality of real number parts, wherein the task quantities respectively corresponding to the CPU, the GPU and the FPGA are real numbers

，

，

Under which condition the task is calculated

The total consumption was:

；

the corresponding limitations are:

；

；

wherein the content of the first and second substances,

representing the true value for the computing task constraint term,

，

，

respectively represent the

The computing task at the moment corresponds to the constraint factors of heterogeneous computing resources such as a CPU (Central processing Unit), a GPU (graphic processing Unit), an FPGA (field programmable Gate array), and the like, and the combination allocation strategy of the heterogeneous computing resources should meet the task constraint items

And calculating the total amount of tasks

Under the condition of (1), the total consumption is reduced as much as possible

The average overhead of the edge network is optimized, and the user experience quality is improved. The process of solving the optimal solution of the objective function can be expressed as a linear programming problem, and the optimal heterogeneous computing resource combination strategy can be obtained by solving the objective function by using a linear programming solver.

3. And learning the resource sharing strategy of the heterogeneous computing nodes.

And (3) learning to obtain a self resource allocation strategy by using a multi-agent strategy learning technology and each edge computing node according to the optimal resource combination strategy obtained in the last step. In the current edge computing node set, each edge computing node shares part of computing resources for processing the current computing task and obtaining corresponding benefits.

Computing nodes for edges in a collection

Setting the resource sharing policy of the current computing task as follows:

；

wherein the content of the first and second substances,

，

，

the proportion of various heterogeneous computing resources (CPU, GPU and FPGA) which are respectively predicted to be allocated to the current computing task to the current available resources of the heterogeneous computing resources is real number between 0 and 1, and the real number satisfies the following conditions:

；

；

；

namely, the computing resources shared by the edge computing nodes can meet the optimal heterogeneous resource allocation strategy of the computing task. The strategy learning goal of each edge computing node is to generate an optimal resource sharing strategy

To make it possible to maximize the currently obtained revenue

：

；

；

；

；

Wherein the content of the first and second substances,

，

，

respectively, representing the rewards that their heterogeneous resources are shared for the current computing task. Rewards indicate that the resource owner will ownThere are various types of heterogeneous computing resources that provide benefits that can be obtained when a task is processed, and each resource owner seeks to maximize the benefits that it obtains. In the above formula

，

，

The values of (1) respectively represent reward values which can be obtained when a unit heterogeneous computing resource processes a task, namely, the basic income when a certain type of resource is shared is measured and is positive and real. Generally, heterogeneous resources exhibit different processing capabilities for different types of computing tasks, and therefore, it is necessary to further improve the efficiency of use of resources and increase the profit level of resource owners by setting the unit profit of heterogeneous resources in the face of computing tasks.

And learning the resource sharing strategy of each edge computing node by using a multi-agent strategy learning technology, and adopting a mechanism of centralized training and distributed execution. The policy learning network (preset policy learning model) for setting edge computing nodes is

The network parameter is

Input into a heterogeneous computing resource combination allocation policy for a current computing task

And the currently available heterogeneous computing resources of the edge computing node

The output of the policy learning network is a resource sharing decision

。

In addition, each edge computing node is provided with a separate value evaluation network

With the parameters of

For obtaining the optimal sharing decision output, the pair is needed

The training process of each edge computing node is as follows:

(1) Computing edges into nodes

Acquired status information

As input to a local decision network, obtaining a current output

Performing an action

The system then transitions to a new state

And corresponding benefits

Will be

Is stored in a collection

；

(2) When the collection

Is more than

From the set

Middle sampling

Next, calculating a target value:

；

wherein the content of the first and second substances,

representing the next context information observed by all edge compute nodes,

a real number between 0 and 1, representing a discount factor;

(3) Updating value evaluation network of each edge computing node according to minimum loss function

Parameter (d) of

The minimization loss function is:

；

wherein the content of the first and second substances,

representing current environment information observed by all edge computing nodes;

（4) Policy learning network

The update gradient of (a) is:

；

(5) After the training is converged or reaches the designated training times, the training is completed and

and deploying the edge computing nodes, and generating a resource sharing strategy of the current edge computing node.

The embodiment of the application provides a task processing system.

As shown in fig. 1, the task processing system may include an edge network 100 and computing nodes 200 deployed in the edge network 100, each computing node 200 being provided with one or more heterogeneous resources, wherein,

the edge network 100 is configured to determine a target computing node 200 and a resource allocation policy according to task information of a target task and local resource information of each computing node 200, and send the resource allocation policy to each target computing node 200; the resource allocation strategy comprises the processing capacity of each heterogeneous resource on the target task;

the target computing node 200 is configured to process the resource allocation policy and the local resource information by using a preset policy learning model, obtain a resource sharing policy, and execute a target task by using the resource sharing policy; the resource sharing strategy comprises the available resources of various heterogeneous resources, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target.

It can be seen that, in the task processing system provided in the embodiment of the present application, a policy learning model is pre-deployed in each computing node of the edge network, and the determination of the resource sharing policy is implemented based on the preset policy learning model, so as to implement task processing by using the resource sharing policy, and since the preset policy learning model is obtained by training with the task processing value of the computing node as an optimization target, the task processing value of the computing node can be maximized by performing the task processing based on the obtained resource sharing policy, and the task processing is ensured to be completed, thereby implementing a win-win situation between the task requester and the resource provider; in addition, a resource allocation strategy is established by the edge network, the computing node learns the resource allocation strategy based on the resource allocation strategy, and the resource allocation strategy and the resource sharing strategy both take heterogeneous resources in the computing node into consideration, so that reasonable allocation of the internal heterogeneous resources of the computing node is further realized.

For the introduction of the system provided in the embodiment of the present application, please refer to the method embodiment described above, which is not described herein again.

The embodiment of the application provides a task processing device.

Referring to fig. 5, fig. 5 is a schematic structural diagram of a task processing device that can be applied to compute nodes in an edge network, where each compute node is provided with one or more heterogeneous resources, and the task processing device includes:

the receiving module 1 is used for receiving a resource allocation strategy issued by an edge network; the resource allocation strategy comprises the processing capacity of each heterogeneous resource on the target task;

the processing module 2 is used for inputting the resource allocation strategy and the local resource information into a preset strategy learning model for processing to obtain a resource sharing strategy; the resource sharing strategy comprises available resources of various heterogeneous resources, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target;

and the execution module 3 is used for executing the target task by utilizing the resource sharing strategy.

It can be seen that, in the task processing device provided in the embodiment of the present application, a policy learning model is pre-deployed in each computing node of the edge network, and the determination of the resource sharing policy is implemented based on the preset policy learning model, so as to implement task processing by using the resource sharing policy, and since the preset policy learning model is obtained by training with the task processing value of the computing node as an optimization target, the task processing value of the computing node can be maximized by performing the task processing based on the obtained resource sharing policy, and the task processing is ensured to be completed, thereby implementing a win-win situation between the task requester and the resource provider; in addition, a resource allocation strategy is established by the edge network, the computing node learns the resource allocation strategy based on the resource allocation strategy, and the resource allocation strategy and the resource sharing strategy both take heterogeneous resources in the computing node into consideration, so that reasonable allocation of the internal heterogeneous resources of the computing node is further realized.

In an embodiment of the present application, the task processing device may further include a reporting module, configured to obtain an execution result of the target task after the target task is executed by using the resource sharing policy; and reporting the execution result to the edge network.

In an embodiment of the present application, the heterogeneous resources may include CPU resources, GPU resources, and FPGA resources.

In an embodiment of the present application, the task processing apparatus may further include a publishing module, configured to perform real-time monitoring on local resource information; and when the local resource information meets the preset condition, the local resource information is issued to the edge network.

In an embodiment of the present application, the publishing module may be specifically configured to determine an available resource according to the local resource information; and when the resource proportion of the available resources reaches a preset threshold value, the local resource information is issued to the edge network.

In an embodiment of the present application, the local resource information may include available resources and resource providing time, and the available resources may include currently available resources of each heterogeneous resource.

For the introduction of the apparatus provided in the embodiment of the present application, please refer to the method embodiment described above, which is not described herein again.

The embodiment of the application provides another task processing device.

Referring to fig. 6, fig. 6 is a schematic structural diagram of another task processing device provided in the present application, where the task processing device is applicable to an edge network, and each compute node in the edge network is provided with one or more heterogeneous resources, including:

the determining module 4 is used for determining a target computing node and a resource allocation strategy according to the task information of the target task and the local resource information of each computing node; the resource allocation strategy comprises the processing capacity of each heterogeneous resource on the target task;

the sending module 5 is configured to send the resource allocation policy to each target computing node, so that each target computing node processes the resource allocation policy and the local resource information by using a preset policy learning model to obtain a resource sharing policy, and executes a target task by using the resource sharing policy; the resource sharing strategy comprises available resources of various heterogeneous resources, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target.

In an embodiment of the present application, the determining module 4 may be specifically configured to determine resource providing time and available resources according to the local resource information; and when the resource providing time and the available resources both meet the task requirement indicated by the task information, determining the computing node as a target computing node.

In an embodiment of the present application, the task processing apparatus may further include a statistics module, configured to, before determining the target computing node and the resource allocation policy according to the task information of the target task and the local resource information of each computing node, perform statistics on local resource information issued by each computing node; and the local resource information is issued to the edge network by the computing node when the local resource information meets the preset condition.

In an embodiment of the present application, the task processing apparatus may further include a feedback module, configured to obtain an execution result of the target task uploaded by the target computing node; and feeding back the execution result to the initiating end of the target task.

The embodiment of the application provides a task processing device.

Referring to fig. 7, fig. 7 is a schematic structural diagram of a task processing device provided in the present application, where the task processing device may include:

a memory for storing a computer program;

a processor, configured to implement the steps of any one of the task processing methods described above when executing the computer program.

As shown in fig. 7, in order to illustrate the structure of the task processing device, the task processing device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through a communication bus 13.

In the embodiment of the present application, the processor 10 may be a Central Processing Unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, etc.

The processor 10 may call a program stored in the memory 11, and in particular, the processor 10 may perform operations in an embodiment of the task processing method.

The memory 11 is used for storing one or more programs, the program may include program codes, the program codes include computer operation instructions, in this embodiment, the memory 11 stores at least the program for implementing the following functions:

receiving a resource allocation strategy issued by an edge network; the resource allocation strategy comprises the processing capacity of each heterogeneous resource on the target task;

inputting the resource allocation strategy and the local resource information into a preset strategy learning model for processing to obtain a resource sharing strategy; the resource sharing strategy comprises available resources of various heterogeneous resources, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target;

executing the target task by utilizing a resource sharing strategy;

alternatively, the first and second electrodes may be,

determining a target computing node and a resource allocation strategy according to task information of the target task and local resource information of each computing node; the resource allocation strategy comprises the processing capacity of each heterogeneous resource on the target task;

sending the resource allocation strategy to each target computing node so that each target computing node processes the resource allocation strategy and local resource information by using a preset strategy learning model to obtain a resource sharing strategy, and executing a target task by using the resource sharing strategy; the resource sharing strategy comprises available resources of various heterogeneous resources, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target.

In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created during use.

Further, the memory 11 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid state storage device.

The communication interface 12 may be an interface of a communication module for connecting with other devices or systems.

Of course, it should be noted that the structure shown in fig. 7 does not constitute a limitation to the task processing device in the embodiment of the present application, and in practical applications, the task processing device may include more or less components than those shown in fig. 7, or some components may be combined.

The embodiment of the application provides a computer readable storage medium.

The computer-readable storage medium provided in the embodiments of the present application stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above task processing methods may be implemented.

The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

For introduction of the computer-readable storage medium provided in the embodiment of the present application, please refer to the above method embodiment, which is not described herein again.

The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.

Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the components and steps of the various examples have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

The technical solutions provided by the present application are described in detail above. The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall into the protection scope of the present application.

Claims

1. A task processing method is applied to computing nodes in an edge network, each computing node is provided with one or more heterogeneous resources, and the method comprises the following steps:

and executing the target task by utilizing the resource sharing strategy.

2. The method of claim 1, wherein after the performing the target task using the resource sharing policy, further comprising:

acquiring an execution result of the target task;

and reporting the execution result to the edge network.

3. The method of claim 1, wherein the heterogeneous resources comprise CPU resources, GPU resources, FPGA resources.

4. The method of any of claims 1 to 3, further comprising:

monitoring the local resource information in real time;

5. The method according to claim 4, wherein the publishing the local resource information to the edge network when the local resource information satisfies a preset condition includes:

determining available resources according to the local resource information;

and when the resource proportion of the available resources reaches a preset threshold value, the local resource information is issued to the edge network.

6. The method of claim 4, wherein the local resource information comprises available resources and resource provisioning times, and wherein the available resources comprise currently available resources of each of the heterogeneous resources.

7. A task processing method applied to an edge network, wherein each computing node in the edge network is provided with one or more heterogeneous resources, the method comprising:

sending the resource allocation strategy to each target computing node, so that each target computing node processes the resource allocation strategy and the local resource information by using a preset strategy learning model to obtain a resource sharing strategy, and executing the target task by using the resource sharing strategy; the resource sharing strategy comprises the available resources of each heterogeneous resource, and the preset strategy learning model is obtained by training with the task processing value of the computing node as an optimization target.

8. The method of claim 7, wherein determining a target compute node based on task information for a target task and local resource information for each of the compute nodes comprises:

9. The method of claim 7, wherein before determining the target compute node and the resource allocation policy based on the task information of the target task and the local resource information of each of the compute nodes, further comprising:

10. The method of claim 7, further comprising:

and feeding back the execution result to the initiating end of the target task.

11. A task processing system comprising an edge network and computing nodes deployed in the edge network, each of the computing nodes being provided with one or more heterogeneous resources, wherein,

12. A task processing apparatus applied to computing nodes in an edge network, each computing node being provided with one or more heterogeneous resources, the apparatus comprising:

13. A task processing apparatus applied to an edge network, each computing node in the edge network being provided with one or more heterogeneous resources, the apparatus comprising:

14. A task processing device characterized by comprising:

a memory for storing a computer program;

a processor for implementing the steps of the task processing method according to any one of claims 1 to 10 when executing the computer program.

15. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, realizes the steps of the task processing method according to any one of claims 1 to 10.