WO2016144220A1 - Method and device for estimating computational network resources required to perform a computational task - Google Patents
Method and device for estimating computational network resources required to perform a computational task Download PDFInfo
- Publication number
- WO2016144220A1 WO2016144220A1 PCT/SE2015/050263 SE2015050263W WO2016144220A1 WO 2016144220 A1 WO2016144220 A1 WO 2016144220A1 SE 2015050263 W SE2015050263 W SE 2015050263W WO 2016144220 A1 WO2016144220 A1 WO 2016144220A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- computational
- network resources
- computational task
- perform
- task
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5019—Workload prediction
Definitions
- the invention relates to a method of estimating computational network resources required to perform a computational task and a device configured to perform the method.
- the invention further relates to a corresponding computer program for causing the device to perform the method according to the invention, and a corresponding computer program product.
- an analytics service when deployed on a virtual machine, e.g., using Platform as a Service (PaaS) or alternatively Infrastructure as a Service (IaaS), a user requesting the analytics service needs to specify how much resources need to be allocated, e.g., random access memory (RAM) size, microprocessor (CPU) capacity, hard disk size, etc. This information can also be used by the service provider to schedule multiple tasks and optimize the total resource utilization.
- PaaS Platform as a Service
- IaaS Infrastructure as a Service
- the analytics services share resources which are dynamically allocated to each service, and users are paying for the resources allocated to them. For users with little experience, or if complex analytics tasks are to be undertaken, it is difficult to estimate the required resources. Typically, the user would like to know the cost to complete the task. If the resource allocation is scarce, it may take too long time for the analytics task to perform. If the resource allocation is over-dimensioned, the user will block an unnecessarily great amount of resources. Another aspect is that if the system knows how much resources are needed for each task, the system can coordinate all tasks to optimize the total system resource utilization.
- An object of the present invention is to solve, or at least mitigate, this problem in the art and to provide an improved estimation of required computational resources provided in a network. This object is attained in a first aspect of the present invention by a method of estimating computational network resources required to perform a
- the method comprises receiving a request for estimating computational network resources, the request comprising information pertaining to a size of a data set to be processed when performing the requested computational task and a description of the computational task, and estimating, based on the information comprised in the request, the computational network resources required to perform the requested computational task.
- the method further comprises providing a response to the received request, the response specifying the estimated computational network resources required to perform the requested computational task.
- a resource estimating device configured to estimate computational network resources required to perform a computational task
- the device comprising a processing unit and a memory, which memory contains instructions executable by the processing unit, whereby the resource estimating device is operative to receive a request for estimating computational network resources, the request comprising information pertaining to a size of a data set to be processed when performing the requested computational task and a description of the computational task, and to estimate, based on the information comprised in the request, the computational network resources required to perform the requested computational task. Further, the resource estimating device is operative to provide a response to the received request, the response specifying the estimated computational network resources required to perform the requested computational task.
- a user wishing to have a computational task performed i.e., to utilize a computational service provided in a network, for instance in a cloud environment, will submit a request to a resource estimating device as provided in an embodiment of the present invention for estimating
- the request comprises information pertaining to a size of a data set to be processed when performing the requested computational task and a description of the computational task to be performed.
- the request may, e.g., specify a task such as "encryption of a set of data files of a certain size" or specify a clustering and classification task performed such as
- the user does not need to specify the resources to be used in terms of for instance computational or storage capacity. It should be noted that the user need not specify the computational task to be performed in any further detail, but rather inputs the aim of the task to be performed, such as, e.g., grouping, predicting, classifying, etc. Thereafter, based on the information pertaining to the size of the data set to be processed and the description of the computational task to be performed, the resource estimating device estimates the computational network resources required to perform the requested computational task.
- the resource estimating device advantageously provides a response to the received request specifying the estimated computational network resources required to perform the requested computational task.
- embodiment of the invention is exemplified as being performed by a resource estimating device, but could be performed by a number of different devices or, e.g., virtual machines, where the steps performed to be able to give an estimate are distributed over the different devices.
- the various devices are located in different parts of the world.
- the response may specify RAM size, CPU capacity, hard disk size, etc., for performing the requested computational task.
- the response provided is provided to the user.
- the estimated computational network resources advantageously specifies the estimated computational network resources in terms of a cost for performing the requested computational task, which for many users often is a preferred measure on the estimated resources, while in a further embodiment the response provided advantageously specifies the estimated computational network resources in terms of a time when the requested computational task is expected to have been completed. Further advantageous is that with the estimated cost and/ or time, the user is enabled to optimize planning of the computational tasks to be performed. For the user, the estimate may function as a benchmark measure when deciding priority of tasks, as well as a means to compare estimates from competing providers.
- not only information pertaining to the size of the data set to be processed and the computational task to be performed is used for performing the estimation, but further any one or more of number of attributes, types of the attributes, value ranges of each attributes, etc., for the data set.
- information pertaining to the size of the data set to be processed and the computational task to be performed is used for performing the estimation, but further any one or more of number of attributes, types of the attributes, value ranges of each attributes, etc., for the data set.
- a machine learning process will advantageously be implemented at the resource estimating device for performing estimation of the computational network resources required for performing the requested computational task.
- the resource estimating device will, in order to improve the estimation, have the information undergo a machine learning process, where the result of the estimation is fed back to the resource estimating device to make use of the result in order to
- the resource estimating device communicates with, e.g., a PaaS manager which supervises the network, for obtaining appropriate historical and/or current data to perform the machine learning process based on the obtained data to accomplish training of a model used for the estimation.
- the particular machine learning process to be used is selected based on the size of the data set to be processed and/ or the requested computational task. For instance, in case the size of the data set exceeds a threshold value, random forests maybe considered to be the best process to use, while in another example in case the computational task pertains to categorization, decision tree learning is considered the most efficient machine learning process.
- Figure 1 shows a schematic overview of a communications network in which a resource estimating device according to an embodiment of the invention may be deployed;
- Figure 2 shows a signalling diagram illustrating an embodiment of a method performed by the resource estimating device for estimating computational network resources required to perform a requested computational task according to the invention;
- Figure 3 illustrates functional entities of a further embodiment of the resource estimating device;
- Figure 4 illustrates a further embodiment of the method of the invention of estimating computational network resources required to perform a computational task
- Figure 5 illustrates yet a further embodiment of the method of the invention of estimating computational network resources required to perform a computational task
- Figure 6 shows an example how the estimation can be performed by using a decision tree approach as machine learning process for estimating requested resources, according to an exemplifying embodiment of the invention.
- FIG. 1 shows a schematic overview of a communications network 10 in which a resource estimating device (RED) 11 according to an embodiment of the invention may be deployed.
- the resource estimating device may for instance be implemented in the network as a virtual machine.
- a user of, e.g., a computer 11 may request a cloud computing service provided by the network 10.
- PaaS Platform as a service
- a platform is provided for allowing users to create, e.g., Web applications, perform various calculations based on provided data sets, classifications, data clustering, etc., without having to build their own infrastructure.
- Other types of cloud computing services included for example Infrastructure as a service (IaaS) and Software as a service (SaaS).
- Figure 2 shows a signalling diagram illustrating an embodiment of a method performed by the resource estimating device n for estimating computational network resources required to perform a computational task requested by the user of the computer 12 according to the invention. Reference is further made to Figure 1 for structural elements.
- the method at the resource estimating device 11 is performed by a processing unit 20 embodied in the form of one or more microprocessors arranged to execute a computer program 22 downloaded to a suitable storage medium 21 associated with the microprocessor, such as a Random Access
- the processing unit 20 is arranged to carry out the method according to embodiments of the present invention when the appropriate computer program 22 comprising computer- executable instructions is downloaded to the storage medium 21 and executed by the processing unit 20.
- the storage medium 21 may also be a computer program product comprising the computer program 22.
- the computer program 22 maybe transferred to the storage medium 21 by means of a suitable computer program product, such as a Digital Versatile Disc (DVD) or a memory stick.
- a suitable computer program product such as a Digital Versatile Disc (DVD) or a memory stick.
- the computer program 22 maybe downloaded to the storage medium 21 over a network.
- the processing unit 20 may alternatively be embodied in the form of a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), etc.
- DSP digital signal processor
- ASIC application specific integrated circuit
- FPGA field-programmable gate array
- CPLD complex programmable logic device
- the method performed by the resource estimating device 11 is not necessarily performed by a single device, but could be distributed over a plurality of network elements/devices, such as a number of virtual machines.
- the resource estimating device 11 maybe the device providing the requested computational task (but not necessarily). Again, the process of performing the computational task may be distributed over a plurality of elements (not shown) in the network 10, which may or may not include the resource estimating device n.
- a user wishing to have a computational task performed by the network 10 submits in step Sioi a request accordingly via her computer 12 to the resource estimating device 11 as provided in an embodiment of the present invention for estimating computational network resources required to perform the task.
- the request comprises information pertaining to a size of a data set to be processed when performing the requested computational task as well as a description of the computational task.
- the request may, e.g., specify a clustering and classification task to be performed such as "grouping users from a provided data set for service provisioning".
- the actual data set to be processed may be provided with the request in order for the resource estimating device 11 to provide a more accurate estimate.
- the resource estimating device 11 estimates in step S102 the computational network resources required to perform the requested computational task.
- the resource estimating device 11 After having estimated the resources, the resource estimating device 11 advantageously provides a response to the received request in step S103 to the computer 12 specifying the estimated computational network resources required to perform the requested computational task, and the user is hence advantageously given an estimate on the computational network resources required for the network to provide the requested computational task.
- the estimated resources could be provided by the resource estimating device 11 to the user as RAM size, CPU capacity, hard disk size, etc., required to perform the requested computational task.
- the response to the user may advantageously specify the estimated computational network resources in terms of a cost for performing the requested computational task, which for many users often is a preferred measure on the estimated resources, while in a further alternative the response provided
- FIG. 3 illustrates a further embodiment of the present invention of the resource estimating device 11, where separate functional entities are shown.
- a user wishing to have a computational task performed by the network 10 submits a request via her computer 12 to the resource estimating device 11 as provided in an embodiment of the present invention for estimating computational network resources required to perform the task.
- the resource estimating device 11 is configured with a receiving entity 111 for receiving the request, which comprises information pertaining to a size of a data set to be processed, or alternatively the data set itself or a link to the data set, when performing the requested computational task as well as a description of the computational task.
- An estimating entity 112 thereafter estimates the computational network resources required to perform the requested computational task based on the information of the request pertaining to the size of the data set to be processed and the description of the computational task to be performed.
- a response-providing entity 113 submits a response to the received request to the computer 12 specifying the estimated computational network resources required to perform the requested computational task, and the user is hence advantageously given an estimate on the computational network resources required for the network to provide the requested computational task, for instance in the form of a cost and time estimate for having the computational task performed.
- the estimation of the computational network resources is performed by using a machine learning process illustrated by means of learning entity 114.
- a prediction model of the estimation is formed in the estimating entity 112 by using algorithms that can learn from estimated data which is fed back to the prediction model by means of the learning entity 114 to continuously improve the prediction model - and hence the estimation - by taking account previously performed estimations of the computational network resources required for performing a particular requested
- Figure 4 illustrates a further embodiment of the method of the invention of estimating computational network resources required to perform a computational task, where the machine learning process used for processing the information received with the request is selected based on the size of the data set to be processed for performing the requested computational task.
- Figure 3 for structural elements.
- the receiving entity 111 of the resource estimating device 11 receives in step S101 a request from the computer 12 of a user desiring to have a computational task performed by the network 10.
- the request comprises information pertaining to the size of the data set to be processed when performing the requested computational task as well as a description of the computational task.
- the estimating entity 112 of the resource estimating device 11 determines the size of the data set to be processed and further selects which particular machine learning process the information of the request is going to be subjected to depending on the size of the data set to be processed.
- the estimating entity 112 can advantageously select a machine learning approach which best suits the data set.
- step S102 the estimating entity 112 of the resource estimating device 11 estimates the computational network resources required to perform the requested computational task using the selected machine learning process, i.e., random forests, logistic regression, or decision tree learning in this particular exemplifying embodiment.
- the selected machine learning process i.e., random forests, logistic regression, or decision tree learning in this particular exemplifying embodiment.
- step S103 a response is provided to the computer 12 specifying the estimated computational network resources, for instance in terms of a cost and a time, required to perform the requested computational task, and the user is hence advantageously given an estimate on the computational network resources required for the network to provide the requested computational task.
- Figure 5 illustrates yet a further embodiment of the method of the invention of estimating computational network resources required to perform a computational task, where the machine learning process used for processing the information received with the request is selected based on the type of computational task to be performed.
- the receiving entity 111 of the resource estimating device 11 receives in step Sioi a request from the computer 12 of a user desiring to have a computational task performed by the network 10.
- the request comprises information pertaining to the size of the data set to be processed when performing the requested computational task as well as a description of the computational task.
- the estimating entity 112 of the resource estimating device 11 determines the computational task to be performed, and further selects which particular machine learning process the information of the request is going to be subjected to depending on the computational task to be performed.
- a random forests approach is selected as machine learning process
- a logistic regression approach is selected as machine learning process
- a decision tree learning approach is selected as machine learning process.
- the estimating entity 112 can advantageously select a machine learning approach which best suits the computational task.
- step S102 the estimating entity 112 of the resource estimating device 11 estimates the computational network resources required to perform the requested computational task using the selected machine learning process, i.e., random forests, logistic regression, or decision tree learning in this particular exemplifying embodiment approach.
- the selected machine learning process i.e., random forests, logistic regression, or decision tree learning in this particular exemplifying embodiment approach.
- a response is subsequently provided in step S103 to the computer 12 specifying the estimated computational network resources, for instance in terms of a cost and a time, required to perform the requested computational task, and the user is hence advantageously given an estimate on the computational network resources required for the network to provide the requested computational task.
- the estimated computational network resources for instance in terms of a cost and a time, required to perform the requested computational task
- the user is hence advantageously given an estimate on the computational network resources required for the network to provide the requested computational task.
- network load or load on the resources required to perform a computational task is monitored by the resource estimating device 11.
- the resource estimating device 112 In case of scarcity of resources, which in this embodiment is taken into account by the estimating entity 112, the cost of having the
- the resource estimating device 11 is capable of measuring the load itself, it may require input from a network device such as a PaaS manager (not shown) supervising the resources in the network 10.
- the resource estimating device 10 is capable of providing a more accurate resource estimating (possibly in terms of cost and/ or time as previously discussed) by taking into account network and/ or resource load.
- Figure 6 shows an example how the estimation can be performed by using a decision tree approach as machine learning process for improving the estimation of requested resources according to an exemplifying embodiment of the present invention.
- input parameters of each data set to be processed contains number of rows (data points), number of columns (features), total number of numeric variables, total number of categorical variables, sum of all categories, and analytics task type, as illustrated by means of Table 1 in the below where 20 data sets are listed (one per row in the Table).
- the output in the form of estimated resources required to process each data set as specified in the Table for a particular computational task contains number of (equal-capacity) CPU cores utilized, RAM size, and hard disk drive size.
- the decision will initially be based on the sum of all categories, and if the sum of all categories is less than 70, the total number of data points will be considered. In this particular example, if the number of data points is less than 52000, 1 CPU core, 256 MB RAM and 10 GB of disk drive storage space are assigned for performing the computational task on the given data set, otherwise 4 CPU cores, 1024 MB RAM and 80 GB of hard disk storage space are assigned. On a right hand side of the decision tree, if the sum of all categories is 70 or more, other parameters are considered in the decision making; such as total number of numeric variables, number of columns, etc. Eventually, the decision tree approach finds an appropriate estimate on required resources for performing the requested computational task.
- the left-most box indicates that 25% of the total number of computational tasks resides in this box and that the box is associated with a single resource class, namely class 1.
- the next box to the right box indicates that 10% of the total number of
- the estimating entity 111 and the learning entity 114 implements a machine learning process, such as the decision tree approach of Figure 6.
- Machine learning processes typically includes a training phase, where both input data and output data are collected from historical data and supplied to the estimation model implemented by the machine learning process.
- there are different approaches for training the model such as neural networks, random forests, logistic regression, etc.
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a method of estimating computational network resources required to perform a computational task and a device configured to perform the method. The method of estimating computational network resources required to perform a computational task comprises receiving (Sioi) a request for estimating computational network resources, the request comprising information pertaining to a size of a data set to be processed when performing the requested computational task and a description of the computational task, and estimating (S102), based on the information comprised in the request, the computational network resources required to perform the requested computational task. The method further comprises providing (S103) a response to the received request, the response specifying the estimated computational network resources required to perform the requested computational task. The invention further relates to a corresponding computer program and a corresponding computer program product.
Description
METHOD AND DEVICE FOR ESTIMATING COMPUTATIONAL NETWORK RESOURCES REQUIRED TO PERFORM A COMPUTATIONAL TASK
TECHNICAL FIELD
The invention relates to a method of estimating computational network resources required to perform a computational task and a device configured to perform the method. The invention further relates to a corresponding computer program for causing the device to perform the method according to the invention, and a corresponding computer program product. BACKGROUND
In the art, when an analytics service is deployed on a virtual machine, e.g., using Platform as a Service (PaaS) or alternatively Infrastructure as a Service (IaaS), a user requesting the analytics service needs to specify how much resources need to be allocated, e.g., random access memory (RAM) size, microprocessor (CPU) capacity, hard disk size, etc. This information can also be used by the service provider to schedule multiple tasks and optimize the total resource utilization.
In a cloud environment, the analytics services share resources which are dynamically allocated to each service, and users are paying for the resources allocated to them. For users with little experience, or if complex analytics tasks are to be undertaken, it is difficult to estimate the required resources. Typically, the user would like to know the cost to complete the task. If the resource allocation is scarce, it may take too long time for the analytics task to perform. If the resource allocation is over-dimensioned, the user will block an unnecessarily great amount of resources. Another aspect is that if the system knows how much resources are needed for each task, the system can coordinate all tasks to optimize the total system resource utilization.
SUMMARY
An object of the present invention is to solve, or at least mitigate, this problem in the art and to provide an improved estimation of required computational resources provided in a network. This object is attained in a first aspect of the present invention by a method of estimating computational network resources required to perform a
computational task. The method comprises receiving a request for estimating computational network resources, the request comprising information pertaining to a size of a data set to be processed when performing the requested computational task and a description of the computational task, and estimating, based on the information comprised in the request, the computational network resources required to perform the requested computational task. The method further comprises providing a response to the received request, the response specifying the estimated computational network resources required to perform the requested computational task.
This object is attained in a second aspect of the present invention by a resource estimating device configured to estimate computational network resources required to perform a computational task, the device comprising a processing unit and a memory, which memory contains instructions executable by the processing unit, whereby the resource estimating device is operative to receive a request for estimating computational network resources, the request comprising information pertaining to a size of a data set to be processed when performing the requested computational task and a description of the computational task, and to estimate, based on the information comprised in the request, the computational network resources required to perform the requested computational task. Further, the resource estimating device is operative to provide a response to the received request, the response specifying the estimated computational network resources required to perform the requested computational task. Thus, a user wishing to have a computational task performed, i.e., to utilize a computational service provided in a network, for instance in a cloud
environment, will submit a request to a resource estimating device as provided in an embodiment of the present invention for estimating
computational network resources.
The request comprises information pertaining to a size of a data set to be processed when performing the requested computational task and a description of the computational task to be performed. Hence, the request may, e.g., specify a task such as "encryption of a set of data files of a certain size" or specify a clustering and classification task performed such as
"grouping users from a provided data set for service provisioning".
Advantageously, the user does not need to specify the resources to be used in terms of for instance computational or storage capacity. It should be noted that the user need not specify the computational task to be performed in any further detail, but rather inputs the aim of the task to be performed, such as, e.g., grouping, predicting, classifying, etc. Thereafter, based on the information pertaining to the size of the data set to be processed and the description of the computational task to be performed, the resource estimating device estimates the computational network resources required to perform the requested computational task.
Finally, the resource estimating device advantageously provides a response to the received request specifying the estimated computational network resources required to perform the requested computational task.
Thus, the user is advantageously given an estimate for the computational network resources required to provide the requested computational task. It should be noted that the method according to the above described
embodiment of the invention is exemplified as being performed by a resource estimating device, but could be performed by a number of different devices or, e.g., virtual machines, where the steps performed to be able to give an estimate are distributed over the different devices. In a cloud environment, it can even be envisaged that the various devices are located in different parts of the world. As previously mentioned, the response may specify RAM size, CPU
capacity, hard disk size, etc., for performing the requested computational task.
In an embodiment of the present invention, the response provided
advantageously specifies the estimated computational network resources in terms of a cost for performing the requested computational task, which for many users often is a preferred measure on the estimated resources, while in a further embodiment the response provided advantageously specifies the estimated computational network resources in terms of a time when the requested computational task is expected to have been completed. Further advantageous is that with the estimated cost and/ or time, the user is enabled to optimize planning of the computational tasks to be performed. For the user, the estimate may function as a benchmark measure when deciding priority of tasks, as well as a means to compare estimates from competing providers. In another embodiment of the present invention, not only information pertaining to the size of the data set to be processed and the computational task to be performed is used for performing the estimation, but further any one or more of number of attributes, types of the attributes, value ranges of each attributes, etc., for the data set. Advantageously, by extracting a great amount of information, a more exact estimation is possible to attain.
In an embodiment of the present invention, a machine learning process will advantageously be implemented at the resource estimating device for performing estimation of the computational network resources required for performing the requested computational task. Thus, the resource estimating device will, in order to improve the estimation, have the information undergo a machine learning process, where the result of the estimation is fed back to the resource estimating device to make use of the result in order to
continuously improve the estimation process, for instance using decision tree learning, random forests, artificial neural networks, logistic regression, etc. It can further be envisaged that the resource estimating device communicates
with, e.g., a PaaS manager which supervises the network, for obtaining appropriate historical and/or current data to perform the machine learning process based on the obtained data to accomplish training of a model used for the estimation. In still another embodiment of the present invention, the particular machine learning process to be used is selected based on the size of the data set to be processed and/ or the requested computational task. For instance, in case the size of the data set exceeds a threshold value, random forests maybe considered to be the best process to use, while in another example in case the computational task pertains to categorization, decision tree learning is considered the most efficient machine learning process.
Further provided is a computer program for causing the resource estimating device to perform the method according to the invention, and a computer program product comprising computer readable medium having the computer program stored thereon.
Preferred embodiments of the invention will be described in the following.
Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a/an/the element, apparatus, component, means, step, etc." are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention is now described, by way of example, with reference to the accompanying drawings, in which:
Figure 1 shows a schematic overview of a communications network in which a resource estimating device according to an embodiment of the invention may be deployed;
Figure 2 shows a signalling diagram illustrating an embodiment of a method performed by the resource estimating device for estimating computational network resources required to perform a requested computational task according to the invention; Figure 3 illustrates functional entities of a further embodiment of the resource estimating device;
Figure 4 illustrates a further embodiment of the method of the invention of estimating computational network resources required to perform a computational task; Figure 5 illustrates yet a further embodiment of the method of the invention of estimating computational network resources required to perform a computational task; and
Figure 6 shows an example how the estimation can be performed by using a decision tree approach as machine learning process for estimating requested resources, according to an exemplifying embodiment of the invention.
DETAILED DESCRIPTION
The invention will now be described more fully hereinafter with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout the description. Figure 1 shows a schematic overview of a communications network 10 in which a resource estimating device (RED) 11 according to an embodiment of the invention may be deployed. The resource estimating device may for instance be implemented in the network as a virtual machine. A user of, e.g., a computer 11 may request a cloud computing service provided by the
network 10. A particular type of computing service is Platform as a service (PaaS), where a platform is provided for allowing users to create, e.g., Web applications, perform various calculations based on provided data sets, classifications, data clustering, etc., without having to build their own infrastructure. Other types of cloud computing services included for example Infrastructure as a service (IaaS) and Software as a service (SaaS).
Figure 2 shows a signalling diagram illustrating an embodiment of a method performed by the resource estimating device n for estimating computational network resources required to perform a computational task requested by the user of the computer 12 according to the invention. Reference is further made to Figure 1 for structural elements.
In practice, the method at the resource estimating device 11 is performed by a processing unit 20 embodied in the form of one or more microprocessors arranged to execute a computer program 22 downloaded to a suitable storage medium 21 associated with the microprocessor, such as a Random Access
Memory (RAM), a Flash memory or a hard disk drive. The processing unit 20 is arranged to carry out the method according to embodiments of the present invention when the appropriate computer program 22 comprising computer- executable instructions is downloaded to the storage medium 21 and executed by the processing unit 20. The storage medium 21 may also be a computer program product comprising the computer program 22.
Alternatively, the computer program 22 maybe transferred to the storage medium 21 by means of a suitable computer program product, such as a Digital Versatile Disc (DVD) or a memory stick. As a further alternative, the computer program 22 maybe downloaded to the storage medium 21 over a network. The processing unit 20 may alternatively be embodied in the form of a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), etc. As previously has been mentioned, the method performed by the resource estimating device 11 is not necessarily performed by a single device, but could
be distributed over a plurality of network elements/devices, such as a number of virtual machines. Further, in the network 10 illustrated with reference to Figure 1, the resource estimating device 11 maybe the device providing the requested computational task (but not necessarily). Again, the process of performing the computational task may be distributed over a plurality of elements (not shown) in the network 10, which may or may not include the resource estimating device n.
With reference to Figure 2, a user wishing to have a computational task performed by the network 10 submits in step Sioi a request accordingly via her computer 12 to the resource estimating device 11 as provided in an embodiment of the present invention for estimating computational network resources required to perform the task.
The request comprises information pertaining to a size of a data set to be processed when performing the requested computational task as well as a description of the computational task. As previously mentioned, the request may, e.g., specify a clustering and classification task to be performed such as "grouping users from a provided data set for service provisioning". It should be noted that in an embodiment of the present invention, the actual data set to be processed may be provided with the request in order for the resource estimating device 11 to provide a more accurate estimate. However, in another embodiment, it is sufficient to provide a measure of the size of the data set to be processed.
Thereafter, based on the information pertaining to the size of the data set to be processed and the description of the computational task to be performed, the resource estimating device 11 estimates in step S102 the computational network resources required to perform the requested computational task.
After having estimated the resources, the resource estimating device 11 advantageously provides a response to the received request in step S103 to the computer 12 specifying the estimated computational network resources required to perform the requested computational task, and the user is hence
advantageously given an estimate on the computational network resources required for the network to provide the requested computational task.
The estimated resources could be provided by the resource estimating device 11 to the user as RAM size, CPU capacity, hard disk size, etc., required to perform the requested computational task. Alternatively, the response to the user may advantageously specify the estimated computational network resources in terms of a cost for performing the requested computational task, which for many users often is a preferred measure on the estimated resources, while in a further alternative the response provided
advantageously specifies the estimated computational network resources in terms of a time when the requested computational task is expected to have been completed. In yet a further alternative, the estimated resources are advantageously provided as both the cost for performing the requested computational task and the estimated time of completion of the task. Figure 3 illustrates a further embodiment of the present invention of the resource estimating device 11, where separate functional entities are shown. Thus, a user wishing to have a computational task performed by the network 10 submits a request via her computer 12 to the resource estimating device 11 as provided in an embodiment of the present invention for estimating computational network resources required to perform the task. The resource estimating device 11 is configured with a receiving entity 111 for receiving the request, which comprises information pertaining to a size of a data set to be processed, or alternatively the data set itself or a link to the data set, when performing the requested computational task as well as a description of the computational task.
An estimating entity 112 thereafter estimates the computational network resources required to perform the requested computational task based on the information of the request pertaining to the size of the data set to be processed and the description of the computational task to be performed.
Finally, a response-providing entity 113 submits a response to the received request to the computer 12 specifying the estimated computational network resources required to perform the requested computational task, and the user is hence advantageously given an estimate on the computational network resources required for the network to provide the requested computational task, for instance in the form of a cost and time estimate for having the computational task performed.
Further, in the embodiment of the resource estimating device 11 illustrated in Figure 3, the estimation of the computational network resources is performed by using a machine learning process illustrated by means of learning entity 114. Hence, a prediction model of the estimation is formed in the estimating entity 112 by using algorithms that can learn from estimated data which is fed back to the prediction model by means of the learning entity 114 to continuously improve the prediction model - and hence the estimation - by taking account previously performed estimations of the computational network resources required for performing a particular requested
computational task.
Figure 4 illustrates a further embodiment of the method of the invention of estimating computational network resources required to perform a computational task, where the machine learning process used for processing the information received with the request is selected based on the size of the data set to be processed for performing the requested computational task. Reference is further made to Figure 3 for structural elements.
Hence, the receiving entity 111 of the resource estimating device 11 receives in step S101 a request from the computer 12 of a user desiring to have a computational task performed by the network 10.
The request comprises information pertaining to the size of the data set to be processed when performing the requested computational task as well as a description of the computational task. In step Si02a, the estimating entity 112 of the resource estimating device 11 determines the size of the data set to be
processed and further selects which particular machine learning process the information of the request is going to be subjected to depending on the size of the data set to be processed.
In this particular exemplifying embodiment, in case the size of the data set is of category "I", a random forests approach is selected as machine learning process, in case the size of the data set is of category "Π", a logistic regression approach is selected as machine learning process, and in case the size of the data set is of category "III", a decision tree learning approach is selected as machine learning process. Thus, by determining the size of the data set to be processed, the estimating entity 112 can advantageously select a machine learning approach which best suits the data set.
In step S102, the estimating entity 112 of the resource estimating device 11 estimates the computational network resources required to perform the requested computational task using the selected machine learning process, i.e., random forests, logistic regression, or decision tree learning in this particular exemplifying embodiment.
Finally, in step S103, a response is provided to the computer 12 specifying the estimated computational network resources, for instance in terms of a cost and a time, required to perform the requested computational task, and the user is hence advantageously given an estimate on the computational network resources required for the network to provide the requested computational task.
Figure 5 illustrates yet a further embodiment of the method of the invention of estimating computational network resources required to perform a computational task, where the machine learning process used for processing the information received with the request is selected based on the type of computational task to be performed. Reference is further made to Figure 3 for structural elements.
Again, the receiving entity 111 of the resource estimating device 11 receives in step Sioi a request from the computer 12 of a user desiring to have a computational task performed by the network 10.
The request comprises information pertaining to the size of the data set to be processed when performing the requested computational task as well as a description of the computational task. In step Si02b, the estimating entity 112 of the resource estimating device 11 determines the computational task to be performed, and further selects which particular machine learning process the information of the request is going to be subjected to depending on the computational task to be performed.
In this particular exemplifying embodiment, in case the task is of
category "I", a random forests approach is selected as machine learning process, in case the task is of category "Π", a logistic regression approach is selected as machine learning process, and in case the task is of category "III", a decision tree learning approach is selected as machine learning process.
Thus, by determining the type of task to be performed, the estimating entity 112 can advantageously select a machine learning approach which best suits the computational task.
In step S102, the estimating entity 112 of the resource estimating device 11 estimates the computational network resources required to perform the requested computational task using the selected machine learning process, i.e., random forests, logistic regression, or decision tree learning in this particular exemplifying embodiment approach.
A response is subsequently provided in step S103 to the computer 12 specifying the estimated computational network resources, for instance in terms of a cost and a time, required to perform the requested computational task, and the user is hence advantageously given an estimate on the computational network resources required for the network to provide the requested computational task.
As previously discussed, in an embodiment of the invention, not only information pertaining to the size of the data set to be processed and the computational task to be performed is used for performing the estimation, but further any one or more of number of attributes, types of the attributes, value ranges of each attributes, etc., for the data set, which further can be used for selecting the machine learning process to be employed in analogy with the embodiments illustrated with reference to Figures 4 and 5. Further, both the size of the data set to be processed and the requested computational task may advantageously be taken into account when selecting the machine learning process to use.
In still another embodiment, network load or load on the resources required to perform a computational task is monitored by the resource estimating device 11. In case of scarcity of resources, which in this embodiment is taken into account by the estimating entity 112, the cost of having the
computational task performed is higher, and time of completion is generally longer. Unless the resource estimating device 11 is capable of measuring the load itself, it may require input from a network device such as a PaaS manager (not shown) supervising the resources in the network 10. The resource estimating device 10 is capable of providing a more accurate resource estimating (possibly in terms of cost and/ or time as previously discussed) by taking into account network and/ or resource load.
Figure 6 shows an example how the estimation can be performed by using a decision tree approach as machine learning process for improving the estimation of requested resources according to an exemplifying embodiment of the present invention. Assuming input parameters of each data set to be processed contains number of rows (data points), number of columns (features), total number of numeric variables, total number of categorical variables, sum of all categories, and analytics task type, as illustrated by means of Table 1 in the below where 20 data sets are listed (one per row in the Table). The output in the form of estimated resources required to process each data set as specified in the Table for a particular computational task
contains number of (equal-capacity) CPU cores utilized, RAM size, and hard disk drive size.
No. No. Num. Cat. No. Task Type Resources
Rows Cols Vars Vars Cats
1000 100 50 50 100 Descriptive ICPU256MB10GB
1000000 100 50 50 100 Descriptive 4CPU1024MB80GB
1500 200 170 30 80 Descriptive 2CPU512MB20GB
2000 100 50 50 100 Predictive 2CPU512MB20GB
1500 100 70 30 60 Predictive ICPU256MB10GB
3000 100 80 20 40 Descriptive ICPU256MB10GB
2500 150 130 20 40 Descriptive ICPU256MB10GB
3000 130 120 10 20 Predictive ICPU256MB10GB
2600 150 130 10 50 Descriptive ICPU256MB10GB
9000 100 60 40 90 Descriptive 2CPU512MB20GB
3000 300 240 60 120 Descriptive 2CPU512MB20GB
6000 160 120 40 90 Descriptive 2CPU512MB20GB
5000 180 120 60 130 Descriptive 2CPU512MB20GB
4000 200 150 50 150 Descriptive 2CPU512MB20GB
100000 150 120 30 60 Predictive 4CPU1024MB80GB
6000 180 50 50 300 Predictive 4CPU1024MB80GB
600000 I50 140 10 30 Descriptive 4CPU1024MB80GB
700000 120 90 30 80 Descriptive 4CPU1024MB80GB
3000 3OO 40 60 3OO Descriptive 4CPU1024MB80GB
2000 I50 20 80 4OO Descriptive 4CPU1024MB80GB
Table 1
Thus, in the Table, there are three different resource classes:
1: ICPU256MB10GB,
2: 2CPU512MB20GB, and 3: 4CPU1024MB80GB.
In the given exemplifying decision tree approach of Figure 6, the decision will initially be based on the sum of all categories, and if the sum of all categories is less than 70, the total number of data points will be considered. In this particular example, if the number of data points is less than 52000, 1 CPU core, 256 MB RAM and 10 GB of disk drive storage space are assigned for performing the computational task on the given data set, otherwise 4 CPU cores, 1024 MB RAM and 80 GB of hard disk storage space are assigned. On a right hand side of the decision tree, if the sum of all categories is 70 or more, other parameters are considered in the decision making; such as total number of numeric variables, number of columns, etc. Eventually, the decision tree approach finds an appropriate estimate on required resources for performing the requested computational task.
Looking at the decision boxes at the bottom of the tree (where the goal is to have a single resource class in each decision box); the left-most box indicates that 25% of the total number of computational tasks resides in this box and that the box is associated with a single resource class, namely class 1. The next box to the right box indicates that 10% of the total number of
computational tasks resides in this box and that the box is associated with a
l6 single resource class, i.e. class 3. The box located even a further step to the right box indicates that 25% of the total number of computational tasks resides in this box and that the box is associated with a resource class 2 only, and so on. Now, with the reference to the resource estimating device 11 of Figure 4, the estimating entity 111 and the learning entity 114 implements a machine learning process, such as the decision tree approach of Figure 6. Machine learning processes typically includes a training phase, where both input data and output data are collected from historical data and supplied to the estimation model implemented by the machine learning process. As previously mentioned, there are different approaches for training the model, such as neural networks, random forests, logistic regression, etc. Once the model is trained, a decision model similar to the decision tree of Figure 5 can be created, which is utilized to estimate the resource usage for requested analytics tasks.
The invention has mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.
Claims
1. A method of estimating computational network resources required to perform a computational task, comprising:
receiving (S101) a request for estimating computational network resources, the request comprising information pertaining to a size of a data set to be processed when performing the requested computational task and a description of the computational task;
estimating (S102), based on the information comprised in the request, the computational network resources required to perform the requested computational task; and
providing (S103) a response to the received request, the response specifying the estimated computational network resources required to perform the requested computational task.
2. The method according to claim 1, wherein the provided response specifies the estimated computational network resources in terms of a cost for performing the requested computational task.
3. The method according to claims 1 or 2, wherein the provided response specifies the estimated computational network resources in terms of a time when the requested computational task is expected to have been completed.
4. The method according to any one of the preceding claims, wherein the estimation is performed by having said information undergo a machine learning process.
5. The method according to claim 4, the machine learning process being any one or more selected from a group comprising: decision tree learning, random forests, artificial neural networks, logistic regression, association rule learning, inductive logic programming, support vector machines.
6. The method according to claim 5, further comprising:
selecting (Si02a) the machine learning process from said group based on the size of a data set to be processed.
l8
7. The method according to claims 5 or 6, further comprising:
selecting (Si02b) the machine learning process from said group based on the requested computational task.
8. The method according to any one of the preceding claims, wherein the estimating of the computational network resources required to perform the requested computational task further is based on network load.
9. A computer program (22) comprising computer-executable instructions for causing a device (11) to perform steps recited in any one of claims 1-8 when the computer-executable instructions are executed on a processing unit (20) included in the device.
10. A computer program product comprising a computer readable medium (21), the computer readable medium having the computer program (22) according to claim 9 embodied therein.
11. A resource estimating device (11) configured to estimate computational network resources required to perform a computational task, the device comprising a processing unit (20) and a memory (21), said memory containing instructions (22) executable by said processing unit, whereby said resource estimating device is operative to:
receive a request for estimating computational network resources, the request comprising information pertaining to a size of a data set to be processed when performing the requested computational task and a description of the computational task;
estimate, based on the information comprised in the request, the computational network resources required to perform the requested computational task; and
provide a response to the received request, the response specifying the estimated computational network resources required to perform the requested computational task.
12. The resource estimating device (11) according to claim 11, wherein the provided response is configured to specify the estimated computational
network resources in terms of a cost for performing the requested
computational task.
13. The resource estimating device (11) according to claims 11 or 12, wherein the provided response is configured to specify the estimated computational network resources in terms of a time when the requested computational task is expected to have been completed.
14. The resource estimating device (11) according to any one of claims 11-13, further being operative to perform the estimation by having said information undergo a machine learning process.
15. The resource estimating device (11) according to claim 14, further being operative to select the machine learning process from any one or more from a group comprising: decision tree learning, random forests, artificial neural networks, logistic regression, association rule learning, inductive logic programming, support vector machines.
16. The resource estimating device (11) according to claim 15, further being operative to:
select the machine learning process from said group based on the size of a data set to be processed.
17. The resource estimating device (11) according to claims 15 or 16, further being configured to:
select the machine learning process from said group based on the requested computational task.
18. The resource estimating device (11) according to any one of claims 11-17, further being configured to estimate the computational network resources required to perform the requested computational task based on network load.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/SE2015/050263 WO2016144220A1 (en) | 2015-03-10 | 2015-03-10 | Method and device for estimating computational network resources required to perform a computational task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/SE2015/050263 WO2016144220A1 (en) | 2015-03-10 | 2015-03-10 | Method and device for estimating computational network resources required to perform a computational task |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2016144220A1 true WO2016144220A1 (en) | 2016-09-15 |
Family
ID=53039555
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/SE2015/050263 WO2016144220A1 (en) | 2015-03-10 | 2015-03-10 | Method and device for estimating computational network resources required to perform a computational task |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2016144220A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11316637B2 (en) | 2015-04-09 | 2022-04-26 | Telefonaktiebolaget Lm Ericsson (Publ) | Physical resource block allocation of physical downlink control and data channels |
| CN115174695A (en) * | 2022-07-18 | 2022-10-11 | 中软航科数据科技(珠海横琴)有限公司 | Scheduling system and method for distributed network resources |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130254196A1 (en) * | 2012-03-26 | 2013-09-26 | Duke University | Cost-based optimization of configuration parameters and cluster sizing for hadoop |
| WO2014124448A1 (en) * | 2013-02-11 | 2014-08-14 | Amazon Technologies, Inc. | Cost-minimizing task scheduler |
-
2015
- 2015-03-10 WO PCT/SE2015/050263 patent/WO2016144220A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130254196A1 (en) * | 2012-03-26 | 2013-09-26 | Duke University | Cost-based optimization of configuration parameters and cluster sizing for hadoop |
| WO2014124448A1 (en) * | 2013-02-11 | 2014-08-14 | Amazon Technologies, Inc. | Cost-minimizing task scheduler |
Non-Patent Citations (1)
| Title |
|---|
| SUNGPACK HONG ET AL: "Efficient Parallel Graph Exploration on Multi-Core CPU and GPU", PARALLEL ARCHITECTURES AND COMPILATION TECHNIQUES (PACT), 2011 INTERNATIONAL CONFERENCE ON, IEEE, 10 October 2011 (2011-10-10), pages 78 - 88, XP032083290, ISBN: 978-1-4577-1794-9, DOI: 10.1109/PACT.2011.14 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11316637B2 (en) | 2015-04-09 | 2022-04-26 | Telefonaktiebolaget Lm Ericsson (Publ) | Physical resource block allocation of physical downlink control and data channels |
| CN115174695A (en) * | 2022-07-18 | 2022-10-11 | 中软航科数据科技(珠海横琴)有限公司 | Scheduling system and method for distributed network resources |
| CN115174695B (en) * | 2022-07-18 | 2024-01-26 | 中软航科数据科技(珠海横琴)有限公司 | Scheduling system and method for distributed network resources |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112153700B (en) | Network slice resource management method and equipment | |
| CN108009016B (en) | Resource load balancing control method and cluster scheduler | |
| US10474504B2 (en) | Distributed node intra-group task scheduling method and system | |
| US10289973B2 (en) | System and method for analytics-driven SLA management and insight generation in clouds | |
| CN106133696B (en) | Dynamic marks target capacity when scaling cloud resource | |
| US20190394132A1 (en) | System and Method for Network Slicing for Service-Oriented Networks | |
| TWI547817B (en) | Method, system and apparatus of planning resources for cluster computing architecture | |
| CN106326002B (en) | Resource scheduling method, device and equipment | |
| CN113434253B (en) | Cluster resource scheduling method, device, equipment and storage medium | |
| CN105703927B (en) | A kind of resource allocation methods, the network equipment and network system | |
| KR102229859B1 (en) | Operation management method, apparatus and system using machine learning and transfer learning | |
| CN107077385A (en) | Calculated examples start the time | |
| US12001868B2 (en) | VM migration system and VM migration method | |
| Bousselmi et al. | QoS-aware scheduling of workflows in cloud computing environments | |
| CN108021441A (en) | A kind of resources of virtual machine collocation method and device based on cloud computing | |
| Zhao et al. | Integrated facility layout design and flow assignment problem under uncertainty | |
| CN105868222A (en) | Task scheduling method and device | |
| CN110460662A (en) | The processing method and system of internet of things data | |
| Genez et al. | A particle swarm optimization approach for workflow scheduling on cloud resources priced by CPU frequency | |
| CN114095382A (en) | Network slice virtual resource scheduling method, system, device and equipment | |
| WO2016144220A1 (en) | Method and device for estimating computational network resources required to perform a computational task | |
| Loheswaran | An upgraded fruit fly optimisation algorithm for solving task scheduling and resource management problem in cloud infrastructure | |
| CN117311998B (en) | Large model deployment method and system | |
| CN111143033B (en) | Operation execution method and device based on scalable operation system | |
| WO2018168695A1 (en) | Distributed machine learning device, distributed machine learning method, and distributed machine learning recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15719867 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 15719867 Country of ref document: EP Kind code of ref document: A1 |