CN117176728A - Industrial Internet of things dispatching method and dispatching system based on cloud edge cooperative technology - Google Patents
Industrial Internet of things dispatching method and dispatching system based on cloud edge cooperative technology Download PDFInfo
- Publication number
- CN117176728A CN117176728A CN202310808763.8A CN202310808763A CN117176728A CN 117176728 A CN117176728 A CN 117176728A CN 202310808763 A CN202310808763 A CN 202310808763A CN 117176728 A CN117176728 A CN 117176728A
- Authority
- CN
- China
- Prior art keywords
- node
- computing
- task
- calculation
- preferred
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000005516 engineering process Methods 0.000 title claims abstract description 22
- 238000004364 calculation method Methods 0.000 claims abstract description 119
- 230000004044 response Effects 0.000 claims abstract description 69
- 238000012360 testing method Methods 0.000 claims abstract description 35
- 238000013142 basic testing Methods 0.000 claims description 9
- 238000012163 sequencing technique Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 abstract description 2
- 230000015654 memory Effects 0.000 description 16
- 230000006870 function Effects 0.000 description 14
- 238000012545 processing Methods 0.000 description 13
- 238000003860 storage Methods 0.000 description 11
- 238000004519 manufacturing process Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 241000544061 Cuculus canorus Species 0.000 description 3
- 238000007621 cluster analysis Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000009776 industrial production Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Landscapes
- Debugging And Monitoring (AREA)
- Multi Processors (AREA)
Abstract
The invention belongs to the technical field of cloud edge coordination, and particularly relates to an industrial Internet of things scheduling method and system based on cloud edge coordination technology. Acquiring a preferred execution node of the computing task according to meta information of the computing task; acquiring a computing node set which has approximate response to the preferred executing node in the unverified node according to the expected response value of the preferred executing node to the computing task; constructing a benchmark test task based on the calculation tasks, and at least selecting one candidate calculation node and a preferred execution node in the calculation node set to execute the benchmark test task; and updating the priority of the computing node for the computing task according to the execution results of the candidate computing node and the preferred executing node selected by the reference test task, and generating a control instruction of an application layer. The scheme of the invention can realize reasonable distribution of the computing resources.
Description
Technical Field
The invention belongs to the technical field of cloud edge coordination, and particularly relates to an industrial Internet of things scheduling method and system based on cloud edge coordination technology.
Background
The distributed industrial internet of things (IIOT) equipment at the edge of the intelligent factory network collects real-time data, the data are uploaded to a server for processing and analysis, and analysis results are transmitted to the bottom hardware equipment in the form of production instructions or control signals. The process involves scheduling and allocation of resources, which if not responded and handled in time, can greatly affect the normal production of the factory. Although the computing power enhancement of the computing nodes is realized through the hardware release, the difficult problems of reasonable distribution of the computing power of the edge computing and the computing power of the cloud computing server cannot be realized.
The prior art relates to such problems with the following solutions:
CN114546623B discloses a task scheduling method and system based on big data system, the method comprises creating task based on task creation request; calculating a task feature matrix, and putting the task into a task pool; acquiring an available computing node set; selecting a target task set from a task pool based on the set of available computing nodes; tasks in a target set of tasks are assigned to the set of available computing nodes.
CN105700948A discloses a method and apparatus for scheduling computing tasks in a cluster; acquiring a plurality of computing tasks to be scheduled in a cluster; dividing the plurality of computing tasks into a plurality of task subsets according to the task load information corresponding to the computing tasks, wherein each task subset corresponds to one cluster node in the cluster.
CN114741165a discloses a processing method, a computer device and a storage device of a data processing platform. The method comprises the following steps: receiving a task processing request of data to be processed; determining a management and control execution node meeting a task processing request; performing affinity scheduling on a first task of an execution task processing request based on the management and control execution node so as to allocate the first execution node; executing a first task on the data to be processed by using a first executing node to obtain intermediate processing data; performing affinity scheduling on a second task of the execution task processing request based on the management and control execution node so as to allocate the second execution node; and executing a second task on the intermediate processing data by using a second execution node to obtain a processing result.
The method cannot solve the problem that when the resource theoretically meets the requirement, the actual response time is longer than the theoretical value, and when the calculation task is associated with two ends of the cloud edge, better division cannot be performed.
Disclosure of Invention
An object of the present invention is to provide a computing node scheduling scheme applied to an intelligent plant, which is used for solving the problem of scheduling one or more computing nodes in the prior art, so as to realize proper task loading and execution, and ensure the scheduling and execution of computing tasks of the intelligent plant based on the task loading and execution.
According to one aspect of the invention, the invention firstly discloses an industrial Internet of things scheduling method based on cloud edge cooperative technology, which comprises the following steps:
acquiring a preferred execution node of the computing task according to meta information of the computing task;
acquiring a computing node set which has approximate response to the preferred executing node in the unverified node according to the expected response value of the preferred executing node to the computing task;
constructing a benchmark test task based on the calculation tasks, and at least selecting one candidate calculation node and a preferred execution node in the calculation node set to execute the benchmark test task;
and updating the priority of the computing node for the computing task according to the execution results of the candidate computing node and the preferred executing node selected by the reference test task, and generating a control instruction of an application layer.
According to one embodiment of the invention, the meta-information of the computing task includes resource requirements, response requirements, computing type, and target application.
According to one embodiment of the invention, the set of computing nodes is obtained as follows:
selecting computing nodes meeting the resource limit configuration according to the types and the demands of the resources, sorting the computing nodes according to the proportion of the available resource limit, grouping the computing nodes according to the information of the network slice to which the nodes belong, and selecting cloud computing nodes and edge computing nodes according to the preset proportion.
According to one embodiment of the invention, the preferred execution node is obtained as follows:
extracting the calculation task type contained in the meta-information of the task, selecting a calculation node sequence with a high grading level according to the calculation task type, and sequencing the calculation node sequence according to the proportion of available resources to obtain the calculation node with the highest proportion of available resources as the preferred execution node.
According to one embodiment of the invention, when the benchmark test task is executed, a callback monitor is built on the scheduling server, a callback layer of the benchmark test task is replaced by a port of the scheduling server monitor, a message is sent to the callback layer of the calculation task according to an execution result of the benchmark test task, and a control instruction for the terminal side device is generated.
According to one embodiment of the invention, when the callback monitor receives the calculation results returned by the candidate calculation node and the preferred execution node, the callback monitor compares the candidate calculation node and the preferred execution node, and determines whether recalculation is needed according to the difference of the candidate calculation node and the preferred execution node; if the candidate computing node does not need to be recalculated, the response rating of the candidate computing node is updated according to the response time length of the candidate computing node.
According to one embodiment of the invention, the step of determining whether recalculation is required comprises:
judging whether the response time length and the calculation time length of the preferred execution node are respectively larger than a corresponding response time length threshold value and a calculation time length threshold value, and reselecting the preferred execution node to execute a calculation task and comparing the result with the candidate calculation node when at least one of the response time length and the calculation time length is higher than the threshold value;
and when the response time length and the calculation time length are lower than the threshold value, selecting different cloud calculation nodes and edge calculation nodes from the calculation node set to execute the benchmark test task.
According to one embodiment of the invention, when the edge side server meets the requirement, adding the edge side server into a high priority node corresponding to the calculation type; and when the cloud server meets the requirements, adding the cloud testing server into a common priority node corresponding to the calculation type.
According to one embodiment of the invention, a computing node is marked as an unverified node when the number of times it performs a computing task exceeds a threshold.
According to a second aspect of the present invention, the present invention discloses an industrial internet of things dispatching system based on cloud edge cooperative technology, comprising:
the preferred execution node acquisition unit is used for acquiring preferred execution nodes of the computing task according to the meta information of the computing task;
the computing node set acquisition unit is used for acquiring a computing node set which has approximate response with the preferred executing node in the unverified node according to the expected response value of the preferred executing node to the computing task;
the basic test task generating unit is used for constructing a basic test task based on the calculation task, and at least selecting one candidate calculation node and a preferred execution node in the calculation node set to execute the basic test task;
and the scheduling control unit is used for updating the priority of the computing node for the computing task according to the execution results of the candidate computing node and the preferred executing node selected by the reference test task and generating a control instruction of an application layer.
Based on the technical scheme, reasonable loading and execution of tasks can be realized. And the execution of the reference task does not affect the production of the factory, and the priority allocation of the computing resources can be realized according to the priority.
Drawings
FIG. 1 is a topological schematic diagram of an industrial Internet of things based on cloud edge cooperative technology;
FIG. 2 illustrates an industrial Internet of things scheduling method based on Yun Bian collaborative technology in one embodiment of the invention;
FIG. 3 is a schematic diagram of a method for selecting a preferred execution node in one embodiment of the invention;
FIG. 4 is a schematic diagram of a method for selecting a preferred execution node in another embodiment of the present invention;
FIG. 5 is a flow diagram of scheduling using callback listeners in one embodiment of the present invention.
Detailed Description
The following are several embodiments of the present invention, which are described and illustrated in detail in relation to the technical content of the present invention.
Before proceeding with a detailed description, some techniques involved in the present invention will be briefly described.
Edge computing techniques can be understood as computing that occurs at the edge of a network. In the traditional computing mode, terminal devices are directly connected with a cloud server through a gateway, the terminal devices directly transmit data to a data center or cloud at a central position, the cloud server uniformly processes the data and sends out instructions, and the mode can process a large amount of complex data.
However, as the number of terminal devices increases, the burden of the cloud server gradually increases, and the data processing speed is also reduced due to the increase of the data volume, so that higher time delay is generated, and the requirements of some industrial production links on time delay sensitive applications cannot be met. As shown in fig. 1, in a normal industrial production process, some data that needs to be processed in time is always accompanied, and if the time delay is large, irreversible loss may be caused. The above problems can be well solved by placing some simple data analysis processing functions in the edge server closer to the terminal.
However, with the development of the internet of things (Internet of Things, ioT) and the popularity of IoT devices, a large number of computationally intensive and delay-sensitive applications are becoming increasingly popular. Since part of the resources are not well satisfied on the edge server side and it should be considered that whether an edge server can function depends on the type of task calculated, rather than simple task loading and task unloading, the implementation of generalization often depends on weakening and more restrictions of the functions so that the functions can always be implemented on existing resources, the above-described modes are not friendly for actual users and developers of the resource service.
Where there are multiple candidate resource selections, proper task loading and execution is also a challenge for the dispatch server. For complex models and complex tasks, if only computing resources are provided, without regard to associated resources, such as server connection data, authentication data, cache data, and bandwidth data, all resources may be replaceable, but hardware-based simplification can be confusing to the service based on actual deployment of service feedback.
For example, a similarly configured computing node, when its facing resource includes a database, may need to provide a check of the computation basis before performing the computation because its response to a database query is inconsistent due to the cache data within its page being inconsistent.
In this regard, the solution of the invention is illustrated by the following examples.
Referring to a connection diagram shown in fig. 1 and fig. 2, the industrial internet of things scheduling method based on cloud-edge cooperative technology of the present invention includes:
acquiring a preferred execution node of the computing task according to meta information of the computing task;
acquiring a computing node set which has approximate response to the preferred executing node in the unverified node according to the expected response value of the preferred executing node to the computing task;
constructing a benchmark test task based on the calculation tasks, and at least selecting one candidate calculation node and a preferred execution node in the calculation node set to execute the benchmark test task;
and updating the priority of the computing node for the computing task according to the execution results of the candidate computing node and the preferred executing node selected by the reference test task, and generating a control instruction of an application layer.
Firstly, in one embodiment of the invention, a plurality of logic interfaces are provided at the internet of things terminal of the intelligent factory; the logic interface is connected with the corresponding network slice; the terminal of the Internet of things is accessed into a 5G network through the network slice; through the virtual multiple logical interfaces, the terminal of the Internet of things can be simultaneously accessed into multiple different network slices, so that user experience is improved.
The logical interfaces include ethernet interfaces and may include 5G type enhanced mobile broadband eMBB network slices, large-scale internet of things emtc network slices, and ultra high reliability low latency communication uilllc network slices.
Because the internet of things equipment accesses the network, the requirements of different types of application scenes on the network are differentiated, and some of the application scenes even conflict with each other. Providing services for different types of application scenes through a single network at the same time can lead to abnormal complex network architecture, low network management efficiency and low resource utilization efficiency. The network slicing technology provides mutually isolated network environments for different application scenes in a virtual independent logic network mode on the same network infrastructure, so that the different application scenes can customize network functions and characteristics according to respective requirements, and the service quality requirements of different services can be practically ensured.
The processes of analyzing, collecting and generating computing tasks of data generated by the sensors and the terminal nodes in the industrial internet of things IIOT can be performed in a common mode.
The end side of the IIOT can periodically send data and generate calculation tasks, and when the device data is reported according to a specific time interval. For example, the sensor collects data once every one minute and sends it to the gateway, which also sends device data to the edge node or cloud server for every one minute.
The cloud server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), basic cloud computing services such as big data and artificial intelligent platforms, and the like. The dispatching node can be directly or indirectly connected with the cloud server through a wired or wireless communication mode.
The edge server can be deployed with various factory application software, and the application software can be deployed at one time on the edge server or can be provided with a trackable program for mirror deployment, loading and unloading. Specifically, after the connection between the edge side node and the resource server is established, an application list may be obtained from the resource server, based on the application list, when any one of the edge side clients connected with the resource server sends a collaboration request to the resource server, the collaboration scheduling unit analyzes and identifies the collaboration request to obtain a request content contained in the collaboration request, one or more corresponding first collaboration thread sets are opened under the control of the collaboration scheduling control unit according to the request content so as to be linked to the edge side client through a cloud network, and a collaboration cloud link is provided for the edge side client, and the edge server obtains an edge application from the cloud storage module correspondingly based on the collaboration cloud link.
The edge server is a functional device for providing a user with a network access to communicate with other server devices, and is usually a group of servers for completing a single function, such as a firewall server, a cache server, a load balancing server and the like, so that for the Internet of things, the edge computing technology breaks through, and many controls are completed through a local edge computing layer, so that the processing efficiency can be improved, and the load of the cloud can be reduced. The need is addressed at the edge by being closer to the user, which may also provide faster response to the user.
After the edge server deploys a local program that can be scheduled by the scheduling node, it forms an edge node within the network slice. The edge nodes may communicate based on API, websocket, API, HTTP, etc.
The characteristics of the computing task include meta information meta info, which includes a task source, time, expected computing program version and time delay requirements, and obviously, in the design of the present invention, the IOT device side should avoid the need of the computing task other than time delay, and the corresponding functions are distributed at the scheduling side. Correspondingly, the scheduling node can acquire the characteristic information according to the type of the task.
The meta-information comprises a task source, a source task version and a time delay, wherein the task source and the source task version are registered in the scheduling server, and obviously, when the scheduling server cannot acquire or understand the meta-information, the scheduling server cannot execute corresponding calculation tasks. When the task source and the source task version are registered in the scheduling server, corresponding node information which can be used for processing the task can be searched in the scheduling server according to key information formed by the task source and the source task version.
The task information is matched with the task information, and the task information is an edge server or a cloud server, and can be used for CPU resource use, memory resource use, cache resource use, memory bandwidth use, network bandwidth resource use, disk capacity resource use, GPU resource use and the like. The usage of each class of resources may be described by a plurality of parameter information, for example, the CPU resource usage may include a CPU core number, a CPU model number, and the like, and the memory resource usage may further include a memory size, a memory remaining capacity, and the like. Although we consider that in the case of sufficient resources, the approximated cloud resources should represent approximated behavior, there are differences in enterprises, for example, the response speed of part of edge nodes to computing tasks in the debugging process always represents a range of normal deviation. In this case, it is of positive importance to comprehensively consider the performance in the factory environment.
In an actual computing process, part of the computing process relies on resources held at the node and on resources on other physical hosts.
For example, when a model is stored locally, in the case where the hardware configuration is substantially consistent before the model fails, the response speed of the node storing the model is theoretically higher than the response speed of the model not storing the model, and therefore, it takes a certain time to re-download the model, load the model, and verify the model. Although requests for resources may be ignored when traffic times exceed seconds in a territory, the accumulation of related resources is a considerable value when multiple tasks are performed. Further, if a portion of the resources depend on already cached resources or more time is required for initialization of a portion of the resources, then the resource status of the node must be considered when performing the computation. Furthermore, the increase in consumption of partially dependent resources is not linear, e.g. the demand for databases is slow when the cache page hit rate is low; when the data size and the buffer size are not matched, the searching speed is also reduced; only when the content of the cache page is basically consistent with the content of the demand, better query efficiency can be obtained. Also, if the consumption of resources of one physical machine may not be linear, its RAM may exhibit a non-linear rapid increase when IO increases rapidly. In this case, a periodic benchmark verification task should be performed to determine the level of response of the computing node to the computing task.
The invention provides a solution for the problem, and when the execution of a computing task is involved, the provided nodes comprise an edge side server and a cloud server according to the deployment position; the nodes are divided into verified nodes and unverified nodes according to whether verification is performed; the verified node is a computing node that has performed a computing task and has a response priority.
Further, the response priorities may include a high priority, a general priority, and a low priority.
In some embodiments, resource weights and priorities are calculated from the resources available to the node. For example, for a computing task, optimal computing hardware requirements and minimum computing hardware requirements may be included, with the computation being performed with the simplest factor, such as computing available resources in terms of available resources of the CPU and the number of cores, weighting the ratio of available resources to optimal computing hardware requirements, and selecting a priority based thereon.
Further, the resource scoring dimension is increased, node information of the recommended task execution node set is obtained, the weight M is set to be the sum of the scores of the CPU and the GPU according to the scoring information, the calculation is carried out according to the optimal calculation hardware requirement and the minimum calculation hardware requirement corresponding to each item, and the sum is carried out to obtain the score. After the score is obtained, the node with the highest score may be selected as the highest priority node to perform the computing task.
After the optimal execution node is selected, the execution node is utilized to execute the calculation task, the output of the execution node is returned to the scheduling node, and the scheduling node generates a control instruction according to the output result.
According to one embodiment of the invention, the meta-information of the computing task includes resource requirements, response requirements, computing type, and target application.
In one embodiment of the present invention, the characteristics of the computing task may include meta information meta info including task source, time, expected computing program version, and latency requirements.
The scheduling node may determine a type of required resources from meta-information of the computing task and determine a node to execute based on the determined type of resources.
The meta-information of the computing task provides a basis for rapid screening of the computing task, so the generation of the computing task should follow specifications to ensure that classification of the meta-information can be achieved by classification and combination of one or more meta-information.
The categorization of the computing tasks may be performed using a clustering algorithm, or using forms such as a bloom filter or a cuckoo filter.
If a cluster analysis method is used, the method described below can be used.
For a single node, acquiring meta-information and response scores of historical execution tasks, wherein the response scores are response time lengths for simplifying a model, and a longer response time length corresponds to a lower speed, and a preset weight matrix can be set for calibrating the response scores for avoiding errors caused by partial heavy-load tasks; if the tasks are divided more thoroughly, then the grouping can be done by workload, and calibration of the response scores can be omitted.
Acquiring index values of meta-information of a plurality of tasks, wherein the index values are obtained by classifying and scoring task sources, time, expected calculation program versions and time delay requirements in the meta-information according to preset rules; for example, according to RAM requirements, a score is respectively corresponding to 128M as a unit, and for example, a 1G memory is used as an example, the score is 8, and other quantifiable indexes can refer to the score to obtain index values.
Because resources are used when the task is executed, whether the resources are deployed locally or at the cloud, a timer can be set for the type of the resources to be used, and the timer is recorded in the execution process to obtain the response time of the resources corresponding to the meta-information of the execution task, and the response time of the specific calculation task is a millisecond value, so that the node can reflect the corresponding trend.
Based on the above, the index values of the meta-information and the correlation coefficient between the response scores are further obtained, and then the correlation is obtained by analyzing the relation between the change rule and the resource requirement in the task meta-information, so that the response of a node to a computing task is divided into a plurality of classes, the correlation can be analyzed by using k-average value cluster analysis, and the particle centers of one or more clusters can be obtained by the way, so that the response value of the node to the computing task can be rapidly analyzed to obtain the scores. The cluster analysis results for a node may be cached and queried and updated by the scheduling server.
When using a bloom filter or a cuckoo filter, response analysis is performed by looking for whether a node exists in the list. Because of the proximity of the two at the bottom layer, only bloom filters are used as examples.
The core implementation of the bloom filter comprises a binary number group and a hash function, wherein the hash function is used for calculating the position of each identifier in the binary number group, and in order to adapt to the calculation of cloud nodes and edge nodes, a plurality of hash functions are included in the bloom filter, and each hash function focuses on data of one dimension.
For example, when the region of the cloud resource is identified by a contracted string, the identification of the resource may be replaced by an identification data string, where the typical identification data is a string of numbers or a combination of numbers and letters, such as an area that may be indexed as an integer;
for example, for the memory resource, the available memory can be classified by using the size of the available memory, for example, the hash value of 0-128MB is 1, the hash value of 128-512MB is 2, the hash value of 512-1024MB is 4, and the hash value of 1024MB-2GB is 5, so that the memories can be distinguished in the above manner;
as for the CPU resource, it may be classified using the number of available cores, for example, 1 core has a hash value of 1,2 cores has a hash value of 2,3 cores has a hash value of 4,4 cores has a hash value of 5, and the number of available cores may be differentiated in the above manner;
such as type resources for the CPU, which may be distinguished using the type of CPU core, such as the CPU of the newer core may set a higher value, e.g., divided according to floating point number capability and integer computing capability;
other resources can be approximately mapped into the sequence according to the required quota or quantifiable resources through the example.
Therefore, after the requirement information data corresponding to the metadata of the task are combined into one character string according to the rule, the identification data are sequentially input into a plurality of hash functions according to the sequence, so that a plurality of hash values can be obtained, and the hash values are converted into binary sequences. I.e., the above procedure can be expressed as Hash (T) = { HashCPUResource (T), hashGPUResource (T), hashHDResource (T), hashNetworkResource (T).. HashRAMResource (T) }, i.e., extracting characteristics of CPU, GPU, disk, network..and memory, respectively, for meta-information of a computing task. The above features are merely examples, and if GPU is not involved, the corresponding hash function may be deleted, if NPU is involved, and the corresponding unit may be added.
In the actual application process, when metadata of a task is acquired and input into a bloom filter, the input data is already executed, namely when the performance meets the preset requirement, the metadata is divided into one or more bloom filters, when approximate calculation is executed subsequently, whether a resource node is suitable for executing the task is judged by judging the approximate task meta-information, the task can be converted according to the bloom filter included in the metadata, only the task source information is needed to be contained in one bloom filter by judging whether the approximate task source information is contained in one bloom filter, one or more scores can be obtained according to the data if the data is bound with one scoring interval for one bloom filter, and a lower value or an average value is taken as the score of the node for calculating the task.
Since the complexity is fixed in this process, no additional overhead is added.
In the rule determination, whether the high-performance bloom filter of one node comprises a calculation task is judged, so that a basis for screening the preferred execution node is provided for meta-information according to the calculation task.
The bloom filter may be provided with a plurality of bloom filters, for example, medium performance and high performance (for example, the response time is below 12ms and is high performance, the response time is between 8 ms and 100ms and is medium performance, so as to avoid performance change of the node caused by sporadic factors, namely, performance drift occurs, and the two may have overlapping response time intervals).
According to one embodiment of the invention, the set of computing nodes is obtained as follows:
selecting computing nodes meeting the resource limit configuration according to the types and the demands of the resources, sorting the computing nodes according to the proportion of the available resource limit, grouping the computing nodes according to the information of the network slice to which the nodes belong, and selecting cloud computing nodes and edge computing nodes according to the preset proportion.
The available resource values of the computing nodes, whether cloud nodes or edge nodes, should be updated or reported. For cloud nodes, the cloud nodes can be obtained through an API provided by a service provider, such as obtaining information of a CPU, a RAM, reading and writing, network flow and the like, and for deployed edge servers, an approximate effect can be achieved through a configured monitoring program, namely, scheduling nodes can screen all computing nodes according to resource types and requirements, and nodes with unsatisfied resources are firstly eliminated. In the filtering, if any dimension resource does not meet the requirement, the resource is excluded, for example, the required disk space is 1G, and if the residual space is less than 1G, the computing node is excluded.
The nodes are then ranked according to the ratio of available resources, where the ratio may be the sum of all available ratios, or its arithmetic mean, or the root mean square of the available ratios.
The sorting operation is simultaneously applied to the cloud computing nodes and the edge nodes, two ordered lists, namely a cloud computing node list and an edge computing node list, are obtained according to the available ratio, and 10% of nodes are respectively obtained from the cloud computing node list and the edge computing node list according to the preset ratio, for example, the ratio of 10%, and the partial nodes are the computing node sets.
To avoid a node in use being added to the set of computing nodes all the time, it is possible to determine whether the node is within the preferred executing node by setting a boolean value to determine meta-information corresponding to a computing task; if within the preferred execution node, no selection is made, otherwise a set of computing nodes may be selected.
Since there will always be some difference in the computing tasks, there will always be some unverified node for one computing task, and thus, the response level of the computing node to the specified task can be reconfirmed in the above manner.
Referring to fig. 3, according to one embodiment of the present invention, the preferred execution node obtains the following manner:
extracting the calculation task type contained in the meta-information of the task, selecting a calculation node sequence with a high grading level according to the calculation task type, and sequencing the calculation node sequence according to the proportion of available resources to obtain the calculation node with the highest proportion of available resources as the preferred execution node.
In the foregoing, it is explained that the relevance between the nodes and the computing tasks is divided according to the meta information of the computing tasks, obviously, when one computing task is allocated, all the computing nodes can be traversed, according to the extracted computing task type or directly according to the information of the computing tasks, more than one computing node is obtained based on the clustering analysis result of the computing nodes on the computing tasks, or according to the bloom filter or the cuckoo filter in the high-performance scoring interval, and then the computing nodes with the highest available resource proportion are obtained by sequencing the computing node sequence according to the available resource proportion as the preferred executing node.
The sorting operation is simultaneously applied to the selected high-performance cloud computing nodes and the edge nodes, two ordered lists are obtained according to the available ratio, namely a high-performance cloud computing node list and an edge computing node list, and 10% of nodes are respectively obtained from the high-performance cloud computing node list and the edge computing node list according to a preset proportion, for example, a proportion of 10%, and the partial nodes are the computing node sets.
Referring to fig. 4, according to one embodiment of the present invention, the preferred execution node obtains the following manner:
and extracting the calculation task types contained in the meta-information of the tasks, sequencing the calculation node sequences according to the proportion of the available resources, and obtaining the calculation node with the highest proportion of the available resources as the preferred execution node. This approach is applicable to cases where no priority is determined and will only be performed during the actual initialization process.
Referring to fig. 5, when a benchmark test task is executed, a callback monitor is constructed at a scheduling server, a callback layer of the benchmark test task is replaced with a port of the scheduling server monitor, a message is sent to the callback layer of the computing task according to an execution result of the benchmark test task, and a control instruction for an end side device is generated.
For a common computing task, corresponding data and an operation instruction after execution of the computing task are finished and are terminated after the operation instruction is sent out, however, the reference test task is involved, because the data needs to be calibrated, the corresponding node may execute the completion of the computation or cannot complete the computation, and therefore the computing task needs to be adjusted, namely, a callback layer of the reference test task is replaced by a port of a monitor of a scheduling server, a message is sent to the callback layer of the computing task according to the execution result of the reference test task, and a control instruction for the terminal side device is generated.
For example, the normal callback address is ServerA, namely, a port A port of the server A, and at the moment, a port B port which is a scheduling node is configured, and the port B is set to forward a message to the port A port of the server A after receiving the message, and a control instruction for the terminal side equipment is generated.
Based on the method, the scheduling of normal production instructions is not affected when the test task is completed.
Furthermore, to avoid delayed issuance of production instructions, a timeout period, such as 300ms, may be set after which a new computing task is started.
According to one embodiment of the invention, when the callback monitor receives the calculation results returned by the candidate calculation node and the preferred execution node, the callback monitor compares the candidate calculation node and the preferred execution node, and determines whether recalculation is needed according to the difference of the candidate calculation node and the preferred execution node; if the candidate computing node does not need to be recalculated, the response rating of the candidate computing node is updated according to the response time length of the candidate computing node.
The calculation results returned by the candidate calculation nodes and the preferred execution nodes may be inconsistent, which may be caused by depending resources or hardware, the candidate calculation nodes and the preferred execution nodes are compared, whether recalculation is needed is determined according to comparison of difference and threshold values of the candidate calculation nodes and the preferred execution nodes, and therefore erroneous scheduling instructions in subsequent production can be avoided.
The threshold should be determined based on the type of computing task.
According to one embodiment of the invention, the step of determining whether recalculation is required comprises:
when the response time length of the preferred execution node is greater than the response time length threshold value or the calculation time length is greater than the calculation time length threshold value, reselecting the preferred execution node to execute the calculation task and comparing the calculation task with the result of the candidate calculation node;
and when the response time length and the calculation time length are lower than the threshold value, selecting different cloud calculation nodes and edge calculation nodes from the calculation node set to execute the benchmark test task.
The response time and the calculation time of the preferred execution node are important bases for judging whether the candidate node is normal or not, and if the occurrence time is too high and the calculation time is too long and too large, the response time and the calculation time of the preferred execution node are possibly caused by insufficient hardware resources or network delay, so that the method has good significance in screening the candidate node.
When judging the difference of the two types of nodes, the response time length of the preferred execution node is larger than the response time length threshold, or the calculation time length is larger than the calculation time length threshold, which indicates that the configured hardware is not matched with the actual calculation power, the preferred execution node should be reselected to execute the calculation task, and the calculation task is compared with the result of the candidate calculation node;
when the response time length of the preferred execution node is not greater than the response time length threshold value and the calculation time length is not greater than the calculation time length threshold value, judging that the preferred execution node is normal at the moment, realizing the designed calculation force, and selecting different cloud calculation nodes and edge calculation nodes in the calculation node set to execute the reference test task.
By this, erroneous instruction issues and impacts the normal production of the plant can be avoided and the computing nodes can be reselected.
According to one embodiment of the invention, when the edge side server meets the requirement, adding the edge side server into a high priority node corresponding to the calculation type; and when the cloud server meets the requirements, adding the cloud testing server into a common priority node corresponding to the calculation type.
When the edge side server meets the requirement, the edge side server is added into the high priority node corresponding to the calculation type instead of the cloud server, so that the cost is effectively reduced.
Further, if the calculation result satisfies the demand, but the response speed is lower than the expected value, it may be added to the low priority node, and the low priority node may not perform the calculation task and solve the problem of slow response by means of diagnosis.
Here, a minimum number of high priority nodes should be set to avoid a mismatch in computational tasks and computational power, and to avoid low priority nodes.
According to one embodiment of the invention, a computing node is marked as an unverified node when the number of times it performs a computing task exceeds a threshold.
The calculation task construction benchmark task is not always required to be executed, and is executed only when a large deviation occurs or a deviation is likely to occur, and for such a situation, judgment can be obtained by analyzing and judging historical abnormal information, or by setting a response threshold, if the moving average value of the response time of one or more calculation tasks is higher than the threshold, the benchmark task is executed, and at this time, the benchmark task can be executed when selected by marking the calculation node as an unverified node.
The benchmarking tasks may be determined for one or more computing tasks, i.e., the unverified node is a node for which the meta-information priority for the computing tasks is not determined.
In another embodiment of the present invention, an industrial internet of things dispatching system based on cloud edge cooperative technology suitable for the above method is also disclosed, the system includes:
the preferred execution node acquisition unit is used for acquiring preferred execution nodes of the computing task according to the meta information of the computing task;
the computing node set acquisition unit is used for acquiring a computing node set which has approximate response with the preferred executing node in the unverified node according to the expected response value of the preferred executing node to the computing task;
the basic test task generating unit is used for constructing a basic test task based on the calculation task, and at least selecting one candidate calculation node and a preferred execution node in the calculation node set to execute the basic test task;
and the scheduling control unit is used for updating the priority of the computing node for the computing task according to the execution results of the candidate computing node and the preferred executing node selected by the reference test task and generating a control instruction of an application layer.
Those of ordinary skill in the art will appreciate that implementing all or part of the above-described method embodiments may be accomplished by a computer program to instruct related hardware, the program may be stored in a computer readable storage medium, and the program may include the above-described method embodiments when executed. And the aforementioned storage medium includes: ROM or random access memory RAM, magnetic or optical disk, etc. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More detailed examples of machine-readable storage media include an electrical connection with one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.
In an exemplary embodiment, a non-transitory computer readable storage medium including instructions, such as a memory including instructions, executable by a processor of a server to perform the method of emoticon recommendation shown in various embodiments of the invention is also provided. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
Claims (10)
1. The industrial Internet of things scheduling method based on cloud edge cooperative technology is characterized by comprising the following steps of:
acquiring a preferred execution node of the computing task according to meta information of the computing task;
acquiring a computing node set which has approximate response to the preferred executing node in the unverified node according to the expected response value of the preferred executing node to the computing task;
constructing a benchmark test task based on the calculation tasks, and at least selecting one candidate calculation node and a preferred execution node in the calculation node set to execute the benchmark test task;
and updating the priority of the computing node for the computing task according to the execution results of the candidate computing node and the preferred executing node selected by the reference test task, and generating a control instruction of an application layer.
2. The industrial internet of things scheduling method based on cloud computing technology as set forth in claim 1, wherein the meta information of the computing task includes resource requirements, response requirements, computing type and target application.
3. The industrial internet of things scheduling method based on cloud edge cooperative technology as set forth in claim 1, wherein the computing node set is obtained as follows:
selecting computing nodes meeting the resource limit configuration according to the types and the demands of the resources, sorting the computing nodes according to the proportion of the available resource limit, grouping the computing nodes according to the information of the network slice to which the nodes belong, and selecting cloud computing nodes and edge computing nodes according to the preset proportion.
4. The industrial internet of things scheduling method based on cloud edge cooperative technology as set forth in claim 1, wherein the preferred execution node obtains the following manner:
extracting the calculation task type contained in the meta-information of the task, selecting a calculation node sequence with a high grading level according to the calculation task type, and sequencing the calculation node sequence according to the proportion of available resources to obtain the calculation node with the highest proportion of available resources as the preferred execution node.
5. The industrial internet of things scheduling method based on cloud edge cooperative technology as set forth in claim 1, wherein when the benchmark test task is executed, a callback monitor is built in the scheduling server, a callback layer of the benchmark test task is replaced with a port of the scheduling server monitor, a message is sent to the callback layer of the computing task according to an execution result of the benchmark test task, and a control instruction for the terminal side device is generated.
6. The industrial internet of things scheduling method based on cloud edge cooperative technology as set forth in claim 5, wherein the callback listener compares the candidate computing node and the preferred executing node when receiving the computing results returned by the candidate computing node and the preferred executing node, and determines whether recalculation is needed according to the difference of the candidate computing node and the preferred executing node; if the candidate computing node does not need to be recalculated, the response rating of the candidate computing node is updated according to the response time length of the candidate computing node.
7. The method for scheduling industrial internet of things based on cloud computing technology as recited in claim 6, wherein the step of determining whether recalculation is required comprises:
judging whether the response time length and the calculation time length of the preferred execution node are respectively larger than a corresponding response time length threshold value and a calculation time length threshold value, and reselecting the preferred execution node to execute a calculation task and comparing the result with the candidate calculation node when at least one of the response time length and the calculation time length is higher than the threshold value;
and when the response time length and the calculation time length are lower than the threshold value, selecting different cloud calculation nodes and edge calculation nodes from the calculation node set to execute the benchmark test task.
8. The industrial internet of things scheduling method based on cloud edge cooperative technology as set forth in claim 5, wherein when the edge side server meets the requirement, adding the edge side server into a high priority node corresponding to the calculation type; and when the cloud server meets the requirements, adding the cloud testing server into a common priority node corresponding to the calculation type.
9. The industrial internet of things scheduling method based on cloud computing technology of claim 1, wherein a computing node is marked as an unverified node when the number of times the computing task is performed by the computing node exceeds a threshold.
10. An industrial internet of things scheduling system based on cloud edge cooperative technology, which is characterized by comprising:
the preferred execution node acquisition unit is used for acquiring preferred execution nodes of the computing task according to the meta information of the computing task;
the computing node set acquisition unit is used for acquiring a computing node set which has approximate response with the preferred executing node in the unverified node according to the expected response value of the preferred executing node to the computing task;
the basic test task generating unit is used for constructing a basic test task based on the calculation task, and at least selecting one candidate calculation node and a preferred execution node in the calculation node set to execute the basic test task;
and the scheduling control unit is used for updating the priority of the computing node for the computing task according to the execution results of the candidate computing node and the preferred executing node selected by the reference test task and generating a control instruction of an application layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310808763.8A CN117176728B (en) | 2023-07-04 | 2023-07-04 | Industrial Internet of things dispatching method and dispatching system based on cloud edge cooperative technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310808763.8A CN117176728B (en) | 2023-07-04 | 2023-07-04 | Industrial Internet of things dispatching method and dispatching system based on cloud edge cooperative technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117176728A true CN117176728A (en) | 2023-12-05 |
| CN117176728B CN117176728B (en) | 2024-07-05 |
Family
ID=88943816
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310808763.8A Active CN117176728B (en) | 2023-07-04 | 2023-07-04 | Industrial Internet of things dispatching method and dispatching system based on cloud edge cooperative technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117176728B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111144715A (en) * | 2019-12-11 | 2020-05-12 | 重庆邮电大学 | Factory electric energy management and control system and method based on edge cloud cooperation |
| CN112148449A (en) * | 2020-09-22 | 2020-12-29 | 行星算力(深圳)科技有限公司 | Local area network scheduling algorithm and system based on edge calculation |
| CN112799789A (en) * | 2021-03-22 | 2021-05-14 | 腾讯科技（深圳）有限公司 | Node cluster management method, device, equipment and storage medium |
| US20210409335A1 (en) * | 2020-09-11 | 2021-12-30 | Intel Corporation | Multi-access management service packet classification and prioritization techniques |
| CN114217920A (en) * | 2021-11-16 | 2022-03-22 | 曙光信息产业(北京)有限公司 | Job scheduling method and device, computer cluster and computer readable storage medium |
| CN114741165A (en) * | 2022-03-02 | 2022-07-12 | 浙江大华技术股份有限公司 | Processing method of data processing platform, computer equipment and storage device |
| TW202234236A (en) * | 2020-12-24 | 2022-09-01 | 美商英特爾公司 | Methods, systems, articles of manufacture and apparatus to optimize resources in edge networks |
| CN116360954A (en) * | 2023-05-31 | 2023-06-30 | 北京百星电子***有限公司 | Industrial Internet of things management and control method and system based on cloud edge cooperative technology |
-
2023
- 2023-07-04 CN CN202310808763.8A patent/CN117176728B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111144715A (en) * | 2019-12-11 | 2020-05-12 | 重庆邮电大学 | Factory electric energy management and control system and method based on edge cloud cooperation |
| US20210409335A1 (en) * | 2020-09-11 | 2021-12-30 | Intel Corporation | Multi-access management service packet classification and prioritization techniques |
| CN112148449A (en) * | 2020-09-22 | 2020-12-29 | 行星算力(深圳)科技有限公司 | Local area network scheduling algorithm and system based on edge calculation |
| TW202234236A (en) * | 2020-12-24 | 2022-09-01 | 美商英特爾公司 | Methods, systems, articles of manufacture and apparatus to optimize resources in edge networks |
| CN112799789A (en) * | 2021-03-22 | 2021-05-14 | 腾讯科技（深圳）有限公司 | Node cluster management method, device, equipment and storage medium |
| CN114217920A (en) * | 2021-11-16 | 2022-03-22 | 曙光信息产业(北京)有限公司 | Job scheduling method and device, computer cluster and computer readable storage medium |
| CN114741165A (en) * | 2022-03-02 | 2022-07-12 | 浙江大华技术股份有限公司 | Processing method of data processing platform, computer equipment and storage device |
| CN116360954A (en) * | 2023-05-31 | 2023-06-30 | 北京百星电子***有限公司 | Industrial Internet of things management and control method and system based on cloud edge cooperative technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117176728B (en) | 2024-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110798517B (en) | Decentralized cluster load balancing method and system, mobile terminal and storage medium | |
| CN111935223B (en) | Internet of things equipment processing method based on 5G and cloud computing center | |
| WO2022001918A1 (en) | Method and apparatus for building predictive model, computing device, and storage medium | |
| CN112506659A (en) | Management method of edge device and related device | |
| CN116360954B (en) | Industrial Internet of things management and control method and system based on cloud edge cooperative technology | |
| US11663172B2 (en) | Cascading payload replication | |
| CN110417903A (en) | A kind of information processing method and system based on cloud computing | |
| CN114091610A (en) | Intelligent decision method and device | |
| CN112764901A (en) | Service data processing method and system based on cloud network fusion and artificial intelligence | |
| CN113553138A (en) | Cloud resource scheduling method and device | |
| CN113934512A (en) | Load balancing implementation method, device, equipment and storage medium | |
| CN112925634A (en) | Heterogeneous resource scheduling method and system | |
| CN111090401B (en) | Storage device performance prediction method and device | |
| CN117176728B (en) | Industrial Internet of things dispatching method and dispatching system based on cloud edge cooperative technology | |
| CN112559525A (en) | Data checking system, method, device and server | |
| CN114172910B (en) | Method, device, equipment and medium for dynamically allocating load based on memory management | |
| Vega et al. | Sharing hardware resources in heterogeneous computer-supported collaboration scenarios | |
| CN106888237B (en) | Data scheduling method and system | |
| CN114281549A (en) | Data processing method and device | |
| CN110177006B (en) | Node testing method and device based on interface prediction model | |
| CN113297358A (en) | Data processing method, device, server and computer readable storage medium | |
| CN114375565A (en) | File block downloading method and device | |
| CN115396319B (en) | Data stream slicing method, device, equipment and storage medium | |
| CN117170985B (en) | Distributed monitoring method and system for open geographic information network service | |
| WO2017213065A1 (en) | Service management system, service management method, and recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240614 Address after: Room 804, No. 56 Feicui Oasis Forest Peninsula Street, Xintang Town, Zengcheng District, Guangzhou City, Guangdong Province, 511300 Applicant after: Zhong Guobiao Country or region after: China Address before: Room 101102 Building D3, Bureau of Military Affairs, No. 10, Jiachuang Road, Tongzhou Park Optical Mechanical Electrical Integration Industrial Base, Zhongguancun Science Park, Tongzhou District, Beijing Applicant before: Beijing Baixing Electronic System Co.,Ltd. Country or region before: China |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |