CN115878286A - Task execution method based on multi-terminal interaction - Google Patents
Task execution method based on multi-terminal interaction Download PDFInfo
- Publication number
- CN115878286A CN115878286A CN202211500608.1A CN202211500608A CN115878286A CN 115878286 A CN115878286 A CN 115878286A CN 202211500608 A CN202211500608 A CN 202211500608A CN 115878286 A CN115878286 A CN 115878286A
- Authority
- CN
- China
- Prior art keywords
- task
- computing nodes
- network server
- data
- subtask
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Multi Processors (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of intelligent scheduling, in particular to a task execution method based on multi-terminal interaction, which comprises the following steps: the mobile terminal uploads data and sends a data analysis request, the network server side analyzes task demand after receiving the uploaded data and a request task, whether the residual computing nodes meet the task demand is analyzed according to the task demand, if yes, the residual computing nodes are distributed to form a computing node set, the computing node set divides a working area after receiving a subtask unit, computing nodes in the working area send analysis results to the network server side after completing subtasks, and the network server side sends the analysis results to a database; after a user sends a data downloading request, a network server side sends competition instructions to the remaining computing nodes, main computing nodes and slave computing nodes are set according to competition instruction results, the main computing nodes are responsible for dividing subtask units and integrating data fragments, and the slave computing nodes extract the data fragments from a database according to subtasks.
Description
Technical Field
The invention relates to the technical field of intelligent scheduling, in particular to a task execution method based on multi-terminal interaction.
Background
With the continuous development of internet technology and the continuous innovation of big data and related intelligent technologies, the requirement of people on the speed of downloading and uploading data in daily life is higher and higher, although the traditional data transmission also adopts multithread processing, a plurality of tasks are still directly processed in parallel without screening and processing, and the process breakdown caused by the calculation error of one task often occurs, which causes great memory and time waste.
The current common method for processing a large amount of data is to adopt a computer group to perform calculation processing, each computer is uniformly managed by a management center, and a large number of computers are required to be called simultaneously due to large tasks, so that the load of the management center is increased rapidly, and errors are easy to occur.
Disclosure of Invention
In view of the above problems in the prior art, an object of the present invention is to provide a task execution method based on multi-port interaction, which enables each computer in a computer group to execute tasks in order according to a working area.
The task execution method based on the multi-terminal interaction comprises the following steps:
the network server marks the computing nodes in the computer group, and marks one computer as one computing node;
the network server side obtains the size of the uploaded data and calculates task request quantity, and whether the requested task is executed or not is judged according to the task request quantity and the comparison result of the rest calculation nodes;
the network server divides the request task into subtask units, and divides a working area according to the type of the subtask units;
splitting the subtask unit into subtasks in the working area, and distributing the subtasks to the computing nodes in the working area;
after receiving the data downloading request task, the network server sends competition instructions to the selected remaining computing nodes, sets a master computing node and a slave computing node according to competition results, and distributes subtask units to the master computing node;
and the main computing node divides the received subtask unit into a plurality of subtasks, sends the subtasks to the slave computing node, and simultaneously extracts the required data segments from the database according to the requirements of the subtasks.
Furthermore, the network server marks computers in the computer group, and one computer is taken as a computing node;
before receiving the request task, the network server side counts the computing nodes which are not distributed with the task, defines the computing nodes as the residual computing nodes and counts the number of the residual computing nodes.
Further, the network server calculates the task request amount by acquiring the length of the uploaded data;
comparing the task request quantity with the rest computing nodes, and if the task request quantity is less than or equal to the rest computing nodes, forming the rest computing nodes required by the task request quantity into a computing node set for completing the request task of uploading data;
and if the task request quantity is larger than the rest computing nodes, the requested task is moved to a task waiting area.
Furthermore, the network server divides the request task into one or more sub-task units according to the uploaded data, and divides one or more working areas in the computing node set according to the categories of the sub-task units, wherein the sub-task units of the same category are orderly arranged in one working area.
Further, after receiving the subtask unit, the working area divides the subtask unit into a plurality of subtasks and distributes the subtasks to the computing nodes in the working area, and all the computing nodes are provided with task process report instructions.
Further, tasks in the same type of subtask unit are divided into: the tasks are completely interfered with each other, and the tasks are partially interfered with each other and are not interfered with each other;
complete mutual intervention tasks: the tasks distributed by the plurality of computing nodes are completely interfered with each other, and the computing nodes sequentially process according to the priority of the distributed tasks in the subtask units;
the tasks partially interfere with each other: the task parts distributed by a plurality of computing nodes interfere with each other, then the computing nodes sequentially process according to the priority of the distributed tasks in the subtask units, and the tasks in the same priority simultaneously process;
tasks that do not interfere with each other: and if the tasks distributed by the plurality of computing nodes do not interfere with each other, the computing nodes process the distributed tasks simultaneously according to the arrangement.
Further, after receiving the data downloading request, the network server sends a competition instruction to each working area, the computing nodes receiving the competition instruction simultaneously send a key value to the network server, the key value of each working area is not repeated, the network server sets the computing nodes with specific key values as main computing nodes and sends subtask units, and the rest computing nodes are set as slave computing nodes.
Furthermore, the main computing node divides the received subtask unit into a plurality of subtasks and sends the subtasks to the secondary computing node, and the secondary computing node extracts the required data segment from the database according to the task requirement; after the slave computing node extracts the data segments required by the tasks, the extraction result is sent to the master computing node, and the master computing node receives the data segments and then distributes the subtasks to the slave computing nodes again until no subtasks exist;
and after all the subtasks are completed, the main computing node integrates all the data segments to form complete data, and the integrated data is sent to the network server.
And when the network server side detects that all the subtask units are finished, the integrated data sent by each working area are sorted and compressed, and the integrated data are uploaded to a database or sent to a request side according to the type of the request task.
Compared with the prior art, the invention has the following beneficial technical effects:
through the division of the working area, a task reporting program is set, so that the progress blockage caused by the cross scheduling of tasks is effectively avoided, and the time utilization rate when a computer is scheduled is maximally improved; for the computing nodes which finish the subtasks, the network server side immediately dispatches the subtasks of the same type to the working nodes after receiving the analysis data, and for the working areas which finish the subtask units, the network server dispatches the subtask units or disperses the working areas, thereby realizing the scheduling optimization;
a competition instruction mode is adopted when data are downloaded from a database, a network server side designates a host computer and a slave computer through competition results, the host computer is responsible for task scheduling and data segment integration, and the slave computer extracts data segments from the database, so that data extraction and integration can be rapidly completed, labor division is clear, and data extraction efficiency is improved.
Drawings
FIG. 1 is a schematic diagram of the basic structure of the present invention;
FIG. 2 is a flow chart of an embodiment of uploading data;
FIG. 3 is a flowchart of downloading data according to a second embodiment.
Detailed Description
In order to make the purpose and technical scheme of the embodiment of the invention clearer, the technical scheme of the invention is clearly and completely described below by combining the attached drawings. As in fig. 1, is basically applicable in the embodiment of the present invention.
The technical solution of the present invention is further explained with reference to the accompanying drawings and specific embodiments.
The first embodiment.
As shown in fig. 1 and fig. 2, the present embodiment provides a task execution method based on multi-end interaction, where the method includes:
the network server marks computers in the computer group, one computer is recorded as a computing node, before receiving a request task, the network server counts computing nodes which are not distributed with tasks, the computing nodes are defined as residual computing nodes, the computing nodes can be called at any time, the number of the residual computing nodes is counted and recorded as n, and n is an integer;
the mobile terminal collects and uploads data to the network server terminal and sends a data analysis task request;
further, the network server calculates a task request amount W by acquiring the length X of the uploaded data, compares the task request amount W with the remaining computing nodes n, and if the task request amount W is less than or equal to the remaining computing nodes n, combines the remaining computing nodes required by the task request amount into a computing node set for completing related tasks of the data; if the task request amount W is larger than the rest of the computing nodes n, the requested task is moved to a task waiting area, wherein the relationship between the task request amount W and the data length X is as follows: w = a X, wherein a is smaller than 1 and is dynamically acquired by a network server by learning a BP neural network according to the data length;
the user uploads the collected data to the network server through the mobile terminal and sends a data analysis request, after the network server receives the data and the data analysis request, the length of the uploaded data is calculated, then a BP neural network is used for learning to dynamically obtain a coefficient alpha, and the task request amount W = alphaX is calculated; if W is less than or equal to n, forming a computing node set according to the rest computing nodes required by the task request quantity, and if W is greater than n, adjusting the request task to a task waiting area until the rest computing nodes reach the requirement.
The network server divides the analysis data task into one or more subtask units aiming at the uploaded data, and divides one or more working areas in the computing node set according to the types of the subtask units, wherein the subtask units of the same type are orderly arranged in one working area;
the tasks in the same type of subtask unit are divided into: the tasks are completely interfered with each other, and the tasks are partially interfered with each other and are not interfered with each other;
and further, after receiving the subtask unit, the working area divides the subtask unit into a plurality of subtasks and distributes the subtasks to the computing nodes in the working area, all the computing nodes are provided with task process report instructions, and for the computing nodes which complete the subtasks, the network server marks the computing nodes as task-free computing nodes and dispatches the uncompleted subtasks to the computing nodes until the working area completes the distributed subtask unit.
Three cases of the same type of task are illustrated below with reference to examples:
the tasks are completely interfered with each other: the tasks distributed by the plurality of computing nodes are completely interfered with each other, and the computing nodes sequentially process according to the priority of the distributed tasks in the subtask units; if the tasks allocated by the two computing nodes a and B completely interfere with each other, if a certain subtask of the subtask unit T can be split into T1 and T2, and T1 and T2 correspond to two successive steps in the subtask unit T, the computing node B can start to execute T2 after the computing node a executes T1;
the tasks partially interfere with each other: the task parts distributed by a plurality of computing nodes interfere with each other, then the computing nodes sequentially process according to the priority of the distributed tasks in the subtask units, and the tasks in the same priority simultaneously process; if the task parts allocated by the two computing nodes a and B interfere with each other, for example, a certain subtask of the subtask unit T can be split into T1, T2, T3, and T4, where T1 and T3 can be performed simultaneously, T2 and T4 can be performed simultaneously, and T2 and T4 are located after the steps of T1 and T3 in the subtask unit T, the computing node a and the computing node B execute T1 and T3 first, and then execute T2 and T4;
do not interfere with each other: if the tasks distributed by the two computing nodes A and B do not interfere with each other, the computing nodes A and B directly and simultaneously execute the distributed tasks.
After all the distributed subtask units in the working area are completed, the network server defines the working area as a task-free area, which indicates that the working area is in an idle state; for the working area in the idle state, the network server side dispatches the unallocated subtask units to the working area, and if no unallocated subtask unit exists at the moment, the network server side dismisses the working area in the idle state and records the computing nodes in the working area as the residual computing nodes;
and when all the analysis processing of the subtask units is completed, the network server side integrates and transmits the analyzed data to the database.
The method can definitely plan the corresponding computing nodes and the tasks to be completed, improves the utilization rate of the computing nodes to the maximum extent, and meanwhile enables task allocation to be intelligent.
The second embodiment.
As shown in fig. 1 and fig. 3, corresponding to the first embodiment, this embodiment provides a task execution method based on multi-end interaction, where the method includes:
the computing node group comprises a plurality of computing nodes, each computing node is connected with the network server, when the network server receives a data downloading request task, the task request quantity of the request task is computed, the residual computing nodes are counted, the size relation between the two computing nodes is compared, and whether the residual computing nodes are enough to complete the request task is judged according to the comparison result;
if the network server judges that the rest computing nodes are enough to complete the request task, the rest computing nodes required by the task request amount form a computing node set, the request task is divided into a plurality of subtask units, and the computing node set is divided into one or a plurality of working areas according to the category of the subtask units;
further, a competition instruction is sent to the computing nodes in each working area, each computing node sends a key value with a mark to a network server after receiving the competition instruction, for example, the key value "Select _ HKey + n", where n is a random integer greater than 0, and n in each working area is not repeated, the network server sets the computing node with the key value "Select _ HKey +1" as a master computing node and sends a subtask unit, and the rest of the computing nodes are set as slave computing nodes;
the main computing node divides the received subtask unit into a plurality of subtasks and sends the subtasks to the slave computing node, and the slave computing node extracts required data segments from the database according to task requirements; after the slave computing node extracts the data segments required by the tasks, the extraction result is sent to the master computing node, and the master computing node receives the data segments and then distributes the subtasks to the slave computing nodes again until no subtasks exist;
specifically, after the subtasks are completely completed, the main computing node integrates all data fragments to form complete data, and sends the integrated data to the network server; and when the network server side detects that all the subtask units are finished, the integrated data sent by the main computing nodes of each working area are sorted, compressed and sent to the mobile side.
Exemplarily, a user uses a mobile terminal to send a data downloading request task, a network server receives the data downloading request task, allocates a working area and sends a competition instruction to a computing node in the working area, and a master computing node and a slave computing node are designated according to a competition result;
the network server side sends the subtask units to the master computing node, the master computing node divides the subtask units into a plurality of subtasks and sends the subtasks to the slave computing nodes, and the slave computing nodes extract data from the database in a segmented manner according to the subtask requirements; assuming that there are 4 slave computing nodes in the work area, namely slave computing node 1, slave computing node 2, slave computing node 3 and slave computing node 4, to extract a segment of number "12345678", then it will simultaneously extract "12", "34", "56" and "78" from the database, and the extraction result will be transmitted to the master computing node, which will integrate the data segments into "12345678"; after the work area finishes all the distributed subtask units, the main computing node integrates all the data and sends the data to the network server;
and when the network server side detects that all the subtask units are completed, the integrated data sent by each working area is sorted and compressed, and the integrated data is sent to the user mobile side.
Although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the spirit and scope of the present invention.
Claims (9)
1. The task execution method based on the multi-terminal interaction is characterized by comprising the following steps of:
the network server marks the computing nodes in the computer group, and marks one computer as one computing node;
the network server side obtains the size of the uploaded data and calculates task request quantity, and whether the requested task is executed or not is judged according to the task request quantity and the comparison result of the rest calculation nodes;
the network server divides the request task into subtask units, and divides a working area according to the types of the subtask units;
splitting the subtask unit into subtasks in the working area, and distributing the subtasks to the computing nodes in the working area;
after receiving the data downloading request task, the network server sends a competition instruction to the selected remaining computing nodes, sets a master computing node and a slave computing node according to a competition result, and distributes a subtask unit to the master computing node;
and the main computing node divides the received subtask unit into a plurality of subtasks, sends the subtasks to the slave computing node, and simultaneously extracts the required data segments from the database according to the requirements of the subtasks.
2. The method for executing task based on multi-port interaction according to claim 1, wherein the network server marks computers in the computer group, and marks one computer as a computing node;
before receiving the request task, the network server side counts the computing nodes which are not distributed with the task, defines the computing nodes as residual computing nodes and counts the number of the residual computing nodes.
3. The task execution method based on the multi-terminal interaction according to claim 2, wherein the network server calculates the task request amount by acquiring the length of the uploaded data;
comparing the task request quantity with the rest computing nodes, and if the task request quantity is less than or equal to the rest computing nodes, forming the rest computing nodes required by the task request quantity into a computing node set for completing a request task of uploading data;
and if the task request quantity is larger than the rest computing nodes, the requested task is moved to a task waiting area.
4. The method according to claim 3, wherein the network server divides the request task into one or more subtask units according to the uploaded data, and divides one or more work areas in the set of computing nodes according to the category of the subtask unit, wherein the subtask units of the same category are arranged in an ordered manner in one work area.
5. The method for executing tasks based on multi-port interaction according to claim 4, wherein after receiving the subtask unit, the working area divides the subtask unit into a plurality of subtasks and distributes the subtasks to the computing nodes in the working area, and all the computing nodes are provided with task progress reporting programs.
6. The method according to claim 5, wherein the tasks in the same type of subtask unit are divided into: the tasks are completely interfered with each other, and the tasks are partially interfered with each other and are not interfered with each other;
the tasks are completely interfered with each other: if the tasks distributed by the plurality of computing nodes completely interfere with each other, the computing nodes sequentially process according to the priority of the distributed tasks in the subtask units;
the tasks partially interfere with each other: the task parts distributed by a plurality of computing nodes interfere with each other, then the computing nodes sequentially process according to the priority of the distributed tasks in the subtask units, and the tasks in the same priority simultaneously process;
tasks that do not interfere with each other: and if the tasks distributed by the plurality of computing nodes do not interfere with each other, the computing nodes process the distributed tasks simultaneously according to the arrangement.
7. The method of claim 3, wherein after receiving the request for downloading data, the network server sends a contention instruction to each working area, and the computing node that receives the contention instruction sends a key value to the network server, and the key value of each working area is not repeated, and the network server sets the computing node with a specific key value as a primary computing node and sends a subtask unit, and sets the remaining computing nodes as secondary computing nodes.
8. The task execution method based on multi-end interaction of claim 7, wherein the master computing node divides the received subtask unit into a plurality of subtasks to send to the slave computing node, and the slave computing node extracts the required data segments from the database according to the task requirements; after the slave computing node extracts the data segments required by the tasks, the extraction result is sent to the master computing node, and the master computing node receives the data segments and distributes the subtasks to the slave computing nodes again until no subtasks exist;
and after the subtasks are completely finished, the main computing node integrates all the data fragments to form complete data and sends the integrated data to the network server.
9. The task execution method based on multi-end interaction as claimed in claim 8, wherein when the network service end detects that the subtask units are all completed, the integrated data sent by each working area is sorted and compressed, and the integrated data is uploaded to a database or sent to a request end according to the type of the requested task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211500608.1A CN115878286A (en) | 2022-11-28 | 2022-11-28 | Task execution method based on multi-terminal interaction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211500608.1A CN115878286A (en) | 2022-11-28 | 2022-11-28 | Task execution method based on multi-terminal interaction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115878286A true CN115878286A (en) | 2023-03-31 |
Family
ID=85764279
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211500608.1A Pending CN115878286A (en) | 2022-11-28 | 2022-11-28 | Task execution method based on multi-terminal interaction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115878286A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101986272A (en) * | 2010-11-05 | 2011-03-16 | 北京大学 | Task scheduling method under cloud computing environment |
| US20140108861A1 (en) * | 2012-10-15 | 2014-04-17 | Hadapt, Inc. | Systems and methods for fault tolerant, adaptive execution of arbitrary queries at low latency |
| CN106126323A (en) * | 2016-06-17 | 2016-11-16 | 四川新环佳科技发展有限公司 | Real-time task scheduling method based on cloud platform |
| CN109857563A (en) * | 2019-02-18 | 2019-06-07 | 重庆西部汽车试验场管理有限公司 | Task executing method, device and task execution system |
| CN115167992A (en) * | 2022-05-23 | 2022-10-11 | 曙光信息产业(北京)有限公司 | Task processing method, system, device, server, medium, and program product |
| CN115271556A (en) * | 2022-09-27 | 2022-11-01 | 江西萤火虫微电子科技有限公司 | Robot task scheduling method and device, readable storage medium and electronic equipment |
| CN115292016A (en) * | 2022-08-09 | 2022-11-04 | 中国平安财产保险股份有限公司 | Task scheduling method based on artificial intelligence and related equipment |
| CN115334084A (en) * | 2022-08-18 | 2022-11-11 | 陈水兰 | Cloud platform based on cloud computing and internet |
-
2022
- 2022-11-28 CN CN202211500608.1A patent/CN115878286A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101986272A (en) * | 2010-11-05 | 2011-03-16 | 北京大学 | Task scheduling method under cloud computing environment |
| US20140108861A1 (en) * | 2012-10-15 | 2014-04-17 | Hadapt, Inc. | Systems and methods for fault tolerant, adaptive execution of arbitrary queries at low latency |
| CN106126323A (en) * | 2016-06-17 | 2016-11-16 | 四川新环佳科技发展有限公司 | Real-time task scheduling method based on cloud platform |
| CN109857563A (en) * | 2019-02-18 | 2019-06-07 | 重庆西部汽车试验场管理有限公司 | Task executing method, device and task execution system |
| CN115167992A (en) * | 2022-05-23 | 2022-10-11 | 曙光信息产业(北京)有限公司 | Task processing method, system, device, server, medium, and program product |
| CN115292016A (en) * | 2022-08-09 | 2022-11-04 | 中国平安财产保险股份有限公司 | Task scheduling method based on artificial intelligence and related equipment |
| CN115334084A (en) * | 2022-08-18 | 2022-11-11 | 陈水兰 | Cloud platform based on cloud computing and internet |
| CN115271556A (en) * | 2022-09-27 | 2022-11-01 | 江西萤火虫微电子科技有限公司 | Robot task scheduling method and device, readable storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI426393B (en) | Cloud-computating resource scheduling method and system applying the same | |
| US7689996B2 (en) | Method to distribute programs using remote Java objects | |
| WO2016078008A1 (en) | Method and apparatus for scheduling data flow task | |
| US8224938B2 (en) | Data processing system and method for iteratively re-distributing objects across all or a minimum number of processing units | |
| CN111108480B (en) | System and method for distributed resource demand and distribution | |
| CN105487930A (en) | Task optimization scheduling method based on Hadoop | |
| CN103761146B (en) | A kind of method that MapReduce dynamically sets slots quantity | |
| CN111381950A (en) | Task scheduling method and system based on multiple copies for edge computing environment | |
| CN108268319A (en) | Method for scheduling task, apparatus and system | |
| CN111459641B (en) | Method and device for task scheduling and task processing across machine room | |
| CN103699433B (en) | One kind dynamically adjusts number of tasks purpose method and system in Hadoop platform | |
| CN112882828B (en) | Method for managing and scheduling a processor in a processor-based SLURM operation scheduling system | |
| CN112988360B (en) | Task distribution system based on big data analysis | |
| CN111258746A (en) | Resource allocation method and service equipment | |
| CN105488134A (en) | Big data processing method and big data processing device | |
| CN110084507B (en) | Scientific workflow scheduling optimization method based on hierarchical perception in cloud computing environment | |
| CN110519386B (en) | Elastic resource supply method and device based on data clustering in cloud environment | |
| CN115878286A (en) | Task execution method based on multi-terminal interaction | |
| CN114860449B (en) | Data processing method, device, equipment and storage medium | |
| CN102904952B (en) | Self-adapting system and method for efficiently processing input of mass data to database | |
| CN116010447A (en) | Load balancing method and device for optimizing heterogeneous database user query | |
| CN111628943B (en) | Intelligent Internet of things method based on communication and perception technology | |
| CN114356515A (en) | Scheduling method of data conversion task | |
| CN113778690A (en) | Task allocation method, device, equipment and storage medium | |
| CN112597173A (en) | Distributed database cluster system peer-to-peer processing system and processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |