CN114679447B - Target range flow task slice scheduling system and method - Google Patents
Target range flow task slice scheduling system and method Download PDFInfo
- Publication number
- CN114679447B CN114679447B CN202210596165.4A CN202210596165A CN114679447B CN 114679447 B CN114679447 B CN 114679447B CN 202210596165 A CN202210596165 A CN 202210596165A CN 114679447 B CN114679447 B CN 114679447B
- Authority
- CN
- China
- Prior art keywords
- flow
- task
- tasks
- actuator
- traffic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 19
- 238000003860 storage Methods 0.000 claims abstract description 16
- 238000012544 monitoring process Methods 0.000 claims abstract description 4
- 230000006378 damage Effects 0.000 claims description 23
- 238000001514 detection method Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000009826 distribution Methods 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000010304 firing Methods 0.000 description 2
- 102100026278 Cysteine sulfinic acid decarboxylase Human genes 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 108010064775 protein C activator peptide Proteins 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a system and a method for scheduling target range flow task slices, wherein a flow file storage is used for storing flow file packets and slices thereof; the flow actuator pool comprises a certain number of flow actuators and can actively destroy the flow actuators and expand the flow actuator pool; the shooting range service acquires the slice address according to the flow file packet, and transmits the slice address, the source IP and the target IP to the flow task scheduling service module for flow slice task registration; the flow task scheduling service registers information of started flow actuators in the flow actuator pool and information of flow slicing tasks, and is used for distributing the flow slicing tasks and monitoring the flow actuator pool so as to destroy idle flow actuators and expand the flow actuator pool; and the flow actuator simulates real-time flow generation according to the distributed flow slicing tasks. The invention can improve the flexibility, fault tolerance and resource adjustability of the flow generation.
Description
Technical Field
The invention relates to a system and a method for scheduling target range flow task slices, belonging to the field of network security and computer software.
Background
The network target Range (Cyber Range) realizes maximum simulation and emulation to a real environment under the condition that one is separated or partially separated from a production environment and a service environment; from the technical implementation point of view, the technology is mainly virtualized; the virtualization technology simulates the operation of hardware equipment and a system in a shooting range environment, so that the requirements of related environments are simulated under the condition of no actual equipment and system, and the virtualization technology comprises various virtualization technologies such as container virtualization, digital simulation, simulators and protocol simulation.
The current deployment scenario for the firing ground traffic task scheduling is shown in fig. 1. The virtual network environment is composed of virtual devices and can include virtual terminals and network devices such as virtual machines, routers and switches. The shooting range service provides a virtual environment operation interface, manages the virtual network environment, and starts and closes flow task execution. The topology node may be called a target machine, i.e. a virtual terminal, a virtual machine, as a target node where traffic occurs. The flow generator is a flow task execution virtual machine and performs flow generation simulation on the target node in real time. The flow file memory is used for storing the flow file. The virtual machine management platform is responsible for starting and closing a virtual network environment and managing virtual machine equipment and the like.
The current task scheduling of the firing ground flow has the following problems: 1. at present, the flow generation mainly depends on a flow generator to simulate real-time flow, the flow generator is started along with a virtual network environment, one flow generator is needed to be started every time one virtual network environment is started, a flow task belongs to real-time operation, and in a shooting range service, the flow task execution operation is carried out on the virtual network environment not at all times, so that the flow generator does not work for most of time in the process of operating the virtual network environment, and the resource waste phenomenon exists. 2. The flow task mainly plays back the PCAP file through the flow generator to simulate real-time flow, and when the flow file is large, the execution time is long, so that the flexibility is insufficient. 3. The flow generator is highly coupled with the virtual network environment, and if the flow generator fails, real-time flow cannot be simulated, so that the defects of high coupling, low availability and inextensibility exist.
Disclosure of Invention
The purpose of the invention is as follows: in view of the problems in the prior art, the present invention is to provide a system and a method for scheduling target yard traffic task slices, which can improve the traffic generation flexibility, fault tolerance and resource adjustability.
The technical scheme is as follows: in order to achieve the purpose, the invention adopts the following technical scheme:
an end-of-range traffic task slice scheduling system, comprising:
a traffic file storage storing traffic file packets and slices thereof;
a flow actuator pool comprising at least one flow actuator;
the system comprises a target range service module, a flow task scheduling service module and a target IP (Internet protocol) module, wherein the target range service module is used for acquiring a slice address according to a flow file packet, and transmitting the slice address, a source IP and a target IP into the flow task scheduling service module for flow slice task registration;
the flow task scheduling service module is registered with information of the started flow executors in the flow executor pool and information of flow slicing tasks; the system is used for distributing flow slicing tasks and monitoring a flow actuator pool so as to destroy an idle flow actuator and an expansion flow actuator pool; detecting a heartbeat thread through a task, calculating and recording an expansion load rate and a destruction load rate according to a fixed time interval, if the expansion load rate is always greater than a set first threshold value within a set time length, expanding the capacity of a flow actuator pool, and if the destruction load is always greater than a defined second threshold value within the set time length, destroying a part of idle flow actuators; the capacity expansion load rate is the number of unallocated tasks divided by the number of tasks accepted by the idle flow executor; the destruction load rate is the difference value of the number of idle flow actuators and the number of flow actuators required by unallocated tasks divided by the number of flow actuators in the flow actuator pool;
and the flow executor downloads the corresponding slice file from the flow file storage according to the slice address after receiving the distributed flow slice task, and simulates the real-time flow generation.
Preferably, in the traffic task scheduling service module, when part of the idle traffic actuators needs to be destroyed, the task allocation permission of the traffic actuators is closed; and detecting the state of the flow actuator through a flow actuator heartbeat detection thread, if the flow actuator cannot work normally all the time within a set time length or the task allocation authority of the flow actuator is closed, cancelling the flow actuator from a flow actuator pool, and if the flow actuator has an unexecuted task, releasing the unexecuted task.
Preferably, when allocating the traffic slice task, the traffic task scheduling service module allocates the traffic slice task according to a fixed number of tasks of each traffic executor.
Preferably, when allocating the traffic slice tasks, the traffic task scheduling service module divides the number of unallocated tasks by the number of idle traffic actuators to obtain the number of tasks of each traffic actuator, allocates according to a calculation result if the number of tasks is within a set range of the number of tasks, allocates according to a minimum value if the number of tasks is lower than the minimum value, and allocates according to a maximum value if the number of tasks is higher than the maximum value.
Preferably, when distributing the traffic slice tasks, the traffic task scheduling service module determines the maximum load task value according to the machine configuration of each idle traffic actuator, and divides the number of unallocated tasks by the sum of the maximum load tasks of all the idle traffic actuators to obtain the distribution proportion, and the number of tasks distributed by each idle traffic actuator is determined according to the maximum load task value and the distribution proportion.
Preferably, the slices of the traffic file packet are sliced according to a fixed size, a number of data packets, or a slicing duration.
Preferably, after the traffic task scheduling service module is started, the virtual machine management platform initializes the traffic actuator pool, and starts a preset number of traffic actuators.
A method for scheduling target range flow task slices comprises the following steps:
initializing a flow executor pool, and registering a started flow executor into a flow task scheduling service;
uploading the flow file packet to a flow file storage, segmenting, and storing the slicing file and the relation between the slicing file and the flow file packet;
after receiving a flow task execution instruction, acquiring a slice address according to a flow file packet configured in a flow task, transmitting the slice address, a source IP and a target IP to a flow task scheduling service, and registering the flow slice task;
the flow task scheduling service distributes flow slicing tasks and monitors a flow executor pool;
after receiving the distributed flow slicing task, the flow executor downloads a corresponding slice file from the flow file storage according to a slice address, and simulates real-time flow generation;
the flow task scheduling service calculates and records the capacity expansion load rate and the destruction load rate according to a fixed time interval by detecting a heartbeat thread through a task, expands the capacity of the flow actuator pool if the capacity expansion load rate is always greater than a set first threshold value within a set time period, and destroys part of idle flow actuators if the destruction load rate is always greater than a defined second threshold value within the set time period; the capacity expansion load rate is the number of unallocated tasks divided by the number of tasks accepted by the idle flow executor; the destruction load rate is the difference value between the number of idle flow executors and the number of flow executors required by the unallocated tasks divided by the number of flow executors in the flow executor pool.
Has the advantages that: compared with the prior art, the invention has the following advantages:
1. the invention adopts the flow executor pool to manage the flow executor, can be decoupled with the virtual network environment, is easier to maintain, and effectively avoids the conditions that the virtual network environment is excessively started, the flow executor is not used for a long time and occupies a large amount of resources.
2. The invention slices the flow file packet and generates a corresponding flow slicing task in the flow scheduling service, thereby quickening the flow execution, and the flow file packet can be executed by a plurality of flow executors at the same time, which is more similar to the flow generation in real life.
3. The invention sets a capacity expansion and destruction mechanism of the flow executor, and the flow executor pool can expand and destroy the flow executor pool according to the capacity expansion load and the destruction load, thereby enhancing the regulation capability of the system.
4. The invention also sets a flow actuator heartbeat detection mechanism, can timely find that the executing flow actuator is damaged or abnormal in execution, can not cause the loss problem of the flow task, and can log off the flow actuator and release the flow slicing task after the heartbeat detection of the flow actuator, thereby greatly improving the fault tolerance.
Drawings
Fig. 1 is a schematic diagram of a task scheduling architecture of conventional shooting range traffic.
Fig. 2 is a schematic diagram of a task slice scheduling architecture for a target range traffic according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a task slice scheduling process of the off-target traffic according to an embodiment of the present invention.
Fig. 4 is a schematic flow chart illustrating a flow of executing a traffic task scheduling service according to an embodiment of the present invention.
Fig. 5 is a schematic flow chart illustrating a flow file packet processing procedure according to an embodiment of the present invention.
Fig. 6 is a flow chart illustrating a flow task execution according to an embodiment of the present invention.
Detailed Description
The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments.
As shown in fig. 2, a system for scheduling a destination traffic task slice according to an embodiment of the present invention mainly includes a traffic file storage, a traffic executor pool, a destination service module, and a traffic task scheduling service module. The flow file storage stores the flow file packet and the slices thereof, can segment the flow file packet according to a user-defined rule, and maintains the relationship between the flow file packet and the slices; the flow actuator pool comprises a certain number of (at least one) flow actuators and can actively destroy the flow actuators and expand the flow actuator pool; the system comprises a target range service module, a flow task scheduling service module and a target IP (Internet protocol) module, wherein the target range service module is used for acquiring a slice address according to a flow file packet, and transmitting the slice address, a source IP and a target IP into the flow task scheduling service module for flow slice task registration; the flow task scheduling service module is registered with information of the started flow executors in the flow executor pool and information of flow slicing tasks; the system is used for distributing flow slicing tasks and monitoring a flow actuator pool so as to destroy an idle flow actuator and an expansion flow actuator pool; and the flow executor downloads the corresponding slice file from the flow file storage according to the slice address after receiving the distributed flow slice task, and simulates real-time flow generation.
Specifically, the flow task scheduling service module detects a heartbeat thread through a task, calculates and records an expansion load rate and a destruction load rate according to a fixed time interval, expands the capacity of the flow actuator pool if the expansion load rate is always greater than a set first threshold value within a set time period, and destroys part of idle flow actuators if the destruction load rate is always greater than a defined second threshold value within the set time period. The capacity expansion load rate is the number of unallocated tasks divided by the number of tasks acceptable to the idle traffic executor. The destruction load rate is the difference between the number of idle flow actuators and the number of flow actuators needed by the unassigned tasks divided by the number of flow actuators in the pool of flow actuators.
In addition, in the traffic task scheduling service module, when part of the idle traffic actuators needs to be destroyed, the task allocation authority of the traffic actuators is closed; and detecting the state of the flow actuator through a flow actuator heartbeat detection thread, if the flow actuator cannot work normally all the time within a set time length or the task allocation authority of the flow actuator is closed, cancelling the flow actuator from a flow actuator pool, and if the flow actuator has an unexecuted task, releasing the unexecuted task.
As shown in fig. 3, based on the shooting range traffic task slice scheduling system, the shooting range traffic task slice scheduling method disclosed in the embodiment of the present invention mainly includes the following steps:
initializing a flow executor pool, and registering a started flow executor into a flow task scheduling service;
uploading the flow file packet to a flow file storage, segmenting, and storing the slicing file and the relation between the slicing file and the flow file packet;
after receiving a flow task execution instruction, acquiring a slice address according to a flow file packet configured in a flow task, transmitting the slice address, a source IP and a target IP to a flow task scheduling service, and registering the flow slice task;
the flow task scheduling service distributes flow slicing tasks and monitors a flow executor pool;
after receiving the distributed flow slicing task, the flow executor downloads a corresponding slice file from the flow file storage according to a slice address, and simulates real-time flow generation;
the flow task scheduling service detects a heartbeat thread through a task, calculates and records an expansion load rate and a destruction load rate according to a fixed time interval, expands the capacity of the flow actuator pool if the expansion load rate is always greater than a set first threshold value within a set time period, and destroys part of idle flow actuators if the destruction load rate is always greater than a defined second threshold value within the set time period.
The following describes in detail an operation process of the traffic task scheduling service, a traffic package processing process, and a traffic task execution process according to the embodiment of the present invention with reference to fig. 4 to 6.
As shown in fig. 4, the operation process of the traffic task scheduling service in the embodiment of the present invention mainly includes:
a1, starting a flow task scheduling service.
And A2, the flow task scheduling service requests the virtual machine management platform to initialize the task executor pool.
And A3, starting the specified quantity of flow executors by the virtual machine management platform.
And A4, registering the started flow executor into a flow task scheduling service according to the IP.
And step A5, starting a heartbeat detection thread of the flow actuator, detecting the state of the flow actuator, and performing heartbeat detection according to a fixed time interval.
And A6, after the heartbeat detection of the fixed period is exceeded, if the flow executor still cannot work normally (if the flow executor cannot be communicated) or the task can be assigned with the right to be closed, the flow executor is cancelled from the flow executor pool, and the unexecuted task is released.
Step A7. the traffic scheduling service performs task allocation for the non-task traffic actuators in the pool of traffic actuators. The allocation mode can adopt a mode of dynamic allocation according to a preset range, static allocation with a fixed quantity, allocation according to load capacity and the like.
The dynamic allocation is a set task number range, for example, when the task is allocated, firstly, the following calculation is carried out:
the number of tasks of each flow executor = the number of unallocated tasks ÷ the number of idle flow executors
And if the task number of each flow executor is within the range of the set task number, distributing the tasks according to the calculation result, and if the task number is lower than the minimum value or higher than the maximum value, distributing the tasks according to the minimum value or the maximum value.
Static allocation is allocation according to a fixed number of tasks for each flow executor.
The load allocation is that when the flow executor is registered, machine configuration is registered, such as relevant information of a memory, a CPU and the like, the number of tasks that can be allocated to the machine is calculated according to a custom calculation mode, for example, the number of tasks that can be allocated to an 8G memory and an 8-core CPU is 8, that is, the number of machine load tasks is 1 to 8, that is, the number of tasks can be allocated is 1 to 8, when the tasks are allocated, the total number of load tasks of all idle machines is calculated first, the allocation proportion is obtained by dividing the number of unallocated tasks by the total number of load tasks, that is, the number of tasks of each machine is the maximum value of load tasks of the machine multiplied by the allocation proportion, when the allocation proportion is too small, and the calculation result is less than 1, the allocation is performed according to 1.
Step A8. starts a task detection heartbeat thread, and load calculation is performed on the unassigned tasks and the number of idle traffic actuators at regular time intervals. Setting the capacity expansion load rate as the number of unallocated tasks divided by the number of tasks accepted by the idle flow executor; the destruction load rate is the difference between the number of idle flow actuators and the number of flow actuators needed by the unassigned tasks divided by the number of flow actuators in the pool of flow actuators.
For example, for a fixed number of static allocation modes, the expansion load rate and the destruction load rate are calculated and recorded according to the following formulas at the beginning of each heartbeat:
capacity expansion load rate = number of unallocated tasks ÷ (number of idle traffic actuators × fixed quota)
Destruction load rate = (number of free traffic actuators- (number of unallocated tasks ÷ fixed quota))/(number of traffic actuators in traffic actuator pool) /)
And for a non-static mode, when the number of tasks accepted by the idle flow executors and the flow executors required by unallocated tasks are calculated, calculating according to the maximum quota of each machine.
And A9., after the heartbeat detection of the fixed period is exceeded, if the capacity expansion load rate is always greater than 0.8, the capacity expansion is carried out on the flow actuator pool.
And step A10, the flow task scheduling service requests the virtual machine management platform to expand the flow actuator pool, and starts the flow actuators with the specified strategy quantity to be added into the flow actuator pool.
Step A11, after the fixed period heartbeat detection is exceeded, if the destruction load rate is always greater than 0.2, modifying the state of part of idle flow actuators, modifying the state to be task unallocated, and waiting for the heartbeat detection of the flow actuators to destroy the machine. The specific threshold value is a configuration item, can be adjusted, and is expanded and destroyed according to the set threshold value in the operation process.
And A12, repeating the step A4 until the task execution is completed. The heartbeat detection thread does not need to be started repeatedly after being started.
As shown in fig. 5, the process file processing flow mainly includes:
and B1, selecting a corresponding flow file packet in the shooting range service, and selecting a custom slice mode for uploading.
And B2, uploading the flow file packet to a flow file storage.
And B3, the flow file memory slices the flow file packet according to a custom slicing rule, for example, slicing according to a fixed size, slicing according to the number of file data packets, slicing according to the time length of the file packet, and the like.
Step B4. stores, associates and maintains the slice file and traffic bundle relationships.
Fig. 6 illustrates an execution process of a flow task in an embodiment of the present invention, which specifically includes the following steps:
and C1, issuing a flow task execution instruction by the target range service, and transmitting a source IP, a destination IP and a flow file packet address.
Step C2. the shooting range service obtains the traffic bundle address corresponding to the traffic task.
Step C3. the gatekeeper obtains the traffic packet slice address from the traffic file store corresponding to the traffic packet.
And C4, the target yard service registers the flow slice task to the flow task scheduling service according to the slice address, the source IP and the target IP as parameters. For different drone aircraft in the same drone environment or drone aircraft in different drone environments, the traffic task comprises a plurality of traffic file packets, a plurality of groups of source IP and destination IP, and the traffic task scheduling service is registered one by one.
And C5., after the flow slice task is registered, performing unified task allocation by the flow task scheduling service, and allocating the flow slice task to an idle flow actuator in the flow actuator pool.
Step C6. the traffic executor obtains from the traffic scheduling service the traffic slicing tasks that need to be executed.
Step C7. the traffic executor downloads the corresponding traffic bundle slice from the file storage server according to the file address corresponding to the task.
Step C8. simulates real-time traffic generation from the source IP, the target IP, and the traffic package slice file.
Step C9. the traffic executor re-acquires the traffic slicing task after it has executed the allocated traffic slicing task.
And C10, when the unallocated flow slice task cannot be acquired, sleeping for a fixed time and acquiring the flow slice task again.
And C11, after the traffic slice task is obtained, repeatedly executing the step C7 until all tasks are executed.
Claims (8)
1. A system for task slice scheduling for off-target traffic, comprising:
a traffic file storage storing traffic file packets and slices thereof;
a flow actuator pool comprising at least one flow actuator;
the system comprises a target range service module, a flow task scheduling service module and a target IP (Internet protocol) module, wherein the target range service module is used for acquiring a slice address according to a flow file packet, and transmitting the slice address, a source IP and a target IP into the flow task scheduling service module for flow slice task registration;
the flow task scheduling service module is registered with information of the started flow executors in the flow executor pool and information of flow slicing tasks; the system is used for distributing flow slicing tasks and monitoring a flow actuator pool so as to destroy an idle flow actuator and expand the flow actuator pool; detecting a heartbeat thread through a task, calculating and recording an expansion load rate and a destruction load rate according to a fixed time interval, if the expansion load rate is always greater than a set first threshold value within a set time length, expanding the capacity of a flow actuator pool, and if the destruction load is always greater than a defined second threshold value within the set time length, destroying a part of idle flow actuators; the capacity expansion load rate is the number of unallocated tasks divided by the number of tasks accepted by the idle traffic executor; the destruction load rate is the difference value of the number of idle flow actuators and the number of flow actuators required by unallocated tasks divided by the number of flow actuators in the flow actuator pool;
the flow executor downloads a corresponding slice file from the flow file storage according to a slice address after receiving the distributed flow slice task, and simulates real-time flow generation;
in the flow task scheduling service module, when part of idle flow executors need to be destroyed, task allocation permission of the flow executors is closed; and detecting the state of the flow actuator through a flow actuator heartbeat detection thread, if the flow actuator cannot work normally all the time within a set time length or the task allocation authority of the flow actuator is closed, cancelling the flow actuator from a flow actuator pool, and if the flow actuator has an unexecuted task, releasing the unexecuted task.
2. The system of claim 1, wherein the traffic task scheduling service module allocates a fixed number of tasks per traffic actuator when allocating the traffic slice tasks.
3. The system according to claim 1, wherein the traffic task scheduling service module divides the number of unallocated tasks by the number of idle traffic actuators to obtain the number of tasks for each traffic actuator when allocating the traffic slice tasks, allocates the number of tasks according to the calculation result if the number of tasks is within a set range of the number of tasks, allocates the number of tasks according to the minimum value if the number of tasks is lower than the minimum value, and allocates the number of tasks according to the maximum value if the number of tasks is higher than the maximum value.
4. The system for backing-up range flow task slice scheduling according to claim 1, wherein when allocating flow slice tasks, the flow task scheduling service module determines a maximum value of a load task according to machine configuration of each idle flow actuator, the number of unallocated tasks is divided by the sum of the maximum values of the load tasks of all idle flow actuators to obtain an allocation proportion, and the number of tasks allocated to each idle flow actuator is determined according to the maximum value of the load task and the allocation proportion.
5. The system of claim 1, wherein the slices of the traffic file packet are sliced according to a fixed size, a number of packets, or a slice duration.
6. The system according to claim 1, wherein the traffic task scheduling service module initializes a pool of traffic actuators through a virtual machine management platform after starting up, and starts up a preset number of traffic actuators.
7. A method for scheduling target range flow task slices is characterized by comprising the following steps:
initializing a flow executor pool, and registering a started flow executor into a flow task scheduling service;
uploading the flow file packet to a flow file storage, segmenting, and storing the slicing file and the relation between the slicing file and the flow file packet;
after receiving a flow task execution instruction, acquiring a slice address according to a flow file packet configured in a flow task, transmitting the slice address, a source IP and a target IP to a flow task scheduling service, and registering the flow slice task;
the flow task scheduling service distributes flow slicing tasks and monitors a flow executor pool;
after receiving the distributed flow slicing task, the flow executor downloads a corresponding slice file from the flow file storage according to a slice address, and simulates real-time flow generation;
the flow task scheduling service calculates and records the capacity expansion load rate and the destruction load rate according to a fixed time interval by detecting a heartbeat thread through a task, expands the capacity of the flow actuator pool if the capacity expansion load rate is always greater than a set first threshold value within a set time period, and destroys part of idle flow actuators if the destruction load rate is always greater than a defined second threshold value within the set time period; the capacity expansion load rate is the number of unallocated tasks divided by the number of tasks accepted by the idle flow executor; the destruction load rate is the difference value between the number of idle flow actuators and the number of flow actuators required by unallocated tasks divided by the number of flow actuators in the flow actuator pool;
in the flow task scheduling service, when part of idle flow executors need to be destroyed, task allocation permission of the flow executors is closed; and detecting the state of the flow actuator through a flow actuator heartbeat detection thread, if the flow actuator cannot work normally all the time within a set time length or the task allocation authority of the flow actuator is closed, cancelling the flow actuator from a flow actuator pool, and if the flow actuator has an unexecuted task, releasing the unexecuted task.
8. The method for scheduling the slice of the off-target traffic task according to claim 7, wherein the traffic task scheduling service allocates a fixed number of tasks per traffic actuator when allocating the traffic slice task;
or,
dividing the number of unallocated tasks by the number of idle flow executors to obtain the number of tasks of each flow executor, if the number of tasks is within a set range of the number of tasks, allocating according to a calculation result, if the number of tasks is lower than a minimum value, allocating according to a minimum value, and if the number of tasks is higher than the maximum value, allocating according to a maximum value;
or,
and determining the maximum value of the load task according to the machine configuration of each idle flow actuator, dividing the number of the unallocated tasks by the sum of the maximum values of the load tasks of all the idle flow actuators to obtain a distribution proportion, and determining the number of the tasks distributed by each idle flow actuator according to the maximum value of the load task and the distribution proportion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210596165.4A CN114679447B (en) | 2022-05-30 | 2022-05-30 | Target range flow task slice scheduling system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210596165.4A CN114679447B (en) | 2022-05-30 | 2022-05-30 | Target range flow task slice scheduling system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114679447A CN114679447A (en) | 2022-06-28 |
| CN114679447B true CN114679447B (en) | 2022-09-16 |
Family
ID=82079117
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210596165.4A Active CN114679447B (en) | 2022-05-30 | 2022-05-30 | Target range flow task slice scheduling system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114679447B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106610870A (en) * | 2016-12-28 | 2017-05-03 | 北京奇艺世纪科技有限公司 | Method and device for adjusting quantity of processing nodes |
| CN108900360A (en) * | 2018-08-10 | 2018-11-27 | 哈尔滨工业大学（威海） | A kind of network context generation system and method based on the playback of multinode flow |
| CN114500110A (en) * | 2022-04-07 | 2022-05-13 | 南京赛宁信息技术有限公司 | Dynamic generation system and method for concurrent flow of network shooting range |
-
2022
- 2022-05-30 CN CN202210596165.4A patent/CN114679447B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106610870A (en) * | 2016-12-28 | 2017-05-03 | 北京奇艺世纪科技有限公司 | Method and device for adjusting quantity of processing nodes |
| CN108900360A (en) * | 2018-08-10 | 2018-11-27 | 哈尔滨工业大学（威海） | A kind of network context generation system and method based on the playback of multinode flow |
| CN114500110A (en) * | 2022-04-07 | 2022-05-13 | 南京赛宁信息技术有限公司 | Dynamic generation system and method for concurrent flow of network shooting range |
Non-Patent Citations (1)
| Title |
|---|
| 基于云计算的网络靶场资源调度与优化;张鹏飞;《中国优秀硕士论文全文数据库》;20200516;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114679447A (en) | 2022-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10838777B2 (en) | Distributed resource allocation method, allocation node, and access node | |
| CN106708622B (en) | Cluster resource processing method and system and resource processing cluster | |
| EP2614436B1 (en) | Controlled automatic healing of data-center services | |
| CN111966305A (en) | Persistent volume allocation method and device, computer equipment and storage medium | |
| EP3340057A1 (en) | Container monitoring method and apparatus | |
| US8185905B2 (en) | Resource allocation in computing systems according to permissible flexibilities in the recommended resource requirements | |
| CN113037794B (en) | Method, device and system for computing resource allocation scheduling | |
| CN109787847B (en) | Cloud firewall full life cycle automatic management method | |
| CN109936593A (en) | A kind of method and system of message distribution | |
| CN109117244B (en) | Method for implementing virtual machine resource application queuing mechanism | |
| CN113382077B (en) | Micro-service scheduling method, micro-service scheduling device, computer equipment and storage medium | |
| CN108268305A (en) | For the system and method for virtual machine scalable appearance automatically | |
| CN112698838B (en) | Multi-cloud container deployment system and container deployment method thereof | |
| CN109302324A (en) | A kind of private clound monitoring and early warning method and system | |
| CN111177160B (en) | Service updating method, device, server and medium | |
| CN114679447B (en) | Target range flow task slice scheduling system and method | |
| CN111756800A (en) | Method and system for processing burst flow | |
| CN112035063A (en) | Hard disk and file system thermal expansion method based on cloud platform | |
| CN108243205A (en) | A kind of method, equipment and system for being used to control cloud platform resource allocation | |
| CN115225645A (en) | Service updating method, device, system and storage medium | |
| CN114546610A (en) | Mass data distributed desensitization device | |
| CN114615268A (en) | Service network, monitoring node, container node and equipment based on Kubernetes cluster | |
| CN114356533A (en) | Micro-service non-perception publishing system and method, electronic equipment and storage medium | |
| CN112711444A (en) | Virtualization method and system for power grid monitoring service unit | |
| CN110365520A (en) | Method for inspecting, device and the equipment of distributed system interior joint |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |