CN114979290A - Situation awareness compression algorithm selection method and device - Google Patents
Situation awareness compression algorithm selection method and device Download PDFInfo
- Publication number
- CN114979290A CN114979290A CN202210391592.9A CN202210391592A CN114979290A CN 114979290 A CN114979290 A CN 114979290A CN 202210391592 A CN202210391592 A CN 202210391592A CN 114979290 A CN114979290 A CN 114979290A
- Authority
- CN
- China
- Prior art keywords
- compression algorithm
- node
- bandwidth rate
- compression
- selecting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a method and a device for selecting a situation awareness compression algorithm, wherein the method comprises the following steps: acquiring a bandwidth rate predicted value of a node in a next acquisition period; calculating the uploading duration corresponding to the same collected data total amount by the node by adopting each compression algorithm; and selecting the compression algorithm corresponding to the shortest uploading time as a target compression algorithm. The method and the device select the optimal compression algorithm according to the bandwidth rate, and can solve the problem that the compression time consumption is long due to the fact that the compression algorithm carried by the situation awareness system is not matched with the real-time network environment, and the safety threat processing time is long, so that the time spent in the links of compressing data and transmitting data is shortened as much as possible, and the response efficiency of the whole situation awareness system is improved.
Description
Technical Field
The invention relates to the technical field of networks, in particular to a method and a device for selecting a situation awareness compression algorithm.
Background
The situation awareness system is a service system which discovers, identifies, analyzes and responds to security threats on the basis of security data gathered by each acquisition point. Therefore, the situation awareness system has higher time limit requirements on discovery and processing of security threats, and the data acquired by the acquisition nodes has the characteristics of large data volume, high acquisition frequency and the like.
At present, a compression algorithm adopted by a collection node of a situation awareness system is generally a compression algorithm carried by each collection node server, and as a network environment is changed, the problems of long data compression time, long security threat processing time and the like caused by poor matching with the network environment when the self-carried compression algorithm is used for a long time tend to exist.
Disclosure of Invention
The invention provides a method and a device for selecting a situation awareness compression algorithm, which aim to solve the technical problem of long data compression time and long security threat processing time caused by a self-carried compression algorithm in a situation awareness system in the related art.
In a first aspect, the present invention provides a method for selecting a situation-aware compression algorithm, including: acquiring a bandwidth rate predicted value of a node in a next acquisition period; calculating the uploading duration corresponding to the same collected data total amount by the node through adopting each compression algorithm, and specifically calculating the uploading duration according to the following formula:
wherein T is uploading duration, D is total data amount, V p For the compression speed, a is the compression ratio, V C The data transmission speed determined by the bandwidth rate predicted value; and selecting the compression algorithm corresponding to the shortest uploading time as a target compression algorithm.
Preferably, the obtaining of the bandwidth rate predicted value of the node in the next acquisition period specifically includes: obtaining a historical array of bandwidth rates for the new node, B ═ B 1 ，b 2 ，…，b n In which b i Is the bandwidth rate of the ith acquisition period, and n is a positive integer; bandwidth speed of the node according to the weighted sliding window average modelAnd processing the historical number sequence of the rate to obtain a bandwidth rate predicted value of the node in the next acquisition period:
wherein, w i Is a weighted weight of the bandwidth rate of the ith acquisition cycle.
Preferably, the acquiring the historical number sequence of the bandwidth rate of the new node specifically includes: after the new node is accessed to the network, the bandwidth rate of the new node in the network environment is counted according to a preset period, and a historical sequence of the bandwidth rate of the new node is obtained.
Preferably, after the obtaining the historical number series of the bandwidth rates of the new node and before the processing the historical number series of the bandwidth rates of the node according to the weighted sliding window average model, the method for selecting the situational awareness compression algorithm further includes: determining w according to positive correlation of influence of a period closer to the next acquisition period on a bandwidth rate predicted value of the next acquisition period i Satisfies the following conditions:
preferably, before the calculating of the upload duration corresponding to each compression algorithm adopted by the node for the same collected data total amount, the method for selecting a situation-aware compression algorithm further includes: and acquiring the compression ratio and the compression speed of each compression algorithm, wherein the compression ratio is the ratio of the total data amount before compression to the total data amount after compression.
Preferably, after the compression algorithm corresponding to the shortest uploading duration is selected as the target compression algorithm, the method for selecting the situation-aware compression algorithm further includes: the node compresses the acquired data according to a target compression algorithm to obtain compressed data; and transmitting the compressed data and the target compression algorithm identification to a central processing service node, so that the central processing service node decompresses the compressed data according to the decompression algorithm corresponding to the target compression algorithm identification.
In a second aspect, the present invention further provides a device for selecting a situation awareness compression algorithm, including an obtaining module, a calculating module, and a selecting module. And the acquisition module is used for acquiring the bandwidth rate predicted value of the node in the next acquisition period. The calculating module is connected with the obtaining module and used for calculating the uploading duration corresponding to the same collected data total amount by the node through each compression algorithm, and the uploading duration is calculated according to the following formula:
wherein T is uploading duration, D is total data amount, V p For the compression speed, a is the compression ratio, V C Is the data transmission speed determined by the bandwidth rate prediction value. And the selection module is connected with the calculation module and is used for selecting the compression algorithm corresponding to the shortest uploading duration as the target compression algorithm.
Preferably, the acquisition module comprises an acquisition unit and a processing unit. An acquisition unit configured to acquire a history sequence B ═ B of a bandwidth rate of a new node 1 ，b 2 ，…，b n In which b is i Is the bandwidth rate of the ith acquisition period, and n is a positive integer. The processing unit is connected with the acquisition unit and used for processing the historical data series of the bandwidth rate of the node according to the weighted sliding window average model to obtain the bandwidth rate predicted value of the node in the next acquisition period:
wherein, w i Is a weighted weight of the bandwidth rate of the ith acquisition period.
Preferably, the acquisition module further comprises a determination unit. The determining unit is connected with the processing unit and used for positively correlating the influence of the period closer to the next acquisition period on the bandwidth rate predicted value of the next acquisition period,determination of w i Satisfies the following conditions:
In a third aspect, the present invention further provides a device for selecting a situational awareness compression algorithm, including a memory and a processor, where the memory stores a computer program, and the processor is configured to execute the computer program to implement the method for selecting a situational awareness compression algorithm in the first aspect.
According to the situation awareness compression algorithm selection method and device provided by the invention, the bandwidth rate of the node in the next acquisition period is predicted, and then the time length required for uploading acquired data by adopting each compression algorithm is predicted according to the obtained bandwidth rate, so that the compression algorithm corresponding to the shortest time length required for uploading the acquired data is selected as the target compression algorithm of the node. The selected target compression algorithm is the optimal compression algorithm determined according to the real-time network environment of the nodes, so that the problem that the compression time consumption of the compression algorithm is long in the self-contained compression algorithm can be solved, the time spent in the links of data compression and data transmission is shortened as far as possible, and the response efficiency of the whole situation awareness system is improved.
Drawings
Fig. 1 is a flowchart of a method for selecting a situation-aware compression algorithm according to embodiment 1 of the present invention;
fig. 2 is a schematic structural diagram of a selection device of a situation-aware compression algorithm according to embodiment 2 of the present invention;
fig. 3 is a schematic structural diagram of a selection device for a situation-aware compression algorithm according to embodiment 3 of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the following detailed description will be made with reference to the accompanying drawings.
It is to be understood that the specific embodiments and figures described herein are merely illustrative of the invention and are not limiting of the invention.
It is to be understood that the embodiments and features of the embodiments can be combined with each other without conflict.
It is to be understood that, for the convenience of description, only parts related to the present invention are shown in the drawings of the present invention, and parts not related to the present invention are not shown in the drawings.
It should be understood that each unit and module related in the embodiments of the present invention may correspond to only one physical structure, may also be composed of multiple physical structures, or multiple units and modules may also be integrated into one physical structure.
It will be understood that, without conflict, the functions, steps, etc. noted in the flowchart and block diagrams of the present invention may occur in an order different from that noted in the figures.
It is to be understood that the flowchart and block diagrams of the present invention illustrate the architecture, functionality, and operation of possible implementations of systems, apparatus, devices and methods according to various embodiments of the present invention. Each block in the flowchart or block diagrams may represent a unit, module, segment, code, which comprises executable instructions for implementing the specified function(s). Furthermore, each block or combination of blocks in the block diagrams and flowchart illustrations can be implemented by a hardware-based system that performs the specified functions or by a combination of hardware and computer instructions.
It is to be understood that the units and modules involved in the embodiments of the present invention may be implemented by software, and may also be implemented by hardware, for example, the units and modules may be located in a processor.
Example 1:
the embodiment provides a method for selecting a situation awareness compression algorithm, which is applied to a situation awareness system or other networks requiring data compression and data transmission, and as shown in fig. 1, the method for selecting a compression algorithm includes:
In this embodiment, the node refers to an acquisition node of a situation awareness system or a node with data acquisition, compression, and transmission in other networks. The collection period can be set according to the requirements of users, such as 24 hours or 30 days. The nodes upload periodically acquired data periodically in an acquisition period, and the bandwidth rate refers to an average rate of the nodes in the process of uploading the acquired data. For example, a node has 5 data uploading behaviors in a certain acquisition period, and each time data is uploaded, there is an average rate, so the bandwidth rate of the acquisition period refers to an average value of 5 average rates. The bandwidth rate predicted value of the node in a future acquisition period is obtained, and prediction can be carried out through artificial intelligence methods or models such as machine learning and neural networks.
102, calculating the uploading duration corresponding to the same collected data total amount by the node through each compression algorithm, specifically calculating the uploading duration according to the following formula:
wherein T is uploading duration, D is total data amount, V p For the compression speed, a is the compression ratio, V C Is the data transmission speed determined by the bandwidth rate prediction value.
In this embodiment, the upload time duration T includes a data compression time duration and a data transmission time duration, and the compression ratio a is a ratio between the total data amount before compression and the total data amount after compression. The bandwidth rate is positively correlated with the data transmission speed, and the larger the bandwidth rate is, the larger the data transmission speed is. The data transmission speed determined by the predicted bandwidth rate value is related to the actual network performance, for example, in an ideal state, the theoretical value of the data transmission speed corresponding to the bandwidth rate of 2Mbps is 256KB/s, and the actual data transmission speed is about 103-200 KB/s. Because the data transmission speeds corresponding to the compression algorithms all come from the same bandwidth speed predicted value, the specific data transmission speed value does not influence the selection of the target compression algorithm from the uploading duration corresponding to each compression algorithm in the embodiment, and in addition, the specific value of the total data amount D does not influence the selection of the target compression algorithmAnd (6) selecting. As shown in table 1, the compression ratio and compression speed of several common compression algorithms are illustrated. Converting the bandwidth rate predicted value obtained from step 101 into data transmission speed and substituting the converted data transmission speed into a calculation formula of uploading duration, or directly substituting the bandwidth rate predicted value into the calculation formula of uploading duration, so that the uploading duration T mainly depends on V of a compression algorithm p And determining a compression ratio a, and sequentially substituting the compression ratio and the compression speed of each compression algorithm into a calculation formula of the uploading duration to obtain the uploading duration corresponding to each compression algorithm at the bandwidth rate of the next acquisition period.
TABLE 1
| Algorithm | Compression ratio | Speed of compression |
| LZ4 fast 8(v1.7.3) | 1.799 | 911MB/s |
| LZ4 default(v1.7.3) | 2.101 | 625MB/s |
| LZO 2.09 | 2.108 | 620MB/s |
| QuickLZ 1.5.0 | 2.238 | 510MB/s |
| Zstandard 1.1.1-1 | 2.876 | 330MB/s |
| Zstandard 1.1.1-3 | 3.164 | 200MB/s |
And 103, selecting the compression algorithm corresponding to the shortest uploading time as a target compression algorithm.
In this embodiment, for the 6 compression algorithms shown in table 1, the corresponding 6 upload durations are obtained, and the compression algorithm corresponding to the shortest upload duration is selected from the 6 upload durations as the target compression algorithm. In this embodiment, the bandwidth rate of the node in the next acquisition period is predicted, and then the uploading duration required by the node to acquire data uploading by adopting each compression algorithm is predicted according to the obtained bandwidth rate, so that the compression algorithm corresponding to the shortest uploading duration is selected as the target compression algorithm of the node. Therefore, the target compression algorithm selected based on the real-time bandwidth rate of the system or the network is the optimal compression algorithm determined according to the real-time network environment of the node, so that the problem that the compression time consumption of the compression algorithm is long can be solved. Because the uploading time in the embodiment includes the data compression time and the data transmission time, the selected target compression algorithm can shorten the time spent in the links of data compression and data transmission as much as possible, so as to improve the response efficiency of the whole situation awareness system.
Optionally, step 101: acquiring a bandwidth rate predicted value of a node in a next acquisition period, specifically comprising: obtaining a historical array of bandwidth rates for a new node, B ═ B 1 ，b 2 ，…，b n In which b is i Is the bandwidth rate of the ith acquisition period, and n is a positive integer; processing the historical sequence of the bandwidth rate of the node according to the weighted sliding window average model to obtain the bandwidth rate of the node in the next acquisition periodRate prediction value:
wherein w i Is a weighted weight of the bandwidth rate of the ith acquisition period.
In this embodiment, since the upload time of each compression algorithm is calculated, the requirements on specific values of the bandwidth rate and the data transmission speed are not strict, so that the historical sequence of the node bandwidth rate is processed by using the weighted sliding window average model to obtain the predicted value of the bandwidth rate in the next acquisition period.
Optionally, the obtaining of the historical number sequence of the bandwidth rate of the new node specifically includes: after the new node is accessed to the network, the bandwidth rate of the new node in the network environment is counted according to a preset period, and a historical sequence of the bandwidth rate of the new node is obtained.
In this embodiment, the preset period is, for example, 24 hours or 30 days. It should be noted that, this embodiment only exemplifies the historical number sequence of counting the bandwidth rate for the node of the new access network, and the historical number sequence of the bandwidth rate of the node existing in the network may also be counted according to the preset period, so as to periodically adjust the target compression algorithm of each node in the network according to the network environment.
Optionally, after acquiring a new historical sequence of bandwidth rates of the nodes and before processing the historical sequence of bandwidth rates of the nodes according to the weighted sliding window average model, the method for selecting the situational awareness compression algorithm further includes: determining w according to positive correlation of influence of a period closer to the next acquisition period on a bandwidth rate predicted value of the next acquisition period i Satisfies the following conditions:
in this embodiment, w is determined according to the positive correlation between the influence of the period closer to the next acquisition period on the predicted value of the bandwidth rate of the next acquisition period i The method is more consistent with the actual condition of a system or a network, namely the setting of the weighting weight enables the predicted value of the bandwidth rate to be more accurate, and the accuracy of the selection of the compression algorithm can be further improved. Specifically, the weighting weight for the nth cycle is 1/2, the weighting weight for the n-1 cycle is 1/4, and the weighting weight for each cycle is 1/2 of the weighting weight for the next cycle.
Optionally, at step 102: before the computing node adopts the uploading duration corresponding to each compression algorithm for the same collected data total amount, the method for selecting the situation awareness compression algorithm further comprises the following steps: and acquiring the compression ratio and the compression speed of each compression algorithm.
Optionally, at step 103: after the compression algorithm corresponding to the shortest uploading duration is selected as the target compression algorithm, the method for selecting the situation awareness compression algorithm further comprises the following steps: the node compresses the acquired data according to a target compression algorithm to obtain compressed data; and transmitting the compressed data and the target compression algorithm identification to a central processing service node, so that the central processing service node decompresses the compressed data according to the decompression algorithm corresponding to the target compression algorithm identification.
In this embodiment, since the target compression algorithm is obtained according to the bandwidth rate prediction value of the next acquisition period, target compression algorithms used in different acquisition periods may be different, and therefore, mapping tables of the compression algorithms and the compression algorithm identifiers are stored at both the node and the central processing service node. And uploading the identification of the target compression algorithm when the node uploads the acquired data, so that the central processing service node can be quickly matched with the corresponding decompression algorithm according to the identification, and the response efficiency of the situation awareness system is improved.
According to the method for selecting the situation awareness compression algorithm, bandwidth rates of the nodes in the next acquisition period are predicted, and then uploading duration required by the nodes for uploading acquired data by adopting each compression algorithm is predicted according to the obtained bandwidth rates, so that the compression algorithm corresponding to the shortest uploading duration is selected as the target compression algorithm of the nodes. Therefore, the target compression algorithm selected based on the real-time bandwidth rate of the system or the network is the optimal compression algorithm determined according to the real-time network environment of the node, so that the problem that the compression time consumption is long due to the fact that the compression algorithm carried by the node is not matched with the real-time network environment can be solved. Because the uploading time length comprises the data compression time length and the data transmission time length, the time spent in the links of compressing data and transmitting data based on the real-time bandwidth rate is shortened as much as possible, and the response efficiency of the whole situation awareness system is improved. Furthermore, a historical sequence of the node bandwidth rate is processed by adopting a weighted sliding window average model to obtain a bandwidth rate predicted value of the next acquisition period, compared with a method adopting machine learning and the like, the method is simpler and more convenient, the calculated amount of the node is reduced, and the selected target compression algorithm can save network resources of the node while meeting the calculation precision. And reasonable weighting weight is set in the weighted sliding window average model, so that the predicted value of the bandwidth rate is more accurate, and the accuracy of the selection of the compression algorithm can be further improved. In addition, the mapping tables of the compression algorithm and the compression algorithm identification are stored at the nodes and the central processing service node, and the acquired data is uploaded at the nodes and the identification of the target compression algorithm is uploaded at the same time, so that the central processing service node can be quickly matched with the corresponding decompression algorithm according to the identification, and the response efficiency of the situation awareness system is further improved.
Example 2:
as shown in fig. 2, the embodiment provides a selection device for a situation-aware compression algorithm, which includes an obtaining module 21, a calculating module 22, and a selecting module 23.
And the obtaining module 21 is configured to obtain a bandwidth rate predicted value of the node in the next acquisition period.
The calculating module 22 is connected to the obtaining module 21, and is configured to calculate an upload time length corresponding to the same collected data total amount by the node using each compression algorithm, specifically calculate the upload time length according to the following formula:
wherein T is uploading duration, D is total data amount, V p For the compression speed, a is the compression ratio, V C Is the data transmission speed determined by the bandwidth rate prediction value.
And the selection module 23 is connected with the calculation module 22 and is used for selecting the compression algorithm corresponding to the shortest uploading duration as the target compression algorithm.
Optionally, the obtaining module includes a obtaining unit and a processing unit.
An acquisition unit configured to acquire a history sequence B ═ B of a bandwidth rate of a new node 1 ，b 2 ，…，b n In which b is i Is the bandwidth rate of the ith acquisition period, and n is a positive integer.
The processing unit is connected with the acquisition unit and used for processing the historical data series of the bandwidth rate of the node according to the weighted sliding window average model to obtain the bandwidth rate predicted value of the node in the next acquisition period:
wherein, w i Is a weighted weight of the bandwidth rate of the ith acquisition period.
Optionally, the obtaining unit is specifically configured to, after the new node accesses the network, count the bandwidth rate of the new node in the network environment according to a preset period, and obtain a historical sequence of the bandwidth rate of the new node.
Optionally, the obtaining module further comprises a determining unit. The determining unit is connected with the processing unit and used for determining w according to positive correlation of influence of a period closer to the next acquisition period on a bandwidth rate predicted value of the next acquisition period i Satisfies the following conditions:
Optionally, the selection device of the situation-aware compression algorithm further includes a transmission module. The transmission module is connected with the selection module and used for compressing the acquired data according to a target compression algorithm to obtain compressed data; and transmitting the compressed data and the target compression algorithm identification to the central processing service node, so that the central processing service node decompresses the compressed data according to the decompression algorithm corresponding to the target compression algorithm identification.
In the device for selecting a situation awareness compression algorithm provided by this embodiment, the acquisition module acquires a bandwidth rate of a node in a next acquisition period, and then the calculation module predicts, according to the bandwidth rate of the acquisition module, an upload duration required by the node to upload acquired data by using each compression algorithm, so that the selection module selects, from the upload duration results obtained by the calculation module, a compression algorithm corresponding to the shortest upload duration as a target compression algorithm of the node. Therefore, the target compression algorithm selected based on the real-time bandwidth rate of the system or the network is the optimal compression algorithm determined according to the real-time network environment of the node, so that the problem that the compression time consumption is long due to the fact that the compression algorithm carried by the node is not matched with the real-time network environment can be solved, the time spent in the links of compressing data and transmitting data is shortened as far as possible, and the response efficiency of the whole situation awareness system is improved.
Example 3:
as shown in fig. 3, the embodiment provides a selection apparatus for a situational awareness compression algorithm, which includes a memory 31 and a processor 32, where the memory 31 stores a computer program, and the processor 32 is configured to execute the computer program to perform the selection method for the situational awareness compression algorithm according to embodiment 1.
The selection device for the situation awareness compression algorithm provided in this embodiment predicts the bandwidth rate of the node in the next acquisition period, and then predicts the upload time length required by the node to upload acquired data by using each compression algorithm according to the obtained bandwidth rate, so as to select the compression algorithm corresponding to the shortest upload time length as the target compression algorithm of the node. Therefore, the target compression algorithm selected based on the real-time bandwidth rate of the system or the network is the optimal compression algorithm determined according to the real-time network environment of the node, so that the problem that the compression time consumption is long due to the fact that the self-contained compression algorithm is not suitable for the real-time network environment can be solved, the time spent in the links of data compression and data transmission is shortened as far as possible, and the response efficiency of the whole situation awareness system is improved.
It will be understood that the above embodiments are merely exemplary embodiments taken to illustrate the principles of the present invention, which is not limited thereto. It will be apparent to those skilled in the art that various modifications and improvements can be made without departing from the spirit and substance of the invention, and these modifications and improvements are also considered to be within the scope of the invention.
Claims (10)
1. A method for selecting a situation-aware compression algorithm, comprising:
acquiring a bandwidth rate predicted value of a node in a next acquisition period;
calculating the uploading duration corresponding to the same collected data total amount by the node through adopting each compression algorithm, and specifically calculating the uploading duration according to the following formula:
wherein T is uploading duration, D is total data amount, V p For the compression speed, a is the compression ratio, V C The data transmission speed determined by the bandwidth rate predicted value;
and selecting the compression algorithm corresponding to the shortest uploading time as a target compression algorithm.
2. The method for selecting a situational awareness compression algorithm according to claim 1, wherein the obtaining of the bandwidth rate predicted value of the node in the next acquisition period specifically includes:
obtaining a historical array of bandwidth rates for a new node, B ═ B 1 ,b 2 ,…,b n In which b is i Is the bandwidth rate of the ith acquisition period, and n is a positive integer;
processing the historical data series of the bandwidth rate of the node according to the weighted sliding window average model to obtain a bandwidth rate predicted value of the node in the next acquisition period:
wherein, w i Is a weighted weight of the bandwidth rate of the ith acquisition period.
3. The method for selecting a situational awareness compression algorithm according to claim 2, wherein the obtaining of the historical number sequence of the bandwidth rate of the new node specifically includes:
after the new node is accessed to the network, the bandwidth rate of the new node in the network environment is counted according to a preset period, and a historical sequence of the bandwidth rate of the new node is obtained.
4. The method for selecting a situational awareness compression algorithm according to claim 2, further comprising, after the obtaining the historical sequence of bandwidth rates of the new node and before the processing the historical sequence of bandwidth rates of the node according to the weighted sliding window average model:
determining w according to positive correlation of influence of a period closer to the next acquisition period on a bandwidth rate predicted value of the next acquisition period i Satisfies the following conditions:
5. the method for selecting the situation awareness compression algorithm according to claim 1, wherein before the calculating the uploading duration corresponding to each compression algorithm for the same total amount of collected data by the node, the method further comprises:
and acquiring the compression ratio and the compression speed of each compression algorithm, wherein the compression ratio is the ratio of the total data amount before compression to the total data amount after compression.
6. The method for selecting a situation-aware compression algorithm according to claim 1, wherein after the selecting the compression algorithm corresponding to the shortest upload duration as the target compression algorithm, the method further comprises:
the node compresses the acquired data according to a target compression algorithm to obtain compressed data;
and transmitting the compressed data and the target compression algorithm identification to a central processing service node, so that the central processing service node decompresses the compressed data according to the decompression algorithm corresponding to the target compression algorithm identification.
7. A selection device of situation awareness compression algorithm is characterized by comprising an acquisition module, a calculation module and a selection module,
an obtaining module, configured to obtain a bandwidth rate predicted value of a node in a next acquisition period,
the calculating module is connected with the obtaining module and used for calculating the uploading duration corresponding to the same collected data total amount by the node through each compression algorithm, and the uploading duration is calculated according to the following formula:
wherein T is uploading duration, D is total data amount, V p For the compression speed, a is the compression ratio, V C For the data transmission speed determined by the bandwidth rate prediction value,
and the selection module is connected with the calculation module and is used for selecting the compression algorithm corresponding to the shortest uploading duration as the target compression algorithm.
8. The situation awareness compression algorithm selection device of claim 7, wherein the obtaining module comprises an obtaining unit and a processing unit,
an acquisition unit configured to acquire a history sequence B ═ B of a bandwidth rate of a new node 1 ,b 2 ,…,b n In which b is i Is the bandwidth rate of the ith acquisition cycle, n is a positive integer,
the processing unit is connected with the acquisition unit and used for processing the historical data series of the bandwidth rate of the node according to the weighted sliding window average model to obtain the bandwidth rate predicted value of the node in the next acquisition period:
wherein, w i Is a weighted weight of the bandwidth rate of the ith acquisition period.
9. The situational awareness compression algorithm selection apparatus of claim 8, wherein the acquisition module further comprises a determination unit,
the determining unit is connected with the processing unit and used for determining w according to positive correlation of influence of a period closer to the next acquisition period on a bandwidth rate predicted value of the next acquisition period i Satisfies the following conditions:
10. A selection apparatus for a situational awareness compression algorithm, comprising a memory having stored therein a computer program and a processor configured to execute the computer program to implement the method of selecting a situational awareness compression algorithm of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210391592.9A CN114979290A (en) | 2022-04-14 | 2022-04-14 | Situation awareness compression algorithm selection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210391592.9A CN114979290A (en) | 2022-04-14 | 2022-04-14 | Situation awareness compression algorithm selection method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114979290A true CN114979290A (en) | 2022-08-30 |
Family
ID=82976752
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210391592.9A Pending CN114979290A (en) | 2022-04-14 | 2022-04-14 | Situation awareness compression algorithm selection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114979290A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115499658A (en) * | 2022-09-20 | 2022-12-20 | 支付宝(杭州)信息技术有限公司 | Data transmission method and device of virtual world |
| CN117527708A (en) * | 2024-01-05 | 2024-02-06 | 杭银消费金融股份有限公司 | Optimized transmission method and system for enterprise data link based on data flow direction |
| WO2024114597A1 (en) * | 2022-12-02 | 2024-06-06 | City University Of Hong Kong | Reinforcement-learning-based network transmission of compressed genome sequence |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110769453A (en) * | 2019-10-31 | 2020-02-07 | 重庆大学 | Multi-modal monitoring data dynamic compression control method under unstable network environment |
| CN111262587A (en) * | 2018-11-30 | 2020-06-09 | 康泰医学***（秦皇岛）股份有限公司 | Data compression method, device, equipment and computer readable storage medium |
| CN113329442A (en) * | 2021-04-20 | 2021-08-31 | 北京连山科技股份有限公司 | Universal multilink load compression and decompression method and system |
| CN114265821A (en) * | 2021-12-28 | 2022-04-01 | 深圳市云语科技有限公司 | File compression algorithm selection method based on capability negotiation |
-
2022
- 2022-04-14 CN CN202210391592.9A patent/CN114979290A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111262587A (en) * | 2018-11-30 | 2020-06-09 | 康泰医学***（秦皇岛）股份有限公司 | Data compression method, device, equipment and computer readable storage medium |
| CN110769453A (en) * | 2019-10-31 | 2020-02-07 | 重庆大学 | Multi-modal monitoring data dynamic compression control method under unstable network environment |
| CN113329442A (en) * | 2021-04-20 | 2021-08-31 | 北京连山科技股份有限公司 | Universal multilink load compression and decompression method and system |
| CN114265821A (en) * | 2021-12-28 | 2022-04-01 | 深圳市云语科技有限公司 | File compression algorithm selection method based on capability negotiation |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115499658A (en) * | 2022-09-20 | 2022-12-20 | 支付宝(杭州)信息技术有限公司 | Data transmission method and device of virtual world |
| CN115499658B (en) * | 2022-09-20 | 2024-05-07 | 支付宝(杭州)信息技术有限公司 | Virtual world data transmission method and device |
| WO2024114597A1 (en) * | 2022-12-02 | 2024-06-06 | City University Of Hong Kong | Reinforcement-learning-based network transmission of compressed genome sequence |
| CN117527708A (en) * | 2024-01-05 | 2024-02-06 | 杭银消费金融股份有限公司 | Optimized transmission method and system for enterprise data link based on data flow direction |
| CN117527708B (en) * | 2024-01-05 | 2024-03-15 | 杭银消费金融股份有限公司 | Optimized transmission method and system for enterprise data link based on data flow direction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114979290A (en) | Situation awareness compression algorithm selection method and device | |
| CN111162934B (en) | Business service test method and device, storage medium and electronic device | |
| CN110572297B (en) | Network performance evaluation method, server and storage medium | |
| CN110351198B (en) | Flow regulation method, data transmission system, storage medium and equipment | |
| US8108332B2 (en) | Methods and systems for selecting features and using the selected features to perform a classification | |
| EP3491793B1 (en) | System and method for resource-aware and time-critical iot frameworks | |
| CN109412602B (en) | Distributed sparse signal detection method and device based on low bit quantization observation value | |
| CN110602178B (en) | Method for calculating and processing temperature sensor data based on edge compression | |
| CN112398700B (en) | Service degradation method and device, storage medium and computer equipment | |
| CN111352733A (en) | Capacity expansion and reduction state prediction method and device | |
| CN114676380A (en) | Trigger type acquisition method and system based on intelligent sensor edge monitoring | |
| CN111901134B (en) | Method and device for predicting network quality based on recurrent neural network model (RNN) | |
| CN109213965B (en) | System capacity prediction method, computer readable storage medium and terminal device | |
| CN112566170B (en) | Network quality evaluation method, device, server and storage medium | |
| CN111989897A (en) | Measurement indicators for computer networks | |
| WO2021147319A1 (en) | Data processing method, apparatus, device, and medium | |
| CN112906883A (en) | Hybrid precision quantization strategy determination method and system for deep neural network | |
| WO2023142351A1 (en) | Weight adjustment method and apparatus, and storage medium and electronic apparatus | |
| EP3834441A1 (en) | Compressive sensing systems and methods using edge nodes of distributed computing networks | |
| CN106533730B (en) | Hadoop cluster component index acquisition method and device | |
| CN114501344A (en) | Massive LoRa node fast ad hoc network communication method based on wireless ranging technology | |
| CN111026622B (en) | Method and device for testing maximum service request quantity of tested system | |
| KR20200046911A (en) | Data Compression Method and Apparatus Thereof | |
| CN115134333B (en) | Rapid networking method and system based on photovoltaic power generation and cloud platform | |
| JP7036143B2 (en) | Data compression system, data compression terminal, relay device, learning model construction device and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |