CN115914388A - Resource data fresh-keeping method based on monitoring data acquisition - Google Patents
Resource data fresh-keeping method based on monitoring data acquisition Download PDFInfo
- Publication number
- CN115914388A CN115914388A CN202211602378.XA CN202211602378A CN115914388A CN 115914388 A CN115914388 A CN 115914388A CN 202211602378 A CN202211602378 A CN 202211602378A CN 115914388 A CN115914388 A CN 115914388A
- Authority
- CN
- China
- Prior art keywords
- data
- content
- freshness
- client
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000012544 monitoring process Methods 0.000 title claims abstract description 17
- 238000013507 mapping Methods 0.000 claims abstract description 4
- 238000012986 modification Methods 0.000 claims description 7
- 230000004048 modification Effects 0.000 claims description 7
- 230000007423 decrease Effects 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 2
- 238000013480 data collection Methods 0.000 claims 1
- 238000004891 communication Methods 0.000 abstract description 13
- 238000005516 engineering process Methods 0.000 abstract description 7
- 230000005540 biological transmission Effects 0.000 abstract description 5
- 238000013461 design Methods 0.000 description 3
- 230000000875 corresponding effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a resource data fresh-keeping method based on monitoring data acquisition, which comprises the following steps of S1, defining the freshness of new content; s2, judging the weight of the new content; s3, comparing and updating the content version; and S4, adding a network element after the client caches the reached content and performing network element mapping management. The invention arranges the priority according to the dynamic freshness of the data items, ensures that the part with the highest priority is transmitted firstly each time under the condition of low bandwidth, namely ensures that the latest modified data of the FDBS is transmitted to the client firstly, associates the transactions and combines the result sets, effectively utilizes the asymmetry of network communication in the mobile environment after reducing the communication data volume and the consumption of the storage space of the client in the synchronization process, updates all mobile client data copies by a data broadcasting technology, and ensures that the mobile client caches the latest data by combining an increment updating algorithm based on the priority, thereby improving the transmission efficiency and the freshness of the data compared with the traditional increment updating algorithm.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a resource data fresh-keeping method based on monitoring data acquisition.
Background
With the rapid development of a large number of new energy sources, micro-grids and interactive devices in the power industry, the continuous emergence of new services such as spot transactions and the like, the management requirements of the power system are increasingly complex, and power enterprises put higher requirements on the data acquisition capacity, the service coverage and the intelligence level of power big data. The method has the advantages that the collection and monitoring of the operation data of the power system play a vital role in the safe operation and maintenance management of the power system, various sensors in the power system can generate a large amount of data for the periodic collection of physical information of the surrounding environment, most of the data can be expired in a short time, and therefore the data of the internet of things is different from the common service data in the network in terms of data volume and data characteristics, and the characteristic of frequent updating of the monitoring data of the internet of things and the requirement of a user on high data freshness are met.
In the prior art, for example, in a "communication channel resource scheduling method" of chinese patent application No. CN111882232A, a system quickly calculates a scheduling scheme meeting the needs of a power grid service by using complete basic data and predefined algorithm rules, and realizes optimized pushing of the scheduling scheme by incorporating various evaluation parameters into the algorithm, thereby providing a high-speed and high-quality auxiliary design tool for communication mode managers, reducing technical difficulty of mode design, and improving working efficiency of mode design. The preservation of the resource data of the communication management system is automatically completed through a mode working flow.
However, in the prior art, with the rapid development of the internet and the internet of things technology, the network scale is continuously enlarged, and a TCP/IP network architecture faces a serious challenge in the aspects of scalability, security, mobility, quality of service, and the like, and internet of things users tend to request the latest information, and provide a user-driven information freshness mechanism and a time for content to stay in a cache.
Disclosure of Invention
The invention aims to provide a resource data fresh-keeping method based on monitoring data acquisition, which aims to solve the problems that the prior mobile copy synchronization technology proposed by the background technology has large communication data volume and much consumed storage space, when the cache space is occupied by data with less request times, the cache efficiency is extremely low, a user request is sent to a server and longer content acquisition delay is generated, so that the user experience is influenced, and particularly when the network bandwidth is reduced, the data of a client side cannot be updated in time, so that the execution of mobile transactions fails.
In order to achieve the purpose, the invention provides the following technical scheme: a method for refreshing resource data based on monitoring data acquisition comprises the following steps:
s1, defining the freshness of new contents: the freshness of the data items is a function of time
The data freshness of the data item at the time t is determined by a time function
Determining
(ii) a The time function is specifically as follows:
where T is the deadline of the data item,
as data itemsThe latest time;
s2, judging the weight of the new content: according to the difference of the access frequency of different data by a user, the data with high access frequency needs to be cached in a client preferentially, and when the freshness of a data item is measured, a weight J is given to each column in a data table cached in a database, which is specifically as follows:
；
j is a measurement of column data access frequency of a user, and columns with high weight are considered first in decision making;
and S3, due to the characteristics of weak stability and low bandwidth of a wireless network in the mobile computing environment, when the bandwidth is sharply reduced, the data updating efficiency is reduced, and the data cached by the client side is invalid. The priority is arranged according to the dynamic freshness of the data, so that the part with the highest priority is transmitted firstly under the condition of low bandwidth, namely, the newly modified data of the FDBS is transmitted to the client firstly, and the transmission efficiency and the freshness of the data are improved compared with the traditional incremental updating algorithm. And (3) associating the transactions and combining the result sets, reducing the communication data volume and the consumption of the storage space of the client in the synchronization process, arranging the data items with the same new content weight and the comparison data items according to the priority, and comparing and updating the content versions: when the data item arrives at the client, judging whether the new arrival content can replace the cache content according to the cache space, the freshness of the new content and the weight comparison of the new content, and then outputting a comparison report of the arrival content and the cache content and carrying out service impact analysis.
Preferably, in step S1, the function of time
The value range is [0,1 ]]The freshness value gradually decreases over time.
Preferably, in step S2, when defining the data freshness of the data items, the field length of each data item is taken into consideration, and the weight of the column may be corrected according to the length.
Preferably, in step S2, the data items with the same weight of the new content and the comparison data items are arranged according to priority; the latest modification time of each data item record in the data items is compared with the data time cached by the client, a queue of the jth data item sorted according to the update interval time length is represented by a data structure Column _ queue (j =1,2, \8230;, n), and the priority is calculated as follows:
wherein the content of the first and second substances,
is the weight of the comparison data item; l is the medium time of the last of the comparison data items.
Preferably, in step S3, after the client caches the content, the client adds a new network element and performs network element mapping management.
Preferably, in step S3, the determining whether the new arrival content can replace the cached content includes the following steps: if the cache space is not full, directly caching; otherwise, the client side matches the names of the cache content and the arrival content.
Preferably, after the name matching is performed, if the name matching fails, calculating a weight of the reached content according to the step S2; if the contents are matched, comparing the freshness of the cache contents with the freshness of the arrival contents according to the step S1;
if the freshness of the cache content is less than the freshness of the arrival content, replacing the cache content with the arrival content; if the weight of the reached content is greater than the minimum weight of the client, replacing the cache content corresponding to the minimum popularity with the reached content; otherwise, the arriving content is not cached.
Preferably, if the freshness of the cached content is greater than the freshness of the reached content, the reached content is not cached and deleted.
Compared with the prior art, the invention has the beneficial effects that:
the invention arranges the priority according to the dynamic freshness of the data items, ensures that the part with the highest priority is transmitted firstly each time under the condition of low bandwidth, namely ensures that the latest modified data of the FDBS is transmitted to the client firstly, associates the transactions and combines the result sets, effectively utilizes the asymmetry of network communication in the mobile environment after reducing the communication data volume and the consumption of the storage space of the client in the synchronization process, updates all mobile client data copies by a data broadcasting technology, and ensures that the mobile client caches the latest data by combining an increment updating algorithm based on the priority, thereby improving the transmission efficiency and the freshness of the data compared with the traditional increment updating algorithm.
Drawings
Fig. 1 is a flowchart of a method for refreshing resource data based on monitoring data acquisition according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to FIG. 1: a method for refreshing resource data based on monitoring data acquisition comprises the following steps:
step one, defining the freshness of new content: all data items of the client side have a global time stamp formed by a main time stamp and a secondary time stamp and a unique identification number ID. The primary time stamp is a time stamp added by FDBS modification data, the primary time stamp of each data object is taken from a global clock on a fixed network, the EMDB cannot modify the primary time stamp, the secondary time stamp is a transaction stamp added by client modification data, and an ID number is used for uniquely identifying each data item. The freshness of the data items is a function of time
The freshness of the data item at time t is determined by a function of time
Determining
The value field of the time function is [0,1 ]]The freshness value gradually decreases with the passage of time; function of time
The method comprises the following specific steps:
wherein, T is the deadline of the data item and is the latest moment of the data item. According to different access frequencies of different data by the mobile user, the data with high access frequency needs to be cached in the client preferentially.
Step two, judging the weight of the new content: according to the difference of the access frequency of different data by a user, the data with high access frequency needs to be cached in a client preferentially, and when the freshness of a data item is measured, a weight J is given to each column in a data table cached in a database, which is specifically as follows:
；
where J is a measure of the access frequency of the column data by the user, and the column with the highest weight is considered first in the decision.
Due to the characteristics of weak stability and low bandwidth of a wireless network in a mobile computing environment, when the bandwidth is sharply reduced, the data updating efficiency is reduced, and the data cached by the client is invalid. The priorities are arranged according to the dynamic freshness of the data items, so that the part with the highest priority is ensured to be transmitted firstly under the condition of low bandwidth, namely, the newly modified data of the FDBS is ensured to be transmitted to the client firstly, and the transmission efficiency and the freshness of the data are improved compared with the traditional incremental updating algorithm. The affairs are correlated and the results are combined, the communication data volume and the consumption of the storage space of the client in the synchronization process are reduced, and the data items with the same weight of the new content and the comparison data items are arranged according to the priority; comparing the latest modification time of each data item record in the data items with the data time cached by the client, and using a data structure Column _ queue (j =1,2, \8230;, n) to represent a queue in which the jth data item is sorted according to the length of the update interval, wherein the interval is arranged in the front, and the priority is calculated as follows:
wherein the content of the first and second substances,
is the weight of the comparison data item; l is the medium time of the last of the comparison data items.
Step three, comparing and updating content versions: when the data item arrives at the client, judging whether the new arrival content can replace the cache content according to the cache space, the freshness of the new content and the weight comparison of the new content, and then outputting a comparison report of the arrival content and the cache content and carrying out service impact analysis. Judging whether the new arrival content can replace the cache content or not, comprising the following steps: if the cache space is not full, directly caching; otherwise, the client will perform name matching on the cached content and the arriving content.
After name matching is carried out, if the name is not matched, calculating the weight of the arrival content according to the step S2; if the contents are matched, comparing the freshness of the cache contents with the freshness of the arrival contents according to the step S1;
if the freshness of the cache content is less than the freshness of the arrival content, replacing the cache content with the arrival content; if the weight of the reached content is greater than the minimum weight of the client, replacing the cache content corresponding to the minimum popularity with the reached content; otherwise, not caching the arrival content; and if the freshness of the cached content is greater than the freshness of the arrived content, not caching the arrived content and deleting the arrived content.
And step four, the data with high priority are put into a broadcast queue in the sequence of high priority and low priority, and when the wireless bandwidth is reduced, the data with high priority are synchronized to the client through the broadcast technology, namely the latest data are ensured to be transmitted first, and the limited communication bandwidth is saved. And adding a network element after the client caches the reached content and performing network element mapping management.
The invention arranges the priority according to the dynamic freshness of the data items, ensures that the part with the highest priority is transmitted firstly each time under the condition of low bandwidth, namely ensures that the latest modified data of the FDBS is transmitted to the client firstly, associates the transactions and combines the result sets, effectively utilizes the asymmetry of network communication in the mobile environment after reducing the communication data volume and the consumption of the storage space of the client in the synchronization process, updates all mobile client data copies by a data broadcasting technology, and ensures that the mobile client caches the latest data by combining an increment updating algorithm based on the priority, thereby improving the transmission efficiency and the freshness of the data compared with the traditional increment updating algorithm.
Although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that various changes in the embodiments and/or modifications of the invention can be made, and equivalents and modifications of some features of the invention can be made without departing from the spirit and scope of the invention.
Claims (8)
1. A resource data fresh-keeping method based on monitoring data acquisition is characterized by comprising the following steps:
s1, defining the freshness of new contents: the freshness of the data items is a function of time
The data freshness of the data item at the time t is determined by a time function
Determining
(ii) a Said function of time
The method comprises the following specific steps:
where T is the expiration time of the data item,
the time when the data item is up to date;
s2, judging the weight of the new content: according to the difference of the access frequency of different data by a user, the data with high access frequency needs to be cached in a client preferentially, and when the freshness of a data item is measured, a weight J is given to each column in a data table cached in a database, which is specifically as follows:
；
j is a measurement of column data access frequency of a user, and columns with high weight are considered first in decision making;
s3, content version comparison and updating: when the data item arrives at the client, judging whether the new arrival content can replace the cache content according to the cache space, the freshness of the new content and the weight comparison of the new content, and then outputting a comparison report of the arrival content and the cache content and carrying out service impact analysis.
2. The method for refreshing resource data based on monitoring data acquisition as claimed in claim 1, wherein in step S1, the time function
The value range is [0,1 ]]The freshness value gradually decreases over time.
3. The method for refreshing resource data based on monitoring data acquisition as claimed in claim 1, wherein in step S2, when defining the data freshness of the data items, the field length of each data item is considered, and the weight of the column is modified according to the length.
4. The method for refreshing resource data based on monitoring data acquisition as claimed in claim 1, wherein in step S2, the data items with equal weight of new content and the comparison data items are arranged according to priority; the latest modification time of each data item record in the data items is compared with the data time cached by the client, a queue of the jth data item sorted according to the update interval time length is represented by a data structure Column _ queue (j =1,2, \8230;, n), and the priority is calculated as follows:
wherein, the first and the second end of the pipe are connected with each other,
is the weight of the comparison data item; l is the medium time of the last of the comparison data items.
5. The method for refreshing resource data based on monitoring data acquisition as claimed in claim 1, wherein in step S3, the client adds new network elements after caching the arriving content and performs network element mapping management.
6. The method for refreshing resource data based on monitoring data collection according to claim 5, wherein said determining whether the new arriving content can replace the cached content in step S3 comprises the following steps: if the cache space is not full, directly caching; otherwise, the client side matches the names of the cache content and the arrival content.
7. The method for refreshing resource data based on monitoring data acquisition as claimed in claim 6, wherein after name matching, if the name matching fails, the weight of the arriving content is calculated according to step S2; if the contents are matched, comparing the freshness of the cache contents with the freshness of the arrival contents according to the step S1;
if the freshness of the cache content is less than the freshness of the arrival content, replacing the cache content with the arrival content; if the weight of the reached content is greater than the minimum weight of the client, replacing the cache content corresponding to the minimum popularity with the reached content; otherwise, the arriving content is not cached.
8. The method of claim 7, wherein if the freshness of the cached content is greater than the freshness of the arrived content, the arrived content is not cached and deleted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211602378.XA CN115914388A (en) | 2022-12-14 | 2022-12-14 | Resource data fresh-keeping method based on monitoring data acquisition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211602378.XA CN115914388A (en) | 2022-12-14 | 2022-12-14 | Resource data fresh-keeping method based on monitoring data acquisition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115914388A true CN115914388A (en) | 2023-04-04 |
Family
ID=86472452
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211602378.XA Pending CN115914388A (en) | 2022-12-14 | 2022-12-14 | Resource data fresh-keeping method based on monitoring data acquisition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115914388A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105512053A (en) * | 2015-12-07 | 2016-04-20 | 中南大学 | Mirror caching method for mobile transparent computing system server terminal multi-user access |
| CN109660598A (en) * | 2018-11-17 | 2019-04-19 | 华中科技大学 | A kind of buffer replacing method and system of Internet of Things Temporal Data |
| CN112286840A (en) * | 2020-11-05 | 2021-01-29 | 南京邮电大学 | Metadata cache replacement method for cloud storage system |
| CN112637273A (en) * | 2020-12-04 | 2021-04-09 | 重庆邮电大学 | Intelligent hot spot data prediction and cache method |
-
2022
- 2022-12-14 CN CN202211602378.XA patent/CN115914388A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105512053A (en) * | 2015-12-07 | 2016-04-20 | 中南大学 | Mirror caching method for mobile transparent computing system server terminal multi-user access |
| CN109660598A (en) * | 2018-11-17 | 2019-04-19 | 华中科技大学 | A kind of buffer replacing method and system of Internet of Things Temporal Data |
| CN112286840A (en) * | 2020-11-05 | 2021-01-29 | 南京邮电大学 | Metadata cache replacement method for cloud storage system |
| CN112637273A (en) * | 2020-12-04 | 2021-04-09 | 重庆邮电大学 | Intelligent hot spot data prediction and cache method |
Non-Patent Citations (1)
| Title |
|---|
| 王瑞峰等: "移动数据库中数据复制同步处理策略的研究" * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tang et al. | A smart caching mechanism for mobile multimedia in information centric networking with edge computing | |
| Romano et al. | A neural network proxy cache replacement strategy and its implementation in the Squid proxy server | |
| Lee et al. | On transaction processing with partial validation and timestamp ordering in mobile broadcast environments | |
| Gessert et al. | Quaestor: Query web caching for database-as-a-service providers | |
| Li et al. | A Load-balancing method for network GISs in a heterogeneous cluster-based system using access density | |
| Sheng et al. | Edge caching for IoT transient data using deep reinforcement learning | |
| CN115914388A (en) | Resource data fresh-keeping method based on monitoring data acquisition | |
| US20220035790A1 (en) | Distributed data processing | |
| Detti et al. | Modeling LRU cache with invalidation | |
| Lam et al. | Approaches for broadcasting temporal data in mobile computing systems | |
| Wang et al. | Dynamic cache consistency schemes for wireless cellular networks | |
| Bahat et al. | Measuring consistency in TTL-based caches | |
| Cohen et al. | On the power of false negative awareness in indicator-based caching systems | |
| Cohen et al. | False negative awareness in indicator-based caching systems | |
| Lam et al. | Concurrency control strategies for ordered data broadcast in mobile computing systems | |
| Chiang et al. | Periodic cache replacement policy for dynamic content at application server | |
| Fadelelmoula | Exploiting Cloud Computing and Web Services to Achieve Data Consistency, Availability, and Partition Tolerance in the Large-Scale Pervasive Systems. | |
| Lam et al. | Broadcasting consistent data to read-only transactions from mobile clients | |
| Hoang et al. | An enhanced semantic-based cache replacement algorithm for web systems | |
| Liu et al. | PCR: caching replacement algorithm in proxy server | |
| Muthuswamy et al. | A hashing scheme for multi-channel wireless broadcast | |
| Leu et al. | An energy efficient re-access scheme for data caching in data broadcast of a mobile computing environment | |
| Dou et al. | Efficient Optimized Strategy of Big Data Retrieval | |
| Katsaros et al. | Cache management for Web-powered databases | |
| Bright | Profile-based data delivery for web applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230404 |