CN101207513A - Apparatus and method for saving historical data - Google Patents
Apparatus and method for saving historical data Download PDFInfo
- Publication number
- CN101207513A CN101207513A CNA2006101707015A CN200610170701A CN101207513A CN 101207513 A CN101207513 A CN 101207513A CN A2006101707015 A CNA2006101707015 A CN A2006101707015A CN 200610170701 A CN200610170701 A CN 200610170701A CN 101207513 A CN101207513 A CN 101207513A
- Authority
- CN
- China
- Prior art keywords
- data
- historical data
- dump list
- upper limit
- sheet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for storing historical data, and relates to the storing of network management historical data. The invention is invented to solve the problem existing in the prior art that the historical data sheet is too large to cause the locking time for the sheet to be inserted with data to be overlong and influence the user to normally inquiry the historical data. The method for storing the historical data comprises the following steps: firstly, the data quantity upper limit of the sheet is set; secondly, whether the historical data in the sheet exceeds the data quantity upper limit is judged. If the historical data in the sheet exceeds the data quantity upper limit, the historical data part which exceeds the data quantity upper limit is inserted to a dump sheet, and is deleted from the sheet. The corresponding device of the invention comprises an original sheet provided with a data appending interface and a data inquiry interface, and also comprises a dump sheet, and the dump sheet is provided with a data inquiry interface. A data transferring interface is arranged between the dump sheet and the original sheet. The invention is mainly applied to store the historical data in a network management system.
Description
Technical field
The present invention relates to the method and apparatus that data are preserved, particularly the method and apparatus of network management historical data preservation.
Background technology
In existing a lot of management systems, need in real time system data to be backed up, the excessive situation because the historical data that backup produces takes up room often appears.For example for a DWDM (dense wave division multipurpose) system, every veneer of network element just was reported to webmaster with the data of one time 15 minutes historical performance every 15 minutes, even according to the engineering minimum requirements, keep 3 months History Performance Data in the database, under situation about 32 ripples all being disposed with 100 network elements, the shared space of the data of historical performance also has about 8G.With the data instance of 15 minutes historical performances of transmission network management, network element just regularly reported historical performance one time every 15 minutes, and when the data of 15 minutes historical performances reached 8G and have index, once inserting 10,000 data probably needed about 1 minute.If the user also carries out the inquiry of historical data simultaneously in this 1 minute, and the tables of data of historical performance is locked because of the process that is in the data insertion at this moment, so the user inquiring historical data will be failed.
Like this owing to database table has added index for search efficiency, so the data after database table is excessive are inserted action need addressing in index, this will expend more time, when putting in storage, historical data may cause the failure of the enquiry of historical data of user on network management interface because of the blocked overlong time of table, reduced the availability of management system, and this situation is very common in present network management system.The basic reason that causes these defectives is that existing historical data all only is kept in the tables of data, and this only tables of data is not carried out data management preferably.
Summary of the invention
For overcoming above-mentioned defective, the object of the present invention is to provide the spent time of a kind of minimizing addressing in index, and the method that saves historical data and the device that reduce the data interlock time.
To achieve the above object of the invention, the present invention adopts following technical scheme: this method that saves historical data comprises:
(1) the data volume upper limit of this table is set;
(2) utilize the data volume upper limit of this table to manage data in this table and the dump list.
Wherein, described table is original table that saves historical data; Described dump list is the newly-increased table that saves historical data.
Wherein, described step (2) is specially: judge whether the historical data in this table exceeds the data volume upper limit, historical data among the Ruo Benbiao exceeds the data volume upper limit, then the historical data that exceeds the data volume upper limit segment in this table is inserted into dump list, and deletion from this table.
Wherein the implementation of step (2) in system is: the operation of described step (2) is joined in the regular task list of database, and regularly carried out during the free time in system by regular task list.
After using this method and saving historical data, data query interface and the data query interface of dump list respectively in Ben Biao and dump list the enquiry of historical data of user by this table; With in this table and the historical data that meets Query Result in the dump list merge and to report the user.
The applied hardware of this method need improve on original basis, the device that saves historical data after the improvement, comprise this table that is provided with newly-increased data-interface and data query interface, also comprise a dump list, this dump list is provided with the data query interface, is used at the dump list enquiry of historical data; Be provided with data between this dump list and the Ben Biao and shift interface, be used for the historical data of this table is transferred to dump list.
By the described method of technique scheme, set the data volume upper limit owing to give the data volume of this table of the daily insertion data of historical data, make the data volume in this table maintain a lower level, can effectively solve in this table and insert table long problem locking time that data cause, and the insertion of large data volume table consuming time operation, just historical data is transferred to the operation of dump list by this table, all regularly carry out, can concentrate the moment that moves on to the webmaster free time, so user's enquiry of historical data on network management system just can can't not inquired about because the locked mistake of tables of data is long-time, reduced failed probability greatly, increased the availability of network management system, and be provided with 2 tables of data in the used hardware of this network management system, and two tables of data all have query interface, guaranteed that 2 tables of data can carry out data query simultaneously, guaranteed the speed of data query.
Description of drawings
The flow chart of the method that Fig. 1 saves historical data for the present invention;
The schematic diagram of this table and dump list in the device that Fig. 2 saves historical data for the present invention.
Embodiment
Below in conjunction with accompanying drawing method and the device that the present invention saves historical data is described in detail.
Excessive in order to solve in the prior art network management historical data table, cause that table inserts the data interlock overlong time, and influence the problem of user's normal queries historical data.Method that the present invention saves historical data and device have proposed two method and devices that tables of data saves historical data of a kind of use.
The principle of the method that the present invention saves historical data is when the historical data in this table surpasses certain data volume, just the historical data in this table is transferred in the dump list, and its method flow is described below:
Step 1, for original this table that saves historical data a data volume upper limit is set, to control the data volume size in this table, the value of this data volume upper limit can be specified by system;
Step 2, newly-increased historical data all is inserted into this table;
Step 3, judge that whether the data volume of the historical data in this table surpasses the data volume upper limit, if surpass the data volume upper limit, then is inserted into the historical data of the part that exceeds in the dump list, and deletes from this table.
For above-mentioned method flow is applied in the practice, we use above-mentioned method and save historical data with the specific embodiment that saves as of the historical data of network management system.
At first need to be used for saving historical data jointly for newly-increased dump list of existing this table that saves historical data in the network management system.In this network management system, originally 15 minutes History Performance Data uses a table that is called " pm_cur15p " to preserve, and needing after the improvement increases an extra dump list " pm_cur15p_store ", promptly uses two tables to preserve jointly.The dump list that increases is to use SQL (Structured Query Language, SQL) statement " CREATE TABLE " to create, the dump list that newly adds except that table name claim with this table different, other are as the literary name section, table index is all same with this epiphase.
As shown in Figure 1, after increasing dump list, also need following steps:
101, for original this table that saves historical data a data volume upper limit is set, to control the data volume size in this table, in this network management system, just need set a upper limit to the data volume in this table, we are made as 1,000,000 with the data volume upper limit of this table " pm_cur15p ".
102, use this table to preserve newly-increased historical data.Owing to have only this table that the Q interface of newly-increased historical data is arranged, promptly can not newly-increased historical data be inserted in the dump list.So when using 15 minutes performance datas of the common reservation history of this table " pm_cur15p " and dump list " pm_cur15p_store ", the History Performance Data that the timing of each network element reports all only is inserted in this table " pm_cur15p ".Newly-increased historical data generally is to use SQL statement " INSERT " to be inserted in this table one by one.
103, the webmaster free time, such as point in the late into the night 1, whether the data of making regular check in the deuterzooid table have surpassed the data volume upper limit, if surpassed the data volume upper limit, execution in step 104; If do not surpass the data volume upper limit, flow process finishes.In this network management system, then need to check the data volume of the historical data of a deuterzooid table in 1 of every late into the night, judge whether this data volume surpasses the data volume upper limit.Wherein, can obtain the record count in this table by SQL statement " SELECT COUNT (*) ", the average byte number with record count and every record multiplies each other the data estimator amount then; Regularly method of operating can be saved timer, conserve system resources like this by realizing in the regular task list that the relevant SQL statement of operation is joined database.
104, the part that exceeds in this table is inserted into dump list, and deletion from this table.Historical data in this table can by and dump list between data shift the dump that interface X realizes data.If surpassed 1,000,000 need to be inserted in the dump list " pm_cur15p_store " to the oldest a part of historical data and from this table " pm_cur15p " deletion,, data are deleted from this table generally realize less than 1,000,000 with the data that guarantee this table " pm_cur15p " by SQL statement " DELETE ".If formulating the upper limit is 0, again empty this table " pm_cur15p " after then all being inserted in the dump list " pm_cur15p_store " to the total data of this table " pm_cur15p " at every turn, but do not advise using 0 the upper limit, because if network element has the data of new historical performance to report in dump procedure, at this moment the data of Xin Zeng historical performance may be just directly not deleted by dump, causes losing of data.
Because the existing device that saves historical data does not have dump list, thus just can't utilize said method to save historical data, so the present invention also cooperates said method to design a device that saves historical data.
As shown in Figure 2, this device that saves historical data comprises this table 11 that is provided with newly-increased data-interface Q (this interface is the interface that newly-increased historical data is write this table) and data query interface F1, this this table 11 is exactly an employed database table in the prior art, also comprise a dump list 22, the database table that this dump list 22 is newly-increased, and have the data structure of the preservation data the same with this table 11.This dump list 22 is provided with data query interface F2, is used at dump list 22 enquiry of historical data; Be provided with data between this dump list 22 and the Ben Biao 11 and shift interface X, be used for the historical data of this table 11 is transferred to dump list 22.
Fig. 2 has represented the exemplary plot that common this table of use 11 and dump list 22 save historical data.Dump list 22 generally is to use SQL (Structured Query Language, SQL) statement " CREATE TABLE " is created, this dump list 22 except table name claim with this table 11 different, other fields, as index, constraint etc. are all in full accord with this table 11.
Because employed method and the applied device of this method in the above-mentioned steps, making network management historical data be stored in two gives in the database table, when the user needs enquiry of historical data, just need inquire about the content in this table and the dump list simultaneously and report the user.And shown in Fig. 2, Ben Biao and dump list all have the query interface of using for user inquiring (F1 and F2), and promptly webmaster can be simultaneously from Ben Biao and dump list data query.In this network management system, because use this table " pm_cur15p " and dump list " pm_cur15p_store " to keep 15 minutes History Performance Data jointly simultaneously, so the data of 15 minutes the historical performance that the user need inquire about all exist in two tables, after need inquiring about the data of 15 minutes historical performances that meet customer requirements in two tables respectively, then result's merging is reported the user.The method of inquiring about the content in this table and the dump list simultaneously can merge from the Query Result of this table with from the Query Result of dump list by the keyword among the SQL " UNION ".
Because said method has been set the upper limit to the data volume of this table " pm_cur15p " of the daily insertion data of historical data, can effectively solve toward the middle data volume of this table " pm_cur15p " excessive, cause and show long problem locking time when inserting data, when the data volume upper limit is 1,000,000, data volume in this table " pm_cur15p " the chances are 50M, 10,000 data of each past this table " pm_cur15p " insertion need only the time less than 1 second, simultaneously the insertion of more mass data table consuming time is operated, it is the moment that the data dump of this table " pm_cur15p " moves on to 1 network management system free time of the late into the night in the operation set of dump list " pm_cur15p_store ", so the user at ordinary times on webmaster 15 minutes performance data failed probability of query history reduce greatly, increased the availability of network management interface.
Claims (6)
1. method that saves historical data is characterized in that: may further comprise the steps:
(1) the data volume upper limit of this table is set;
(2) utilize the data volume upper limit of this table to manage data in this table and the dump list.
2. according to the described method that saves historical data of claim 1, it is characterized in that: described table is original table that saves historical data; Described dump list is the newly-increased table that saves historical data.
3. according to the described method that saves historical data of claim 2, it is characterized in that: described step (2) is specially: judge whether the historical data in this table exceeds the data volume upper limit, historical data among the Ruo Benbiao exceeds the data volume upper limit, then the historical data that exceeds the data volume upper limit segment in this table is inserted into dump list, and deletion from this table.
4. according to the described method that saves historical data of claim 3, it is characterized in that: the operation of described step (2) is joined in the regular task list of database, and regularly carry out during the free time in system by regular task list.
5. according to the described method that saves historical data of claim 3, it is characterized in that: data query interface and the data query interface of dump list respectively in Ben Biao and dump list the enquiry of historical data of user by this table; With in this table and the historical data that meets Query Result in the dump list merge and to report the user.
6. a device that saves historical data comprises this table that is provided with newly-increased data-interface and data query interface, and it is characterized in that: also comprise a dump list, this dump list is provided with the data query interface, is used at the dump list enquiry of historical data; Be provided with data between this dump list and the Ben Biao and shift interface, be used for the historical data of this table is transferred to dump list.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200610170701A CN101207513B (en) | 2006-12-22 | 2006-12-22 | Apparatus and method for saving historical data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200610170701A CN101207513B (en) | 2006-12-22 | 2006-12-22 | Apparatus and method for saving historical data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101207513A true CN101207513A (en) | 2008-06-25 |
| CN101207513B CN101207513B (en) | 2012-09-05 |
Family
ID=39567417
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200610170701A Expired - Fee Related CN101207513B (en) | 2006-12-22 | 2006-12-22 | Apparatus and method for saving historical data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101207513B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101344893B (en) * | 2008-07-17 | 2010-06-02 | 中兴通讯股份有限公司 | History data access method and apparatus |
| CN102541908A (en) * | 2010-12-27 | 2012-07-04 | 沈阳中科博微自动化技术有限公司 | Log database system sub-volume management method |
| CN103473271A (en) * | 2013-08-20 | 2013-12-25 | 苏州迈科网络安全技术股份有限公司 | Optimized storing method for mass data |
| CN104636353A (en) * | 2013-11-07 | 2015-05-20 | 中国科学院沈阳自动化研究所 | High-performance log record query method for integrated circuit production line carrying system |
| CN105095393A (en) * | 2015-06-30 | 2015-11-25 | 努比亚技术有限公司 | Method and device for data storage |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1249581C (en) * | 2002-11-18 | 2006-04-05 | 华为技术有限公司 | A hot backup data migration method |
| US20060136525A1 (en) * | 2004-12-21 | 2006-06-22 | Jens-Peter Akelbein | Method, computer program product and mass storage device for dynamically managing a mass storage device |
-
2006
- 2006-12-22 CN CN200610170701A patent/CN101207513B/en not_active Expired - Fee Related
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101344893B (en) * | 2008-07-17 | 2010-06-02 | 中兴通讯股份有限公司 | History data access method and apparatus |
| CN102541908A (en) * | 2010-12-27 | 2012-07-04 | 沈阳中科博微自动化技术有限公司 | Log database system sub-volume management method |
| CN103473271A (en) * | 2013-08-20 | 2013-12-25 | 苏州迈科网络安全技术股份有限公司 | Optimized storing method for mass data |
| CN104636353A (en) * | 2013-11-07 | 2015-05-20 | 中国科学院沈阳自动化研究所 | High-performance log record query method for integrated circuit production line carrying system |
| CN104636353B (en) * | 2013-11-07 | 2017-09-15 | 中国科学院沈阳自动化研究所 | The high-performance log record queries method of integrated circuit production line handling system |
| CN105095393A (en) * | 2015-06-30 | 2015-11-25 | 努比亚技术有限公司 | Method and device for data storage |
| CN105095393B (en) * | 2015-06-30 | 2018-11-16 | 努比亚技术有限公司 | A kind of date storage method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101207513B (en) | 2012-09-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20080222111A1 (en) | Database system with dynamic database caching | |
| US7783615B1 (en) | Apparatus and method for building a file system index | |
| Lampson et al. | A new presumed commit optimization for two phase commit | |
| US5799322A (en) | System and method for stopping updates at a specified timestamp in a remote duplicate database facility | |
| US5799323A (en) | Remote duplicate databased facility with triple contingency protection | |
| US11442961B2 (en) | Active transaction list synchronization method and apparatus | |
| CN100565512C (en) | Eliminate the system and method for redundant file in the document storage system | |
| CN104679772A (en) | Method, device, equipment and system for deleting files in distributed data warehouse | |
| CN103024329B (en) | A kind of memory management method of monitoring video | |
| CN110990432B (en) | Device and method for synchronizing distributed cache clusters across machine room | |
| CN101739313B (en) | Method for protecting and restoring continuous data | |
| CN101207513B (en) | Apparatus and method for saving historical data | |
| KR102038529B1 (en) | System for processing real-time data modification of in-memory database | |
| US6691115B2 (en) | System and method for purging database update image files after completion of associated transactions for a database replication system with multiple audit logs | |
| CN106445738A (en) | Database backup method and device | |
| US8712966B1 (en) | Backup and recovery of distributed storage areas | |
| CN111125260A (en) | Data synchronization method and system based on SQL Server | |
| CN103186624A (en) | Data synchronization method and data synchronization device | |
| CN102521269A (en) | Index-based computer continuous data protection method | |
| CN101408889A (en) | Method, apparatus and system for monitoring performance | |
| US20220035786A1 (en) | Distributed database management system with dynamically split b-tree indexes | |
| CN102411639A (en) | Multi-copy storage management method and system of metadata | |
| CN106790422A (en) | A kind of data buffer storage cluster and data retrieval method for WEB application | |
| CN103164528A (en) | Index establishing method for audio/video data | |
| CN107291881A (en) | Massive logs storage and querying method based on HBase |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120905 Termination date: 20151222 |
|
| EXPY | Termination of patent right or utility model |