CN1904882A - Compression method of database near-line data - Google Patents
Compression method of database near-line data Download PDFInfo
- Publication number
- CN1904882A CN1904882A CN 200510087061 CN200510087061A CN1904882A CN 1904882 A CN1904882 A CN 1904882A CN 200510087061 CN200510087061 CN 200510087061 CN 200510087061 A CN200510087061 A CN 200510087061A CN 1904882 A CN1904882 A CN 1904882A
- Authority
- CN
- China
- Prior art keywords
- data
- compression
- database
- line data
- compression method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a compressing method for database near line. It could effectively expand the storing capacity and the useful time of near line database. It could be benefit to improve accessing efficiency of online data and decrease the possibility of inquiring to off-line data.
Description
Technical field
The invention belongs to the database compress technique.
Background technology
Data of database by its user mode, generally is divided into following three kinds: online data, and promptly high frequency is the data of inquiry in real time; Near-line data, the i.e. lower data of visiting frequency; Off-line data is promptly filed stored history, and these data are no longer often inquired about.
For near-line data, existing database technology does not provide compression method as yet, and the present invention is a kind of technological innovation to this field.
Near-line data is compressed, can enlarge near line data of database capacity, help to prolong its life cycle, thereby can alleviate inquiry pressure, improve its response time and service efficiency, also can reduce query demand the filing historical data to online data.
Summary of the invention
Of the present inventionly to the effect that provide data compression method, when data base management system (DBMS) during swap data, is carried out compression/decompression processes to data block between core buffer and disk file for the near-line data of database.
Description of drawings
Fig. 1: the synoptic diagram of general data base management system
Fig. 2: synoptic diagram with data base management system (DBMS) of data compression function
Embodiment
Core technology of the present invention is that data compression technique is embedded in the data base management system (DBMS).When data base management system (DBMS) will write disk to the data block in the buffer memory, earlier data block is done compression and handle, and then write disk; Otherwise, when data base management system (DBMS) with data block when disk reads in buffer memory, data block is made corresponding decompression.
The synoptic diagram of general data base management system as shown in Figure 1.
The related physical data file of data base management system (DBMS) generally has four types: Parameter File, control documents, data file and journal file.
The data volume of Parameter File and control documents is not very big, does not generally consider their compression is handled.The data volume of data file is very big, can increase compression/decompression processes when the read-write of data block.The data volume of journal file generally also is very big, but database as the near-line data processing, consider that its frequency to data database data retouching operation is not very high, the data volume of journal file also may not be very big, so the compression/decompression processes when the journal file data block read and write can be used as option.This option can be selected by the system manager when data base management system (DBMS) is installed.
The synoptic diagram of the data base management system (DBMS) after the increase compression/decompression processes module as shown in Figure 2.
Compare with original data base management system (DBMS), after the increase compression/decompression processes, the response time of data base management system (DBMS) data access may have some prolongations.But consider following some factor, this delay is not too large.Because: the one, to the compression/decompression processes of data block, follow the read-write operation of data block equally, finish by background process, to a certain extent can parallel processing to data write and two kinds of processing of compression/de-compression; The 2nd, the visiting frequency for near-line data is significantly less than online data usually, and there is abundant relatively processing power in system.
Chief value of the present invention is:
1, under the situation that does not increase the memory device input, increases the memory capacity of near-line data greatly.
2, to the queried access of near-line data, can use the application software identical, separately the Application and Development system with online data.
Claims (3)
1, a kind of data compression method in near-line data storehouse, its principal character is: when data base management system (DBMS) during swap data, is carried out compression/decompression processes to data block between memory cache district and disk file.
2, the data compression method in near-line data as claimed in claim 1 storehouse carries out compression/decompression processes to the data file of database.
3, the data compression method in near-line data as claimed in claim 1 storehouse carries out compression/decompression processes to journal file or other file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200510087061 CN1904882A (en) | 2005-07-26 | 2005-07-26 | Compression method of database near-line data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200510087061 CN1904882A (en) | 2005-07-26 | 2005-07-26 | Compression method of database near-line data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN1904882A true CN1904882A (en) | 2007-01-31 |
Family
ID=37674148
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200510087061 Pending CN1904882A (en) | 2005-07-26 | 2005-07-26 | Compression method of database near-line data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1904882A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101446951A (en) * | 2007-11-20 | 2009-06-03 | 通用电气公司 | Compressed data storage to provide recent and summary data |
| CN101847162A (en) * | 2010-05-25 | 2010-09-29 | 中国电力科学研究院 | Electric system simulation centre data processing method based on file and database exchange |
| CN101576855B (en) * | 2009-06-19 | 2011-05-04 | 深圳市科陆电子科技股份有限公司 | Data storing system and method based on cache |
| CN104462176A (en) * | 2013-09-19 | 2015-03-25 | 国际商业机器公司 | System and method for data access using decompression maps |
| CN107436738A (en) * | 2017-08-17 | 2017-12-05 | 北京理工大学 | A kind of date storage method and system |
| US10432217B2 (en) | 2016-06-28 | 2019-10-01 | International Business Machines Corporation | Page filtering via compression dictionary filtering |
-
2005
- 2005-07-26 CN CN 200510087061 patent/CN1904882A/en active Pending
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101446951A (en) * | 2007-11-20 | 2009-06-03 | 通用电气公司 | Compressed data storage to provide recent and summary data |
| CN101576855B (en) * | 2009-06-19 | 2011-05-04 | 深圳市科陆电子科技股份有限公司 | Data storing system and method based on cache |
| CN101847162A (en) * | 2010-05-25 | 2010-09-29 | 中国电力科学研究院 | Electric system simulation centre data processing method based on file and database exchange |
| US10437826B2 (en) | 2013-09-19 | 2019-10-08 | International Business Machines Corporation | Data access performance using decompression maps |
| CN104462176A (en) * | 2013-09-19 | 2015-03-25 | 国际商业机器公司 | System and method for data access using decompression maps |
| US10437827B2 (en) | 2013-09-19 | 2019-10-08 | International Business Machines Corporation | Data access performance using decompression maps |
| CN104462176B (en) * | 2013-09-19 | 2018-05-11 | 国际商业机器公司 | System and method for carrying out data access by using decompression contract drawing |
| US10432217B2 (en) | 2016-06-28 | 2019-10-01 | International Business Machines Corporation | Page filtering via compression dictionary filtering |
| US10439638B2 (en) | 2016-06-28 | 2019-10-08 | International Business Machines Corporation | Page filtering via compression dictionary filtering |
| US10903850B2 (en) | 2016-06-28 | 2021-01-26 | International Business Machines Corporation | Page filtering via compression dictionary filtering |
| US10903851B2 (en) | 2016-06-28 | 2021-01-26 | International Business Machines Corporation | Page filtering via compression dictionary filtering |
| CN107436738A (en) * | 2017-08-17 | 2017-12-05 | 北京理工大学 | A kind of date storage method and system |
| CN107436738B (en) * | 2017-08-17 | 2019-10-25 | 北京理工大学 | A kind of date storage method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dong et al. | Optimizing Space Amplification in RocksDB. | |
| CN103488709B (en) | A kind of index establishing method and system, search method and system | |
| CN103116661B (en) | A kind of data processing method of database | |
| CN1904882A (en) | Compression method of database near-line data | |
| CN102609360A (en) | Data processing method, data processing device and data processing system | |
| US20080162523A1 (en) | Techniques for selective compression of database information | |
| CN103020205A (en) | Compression and decompression method based on hardware accelerator card on distributive-type file system | |
| US9996557B2 (en) | Database storage system based on optical disk and method using the system | |
| CN105027122A (en) | Compression and deduplication layered driver | |
| CN108427539A (en) | Offline duplicate removal compression method, device and the readable storage medium storing program for executing of buffer memory device data | |
| CN101493821A (en) | Data caching method and device | |
| CN101625703B (en) | Method and system for merging logs of memory database | |
| WO2014188528A1 (en) | Memory device, computer system, and memory device control method | |
| CN101221585A (en) | Data storage method and device | |
| CN105094709A (en) | Dynamic data compression method for solid-state disc storage system | |
| CN111475507B (en) | Key value data indexing method for workload adaptive single-layer LSMT | |
| CN110908608A (en) | Storage space saving method and system | |
| CN105378716A (en) | Method and apparatus for conversion of data storage formats | |
| CN102147768A (en) | Memory, solid caching system and caching data processing method | |
| CN103488685A (en) | Fragmented-file storage method based on distributed storage system | |
| CN107656971A (en) | A kind of intelligent grid collection Monitoring Data storage method based on Redis | |
| WO2023185111A1 (en) | Quick access method and device for data file | |
| WO2023024459A1 (en) | Data processing method and device | |
| CN101739359B (en) | Storage device, access method for mobile terminal and data, and frequency modulation method | |
| CN103020077A (en) | Method for managing memory of real-time database of power system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |