CN101290628A - Data file updating storage method - Google Patents

Data file updating storage method Download PDF

Info

Publication number
CN101290628A
CN101290628A CNA2008100678739A CN200810067873A CN101290628A CN 101290628 A CN101290628 A CN 101290628A CN A2008100678739 A CNA2008100678739 A CN A2008100678739A CN 200810067873 A CN200810067873 A CN 200810067873A CN 101290628 A CN101290628 A CN 101290628A
Authority
CN
China
Prior art keywords
data file
deblocking
data
check code
file
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CNA2008100678739A
Other languages
Chinese (zh)
Other versions
CN101290628B (en
Inventor
梁秀帅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN2008100678739A priority Critical patent/CN101290628B/en
Publication of CN101290628A publication Critical patent/CN101290628A/en
Application granted granted Critical
Publication of CN101290628B publication Critical patent/CN101290628B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a method for updating and storing a data file. The method comprises the following processing steps that: (1) a to-be-stored data file is divided into a plurality of data blocks, and a check code of each data block is respectively calculated and is attached to a fixed region of the data file; (2) the check code corresponding to each data block in an updated and stored new data file is respectively compared with the check code of a corresponding data block in an original data file; (3) the data blocks with different check codes are updated and stored, and the check codes of the data file are updated and stored. The method can reduce unnecessary data storage to increase storage efficiency and reduce data storage time, and be suitable to be used in the technical fields such as the data storage and the backup of communication equipment.

Description

A kind of data file updating storage method
Technical field
The present invention relates to a kind of areas of information technology, relate in particular to the data storage technology of the communications field.
Background technology
Along with the development of infotech, multimedia application extension is to each field, and this makes the data volume of communication increasing, and the data that communication causes transmit more and more frequent, needs the data of storage more and more.Data storage comprises two aspects, the one, preserve a large amount of new datas, and the 2nd, existing data need frequent updating.
At present, normally with the timestamp judgment data of file no change is arranged for Data Update, this method is simple, but this may cause following problem:
(1) file content does not have actual change, still needs to have carried out to upgrade to preserve;
(2) file has only partial content to revise, and other parts do not change, and still need whole file is covered preservation.
So, improve data storage efficiency, how to reduce Data Update as far as possible and deposit operation, be an important problem.
Summary of the invention
The technical matters that the present invention solves is to have proposed a kind of data file updating storage method, has improved the efficient of data file update storage, reduces unnecessary data and upgrades, thereby reduce time data memory.
The data file updating storage method that the present invention proposes comprises following processing procedure:
1) data file to be stored is cut into a plurality of deblockings, each deblocking is distinguished the calculation check sign indicating number, and check code is attached to the fixed area of data file;
2) check code with deblocking corresponding in each deblocking corresponding check sign indicating number in the new data file of updated stored and the former data file compares respectively;
3) carry out updated stored for the different deblocking of check code, and the check code of updated stored data file.
Preferably, be specially in the described step 1) data file to be stored is cut into a plurality of deblockings by fixed size, the size of last deblocking is the actual size after the cutting.
Preferably, carry out updated stored for the different deblocking of check code in the described step 3) and specifically comprise following processing procedure:
31) when the check code under the deblocking of new data file and former data file correspondence is inequality, upgrade the deblocking of former data file with the deblocking of new data file;
32) when the new data file does not have the deblocking of former data file correspondence, delete this deblocking of former data file;
33) when former data file does not have the deblocking of new data file correspondence, in former data file, preserve this deblocking.
Preferably, be specially in the described step 1) each deblocking is calculated its cyclic redundancy check (CRC) code respectively.
Preferably, be specially the head that check code is attached to data file in the described step 1).
The data file updating storage method that adopts the present invention to propose, because the data file has been carried out pre-service, in pre-service to the data file calculation check sign indicating number that carried out piecemeal, make the deblocking that when updated stored, only need upgrade variation, therefore can reduce unnecessary data preserves, improve storage efficiency, reduce time data memory.
Description of drawings
Fig. 1 is a data file updating storage method process flow diagram of the present invention;
Fig. 2 is the structural representation of data file updating storage method preferred embodiment backup data file of the present invention;
Fig. 3 is the preservation processing flow chart of data file updating storage method preferred embodiment backup data file of the present invention.
Embodiment
Below in conjunction with also in conjunction with the accompanying drawings technical scheme of the present invention being elaborated by embodiment.
Core concept of the present invention is: data file is divided into the plurality of data piecemeal, calculate the check code (for example :) of each deblocking with the CRC sign indicating number of CRC algorithm computation deblocking, and the check code of all deblockings is attached to the fixed area of data file, in storage during data, judge whether the needs updated stored according to the check code of each deblocking of data file.For the piecemeal of data file for simplicity, can carry out piecemeal by fixed size.
Updated stored flow process as shown in Figure 1:
The first step: obj ect file (new file) is divided into the plurality of data piecemeal by fixed size, for example, the file that a size is M, be divided into N data piecemeal, wherein preceding N-1 data block size is identical, and last deblocking size is the actual size of residue file.Then, each deblocking is calculated CRC sign indicating number (during preservation the CRC sign indicating number of each deblocking concentrated and be kept at a fixed area, such as, with the additional head that has file of CRC sign indicating number of all deblockings of calculating).
Second step: when preserving file, judge whether there has been this document (file of the same name) in the storage space earlier,, then preserve whole file (comprising the CRC sign indicating number) if do not have, otherwise, the 3rd step;
The 3rd step: the CRC sign indicating number of newer one by one file and each deblocking of ancient deed, do different operating according to different results;
The 4th step:, then do not upgrade the data of this deblocking, more next deblocking if the CRC sign indicating number of two data piecemeals is identical;
If the CRC sign indicating number of two data piecemeals is inequality, upgrade the deblocking of ancient deed with this deblocking of new file, then, more next deblocking;
If the deblocking that new file has, and old file would not have, and then preserves this deblocking and deblocking afterwards in storage space;
If the deblocking that old file has, and new file would not have, and then deletes the deblocking of ancient deed.
The 5th step: the CRC sign indicating number that upgrades ancient deed with the CRC sign indicating number of new file.
Further specify the implementation procedure of updated stored below again by the embodiment of a communication facilities.
This embodiment is that example describes with the data backup function of certain veneer.The data backup function regularly or trigger veneer under certain conditions data are backed up.Backed up data is kept among the Flash, according to fixed format preserves, and the length of backup file is constant, preserves file structure as shown in Figure 2.The information of backup comprises: single board information, operation system information, configuration information, warning information and register information, the length scale of each several part is respectively the 150K byte, 200K byte, 600K byte, 550K byte and 300K byte, solid line indication each several part size among the figure.
After veneer produces Backup Data for the first time, before preservation,, be that area size is divided into 18 districts (dotted line is indicated different subregions among Fig. 2) with Backup Data, and use each district CRC sign indicating number of CRC32 algorithm computation with the 100K byte according to treatment step of the present invention.Be checked through then and do not preserve backup data file, together preserve with file before just the CRC sign indicating number in all districts being attached to file.
When carrying out Backup Data later on, before preservation, with data partition, and calculate the CRC sign indicating number in each district according to the 100K byte-sized.Check then to have backup data file, whether consistent with regard to each regional CRC sign indicating number of comparison document.Suppose below to have revised configuration, make the 7th district's Backup Data that change take place, simultaneously, register configuration changes, and the 16th district's Backup Data changed, so the CRC sign indicating number that the CRC sign indicating numbers of these two district's calculating and original file are preserved is inconsistent.
Figure 3 shows that the step of handling according to the present invention:
1. calculate the CRC sign indicating number of each subregion correspondence of new Backup Data.
2. judge to have had backup data files, subregion compares the CRC sign indicating number one by one
3. the CRC sign indicating number of new and old first subregion, unanimity, relatively second, go down successively, to the 6th, relatively the 7th then.
4. the CRC sign indicating number of the 7th subregion is through relatively more different, and then the subregion content update with existing file is the subregion content of new Backup Data, compares the 8th subregion then
5. the CRC sign indicating number of the 8th to the 15th subregion does not change, relatively the 16th subregion.
6. the CRC sign indicating number of the 16th subregion is through relatively more different, and then the subregion content update with existing file is the subregion content of new Backup Data, compares the 17th then
7. the CRC sign indicating number of the 17th and 18 subregions does not change.
8. upgrade the CRC sign indicating number of preserving backup data file, storing process finishes.
Above content be in conjunction with concrete preferred implementation to further describing that the present invention did, can not assert that concrete enforcement of the present invention is confined to these explanations.For the general technical staff of the technical field of the invention, without departing from the inventive concept of the premise, can also make some simple deduction or replace, all should be considered as belonging to protection scope of the present invention.

Claims (5)

1. a data file updating storage method is characterized in that, described method comprises following processing procedure:
1) data file to be stored is cut into a plurality of deblockings, each deblocking is distinguished the calculation check sign indicating number, and check code is attached to the fixed area of data file;
2) check code with deblocking corresponding in each deblocking corresponding check sign indicating number in the new data file of updated stored and the former data file compares respectively;
3) carry out updated stored for the different deblocking of check code, and the check code of updated stored data file.
2. data file updating storage method according to claim 1, it is characterized in that, be specially in the described step 1) data file to be stored is cut into a plurality of deblockings by fixed size, the size of last deblocking is the actual size after the cutting.
3. data file updating storage method according to claim 1 is characterized in that, carries out updated stored for the different deblocking of check code in the described step 3) and specifically comprises following processing procedure:
31) when the check code under the deblocking of new data file and former data file correspondence is inequality, upgrade the deblocking of former data file with the deblocking of new data file;
32) when the new data file does not have the deblocking of former data file correspondence, delete this deblocking of former data file;
33) when former data file does not have the deblocking of new data file correspondence, in former data file, preserve this deblocking.
4. according to each described data file updating storage method of claim 1-3, it is characterized in that, be specially in the described step 1) each deblocking is calculated its cyclic redundancy check (CRC) code respectively.
5. according to each described data file updating storage method of claim 1-3, it is characterized in that, be specially the head that check code is attached to data file in the described step 1).
CN2008100678739A 2008-06-17 2008-06-17 Data file updating storage method Active CN101290628B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2008100678739A CN101290628B (en) 2008-06-17 2008-06-17 Data file updating storage method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2008100678739A CN101290628B (en) 2008-06-17 2008-06-17 Data file updating storage method

Publications (2)

Publication Number Publication Date
CN101290628A true CN101290628A (en) 2008-10-22
CN101290628B CN101290628B (en) 2010-06-16

Family

ID=40034886

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2008100678739A Active CN101290628B (en) 2008-06-17 2008-06-17 Data file updating storage method

Country Status (1)

Country Link
CN (1) CN101290628B (en)

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101807207A (en) * 2010-03-22 2010-08-18 北京大用科技有限责任公司 Method for sharing document based on content difference comparison
CN102073560A (en) * 2011-01-17 2011-05-25 北京深思洛克软件技术股份有限公司 Data backup method and device
CN102202098A (en) * 2011-05-25 2011-09-28 成都市华为赛门铁克科技有限公司 Data processing method and device
CN101615145B (en) * 2009-07-24 2011-12-07 中兴通讯股份有限公司 Method and device for improving reliability of data caching of memorizer
CN102567131A (en) * 2011-12-27 2012-07-11 创新科存储技术有限公司 Asynchronous mirroring method
CN102714789A (en) * 2011-04-19 2012-10-03 华为终端有限公司 Method for backuping and recovering data of mobile terminal and mobile terminal thereof
CN102799598A (en) * 2011-05-25 2012-11-28 英业达股份有限公司 Data recovery method for deleting repeated data
CN103166992A (en) * 2011-12-14 2013-06-19 中国电信股份有限公司 Method and system of fast file updating applied to cloud storage
CN103377251A (en) * 2012-04-27 2013-10-30 百度在线网络技术(北京)有限公司 File comparison method and device for HDFS (Hadoop Distributed File System)
CN104123280A (en) * 2013-04-24 2014-10-29 ***股份有限公司 File comparison method and device
CN104142954A (en) * 2013-05-10 2014-11-12 中国电信股份有限公司 Data sheet comparing and updating method and device based on frequentness partition
CN101510175B (en) * 2009-04-02 2015-06-03 北京中星微电子有限公司 Method for updating target data to memory and apparatus thereof
CN104778223A (en) * 2015-03-25 2015-07-15 广州多益网络科技有限公司 Method and device for establishing data model of hand-tour house
CN105095300A (en) * 2014-05-16 2015-11-25 阿里巴巴集团控股有限公司 Method and system for database backup
CN105406934A (en) * 2015-10-27 2016-03-16 上海斐讯数据通信技术有限公司 Passive optical network system main and standby control board synchronization method and system
CN101840363B (en) * 2009-11-10 2016-03-30 创新科存储技术有限公司 A kind of file block comparative approach and device
CN105656802A (en) * 2016-03-09 2016-06-08 成都爆米花信息技术有限公司 Data synchronization method
CN105955670A (en) * 2016-05-12 2016-09-21 武汉斗鱼网络科技有限公司 Method and system for checking repeated list data in application
CN103166992B (en) * 2011-12-14 2016-12-14 中国电信股份有限公司 It is applied to rapid file update method and the system of cloud storage
CN106528125A (en) * 2016-10-26 2017-03-22 腾讯科技(深圳)有限公司 Data file incremental updating method, server, client and system
CN107015882A (en) * 2016-12-26 2017-08-04 阿里巴巴集团控股有限公司 A kind of block data method of calibration and device
CN107291572A (en) * 2017-06-19 2017-10-24 郑州云海信息技术有限公司 The storage method and device of a kind of image file
CN107704342A (en) * 2017-09-26 2018-02-16 郑州云海信息技术有限公司 A kind of snap copy method, system, device and readable storage medium storing program for executing
CN108304583A (en) * 2018-03-06 2018-07-20 北京顶象技术有限公司 A kind of file collection method and device
CN110515964A (en) * 2019-08-30 2019-11-29 百度在线网络技术(北京)有限公司 A kind of file updating method, device, electronic equipment and medium
CN111104248A (en) * 2018-10-26 2020-05-05 伊姆西Ip控股有限责任公司 Method, apparatus and computer program product for verifying cached files
CN111930560A (en) * 2020-06-29 2020-11-13 东风汽车集团有限公司 ECU self-learning data backup method and system
WO2021070135A1 (en) * 2019-10-12 2021-04-15 International Business Machines Corporation Updating and implementing document from audio proceeding
CN117472291A (en) * 2023-12-27 2024-01-30 苏州元脑智能科技有限公司 Data block verification method and device, storage medium and electronic equipment

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060022903A (en) * 2004-09-08 2006-03-13 엘지전자 주식회사 Method for processing si channel list in cable digital tv using profile 1

Cited By (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101510175B (en) * 2009-04-02 2015-06-03 北京中星微电子有限公司 Method for updating target data to memory and apparatus thereof
CN101615145B (en) * 2009-07-24 2011-12-07 中兴通讯股份有限公司 Method and device for improving reliability of data caching of memorizer
CN101840363B (en) * 2009-11-10 2016-03-30 创新科存储技术有限公司 A kind of file block comparative approach and device
CN101807207A (en) * 2010-03-22 2010-08-18 北京大用科技有限责任公司 Method for sharing document based on content difference comparison
CN102073560A (en) * 2011-01-17 2011-05-25 北京深思洛克软件技术股份有限公司 Data backup method and device
CN102714789B (en) * 2011-04-19 2014-04-02 华为终端有限公司 Method for backuping and recovering data of mobile terminal and mobile terminal thereof
US10095715B2 (en) 2011-04-19 2018-10-09 Huawei Device (Dongguan) Co., Ltd. Data backup and recovery method for mobile terminal and mobile terminal
CN102714789A (en) * 2011-04-19 2012-10-03 华为终端有限公司 Method for backuping and recovering data of mobile terminal and mobile terminal thereof
CN102799598A (en) * 2011-05-25 2012-11-28 英业达股份有限公司 Data recovery method for deleting repeated data
CN102202098A (en) * 2011-05-25 2011-09-28 成都市华为赛门铁克科技有限公司 Data processing method and device
CN103166992A (en) * 2011-12-14 2013-06-19 中国电信股份有限公司 Method and system of fast file updating applied to cloud storage
CN103166992B (en) * 2011-12-14 2016-12-14 中国电信股份有限公司 It is applied to rapid file update method and the system of cloud storage
CN102567131A (en) * 2011-12-27 2012-07-11 创新科存储技术有限公司 Asynchronous mirroring method
CN103377251B (en) * 2012-04-27 2017-05-10 百度在线网络技术(北京)有限公司 File comparison method and device for HDFS (Hadoop Distributed File System)
CN103377251A (en) * 2012-04-27 2013-10-30 百度在线网络技术(北京)有限公司 File comparison method and device for HDFS (Hadoop Distributed File System)
CN104123280A (en) * 2013-04-24 2014-10-29 ***股份有限公司 File comparison method and device
CN104123280B (en) * 2013-04-24 2017-08-25 ***股份有限公司 File comparison method and equipment
CN104142954A (en) * 2013-05-10 2014-11-12 中国电信股份有限公司 Data sheet comparing and updating method and device based on frequentness partition
CN104142954B (en) * 2013-05-10 2017-11-21 中国电信股份有限公司 A kind of tables of data based on frequency subregion compares update method and device
CN105095300A (en) * 2014-05-16 2015-11-25 阿里巴巴集团控股有限公司 Method and system for database backup
CN104778223A (en) * 2015-03-25 2015-07-15 广州多益网络科技有限公司 Method and device for establishing data model of hand-tour house
CN104778223B (en) * 2015-03-25 2018-08-21 广州多益网络股份有限公司 The data model method for building up and device in a kind of hand trip house
CN105406934A (en) * 2015-10-27 2016-03-16 上海斐讯数据通信技术有限公司 Passive optical network system main and standby control board synchronization method and system
CN105656802A (en) * 2016-03-09 2016-06-08 成都爆米花信息技术有限公司 Data synchronization method
CN105955670A (en) * 2016-05-12 2016-09-21 武汉斗鱼网络科技有限公司 Method and system for checking repeated list data in application
CN106528125A (en) * 2016-10-26 2017-03-22 腾讯科技(深圳)有限公司 Data file incremental updating method, server, client and system
CN107015882A (en) * 2016-12-26 2017-08-04 阿里巴巴集团控股有限公司 A kind of block data method of calibration and device
CN107015882B (en) * 2016-12-26 2019-11-22 阿里巴巴集团控股有限公司 A kind of block data method of calibration and device
US10560267B2 (en) 2016-12-26 2020-02-11 Alibaba Group Holding Limited Method and apparatus for verifying block data in a blockchain
US10790981B2 (en) 2016-12-26 2020-09-29 Alibaba Group Holding Limited Method and apparatus for verifying block data in a blockchain
CN107291572A (en) * 2017-06-19 2017-10-24 郑州云海信息技术有限公司 The storage method and device of a kind of image file
CN107704342A (en) * 2017-09-26 2018-02-16 郑州云海信息技术有限公司 A kind of snap copy method, system, device and readable storage medium storing program for executing
CN108304583A (en) * 2018-03-06 2018-07-20 北京顶象技术有限公司 A kind of file collection method and device
CN111104248A (en) * 2018-10-26 2020-05-05 伊姆西Ip控股有限责任公司 Method, apparatus and computer program product for verifying cached files
CN110515964A (en) * 2019-08-30 2019-11-29 百度在线网络技术(北京)有限公司 A kind of file updating method, device, electronic equipment and medium
GB2604507A (en) * 2019-10-12 2022-09-07 Ibm Updating and implementing document from audio proceeding
WO2021070135A1 (en) * 2019-10-12 2021-04-15 International Business Machines Corporation Updating and implementing document from audio proceeding
CN114450747A (en) * 2019-10-12 2022-05-06 国际商业机器公司 Updating and implementing documents from audio conferences
US11354920B2 (en) 2019-10-12 2022-06-07 International Business Machines Corporation Updating and implementing a document from an audio proceeding
CN114450747B (en) * 2019-10-12 2022-11-22 国际商业机器公司 Method, system, and computer-readable medium for updating documents based on audio files
CN111930560A (en) * 2020-06-29 2020-11-13 东风汽车集团有限公司 ECU self-learning data backup method and system
CN111930560B (en) * 2020-06-29 2023-02-28 东风汽车集团有限公司 ECU self-learning data backup method and system
CN117472291A (en) * 2023-12-27 2024-01-30 苏州元脑智能科技有限公司 Data block verification method and device, storage medium and electronic equipment
CN117472291B (en) * 2023-12-27 2024-03-22 苏州元脑智能科技有限公司 Data block verification method and device, storage medium and electronic equipment

Also Published As

Publication number Publication date
CN101290628B (en) 2010-06-16

Similar Documents

Publication Publication Date Title
CN101290628B (en) Data file updating storage method
CN105009067B (en) Managing operations on units of stored data
US7840774B2 (en) Compressibility checking avoidance
CN101326492A (en) Methods and systems for updating content including a compressed version
CN107046812A (en) A kind of data save method and device
CN110941514B (en) Data backup method, data recovery method, computer equipment and storage medium
KR20150125010A (en) Managing operations on stored data units
CN104142954A (en) Data sheet comparing and updating method and device based on frequentness partition
CN103248369A (en) Compression system and method based on FPFA (Field Programmable Gate Array)
CN107590019A (en) A kind of method and apparatus of data storage
CN102156719A (en) Methods for encoding and decoding observed value file of global positioning system
CN106874399A (en) One kind networking standby system and backup method
KR102275240B1 (en) Managing operations on stored data units
KR20180021074A (en) Perform multidimensional navigation, content-related retrieval, and keyword-based navigation and retrieval using a default data sheave for lossless data
CN101599080A (en) A kind of organization system of Backup Data and method
KR20180094004A (en) Reduction of data and audio data stored on a block processing storage system
CN110083487A (en) A kind of reference data block fragment removing method and system based on data locality
CN101174294B (en) Software protection method based on version incremental backup
CN108959614A (en) A kind of snapshot management method, system, device, equipment and readable storage medium storing program for executing
CN107085603A (en) A kind of data processing method and device
CN111078652A (en) Filing and compressing method and device for logistics box codes
CN100551066C (en) The implementation method of encoder and adaptive arithmetic code and device
US6226411B1 (en) Method for data compression and restoration
CN114546886A (en) Space recovery method of value log system
CN113407577B (en) Query method and device for kafka data and computer readable storage medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant