CN101814045A - Data organization method for backup services - Google Patents

Data organization method for backup services Download PDF

Info

Publication number
CN101814045A
CN101814045A CN 201010152397 CN201010152397A CN101814045A CN 101814045 A CN101814045 A CN 101814045A CN 201010152397 CN201010152397 CN 201010152397 CN 201010152397 A CN201010152397 A CN 201010152397A CN 101814045 A CN101814045 A CN 101814045A
Authority
CN
China
Prior art keywords
data
backup
space
storage server
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN 201010152397
Other languages
Chinese (zh)
Other versions
CN101814045B (en
Inventor
周可
王桦
张鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN2010101523978A priority Critical patent/CN101814045B/en
Publication of CN101814045A publication Critical patent/CN101814045A/en
Application granted granted Critical
Publication of CN101814045B publication Critical patent/CN101814045B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a data organization method for a backup service software storage server side for improving data organization and data management efficiency of the storage server side. The method comprises the following steps: (1) initializing a storage space of the storage server as a metadata area (comprising main record, index head and data index) and a data area; (2) receiving and judging a user operating command, sequentially performing a backup operation, recovering the operation and turning to step (4), and canceling the operation and turning to step (5); (3) processing a user backup operation, backing up user data to the data area of the storage server, meanwhile avoiding repeated data backup by using repeated data canceling technology, and turning to the step (2); (4) processing the recovery operation, positioning and searching a recovered data list specified by a user in the data area of the storage server, then transmitting the recovered data list to a user side, and turning to the step (2); and (5) processing the canceling operation, searching the canceled data specified by the user, introducing counting according to backup data blocks of the data in the data area of the storage server to perform corresponding processing, and turning to the step (2). The method improves the utilization rate and manageability of the storage server side and the expandability of the system, saves the network bandwidth, and improves the backup efficiency.

Description

A kind of data organization method that is used for backup services
Technical field
The invention belongs to computer data storage and backup method, be specifically related to a kind of data organization method of backup services, this method has realized the deletion of piece level repeating data.
Background technology
Along with the develop rapidly of informationized society, the service operation from the daily life to the enterprise, all the infosystem of being permeated is day by day surrounded, and is also increasing to its dependence.Especially in industries such as finance, communication, traffic and insurance,, bring immeasurable loss can for individual and enterprise in case critical data is lost or damaged.
Here said backup services is one in essence provides the backup of certain disaster tolerance function to recover software systems, can provide perfect data backup, recovery and related management task for individual and enterprise customer, and can customize various backup policy according to self actual demand.The backup services here also is a kind of software pattern simultaneously, for information-based needed all-network infrastructure and software, hardware running platform are built by enterprise, and be responsible for the enforcement in all early stages, a series of services such as maintenance in later stage, enterprise need not to buy software and hardware, builds machine room, recruits the IT personnel, can use infosystem by the internet.Just as opening just energy water of water tap, enterprise leases software service according to actual needs.
Data backup is the important measures that ensure information safety with recovering.Data importance show constantly that the data that require on the storage system can obtain effectively and comprehensively protection especially.Along with the appearance and the development of express network and the communication technology, mass memory innovation technology, basic storage resources is compared snafu variation in the past.The application of increasing various infosystems also makes the data volume of conservation value be the geometric series rising, and these all are that data backup has been researched and proposed higher requirement with exploitation and the correlation technique of recovering software.
Demand to storage space and data aspect when the user uses backup services generally includes: can increase or reduce the use amount to storage space according to demand; Can accessible use both have living space, as long as promptly have remaining space and network to reach, the data backup task can both correct execution; Can recover backed up data at any time.In order to satisfy these demands, require user's space and data to possess certain logical independence, so need research user's space way to manage and Backup Data method for organizing.In addition, also need to design allocation of space, reclaim mechanism efficiently, when fully excavating the coupling of repeating data storage space, keep the logical independence of user data, and realize data search and visit efficiently.
In general backup software framework, storage server is the physical medium that the supervisor console of process data backup software authenticated, and it can be a hard drive space on the server, the memory device that server is plug-in, perhaps a disk mapping on the network.Can dispose a plurality of storage servers by supervisor console, under the unified management of backup server, backup client backups to data on the corresponding storage server.
In the design of before backup software storage server end, adopt the backup method of file-level mostly.The backup of file-level, promptly backup software can only perceive this one deck of file, with files all on the source disk, backups on another purpose medium.So the file-level backup software, otherwise the file system interface that relies on operating system to provide comes backup file, or self have the function of file system, can discern file system metadata.In brief, it is that unit is read with the file that the mechanism of file-level backup software is exactly data, and then with the file storage of reading on the another one medium.Obviously this has formed performance bottleneck for PB level large-scale storage systems, because the data cell of storage server end management is exactly a file, this inevitably causes the backup of a large amount of repeating datas, management to the storage server end has also brought very big inconvenience, can address these problems to a great extent and utilize piece level data de-duplication technology to carry out data backup.
On the other hand, in backup software before, storage server end original allocation fixes for user's free memory capacity often, greatly reduces the extensibility of system like this.In the practical application, system can't expect that each user who is faced finally can use much storage spaces (may a maximum available storage restriction be arranged according to user's authority and type certainly), distributed big and caused waste of storage space and utilization factor to descend possibly, distributed little and may bring very big restriction user's use.
Recently Avamar company has been purchased by EMC Inc., and this company obtains the data de-duplication of patent and overall single example memory technology and can guarantee the Backup Data section only storage is once in global scope.This can be effectively will move and 300 times of data recovered amount reductions, can also realize full backup and fast quick-recovery every day simultaneously.At each 24KB data segment, Avamar generates unique 20 byte ID sign, uses the SHA-1 cryptographic algorithm.This unique ID is exactly the fingerprint of this data segment, so the software of Avamar can use this unique ID to determine whether that a data segment once was stored before this.But SHA-1 cryptographic algorithm calculation of complex is very big to the consumption of CPU.Because data segment is too small, the fingerprint space that consumes when the user ID data volume is very big is also very big, also has certain scalability problem simultaneously simultaneously.
Summary of the invention
The object of the present invention is to provide a kind of data organization method that is used for backup services, this method can realize the deletion of piece level repeating data, can improve the data organization and the efficiency of management.
The invention provides a kind of data organization method that is used for backup services, this method comprises the steps:
(1) initialization:
The metadata information partially-initialized comprises that indexing head information, data directory information, the data field metadata information to meta-data region composed initial value;
Prepare to accept user's backup request in data space of data field predistribution;
(2) receive user command and judge the user command type:
Judge the user command type, if backup operation enters step (3), if recovery operation then changes step (4) over to, if deletion action then changes step (5) over to;
(3) carry out back-up processing according to following process:
(3.1) file block at first that the user is to be backed up, then the content of data piecemeal is carried out Hash with the MD5 algorithm, obtain the fingerprint of a unique identification data piecemeal, deblocking is that index stores is in the indexing head and data directory of storage server end member data with the fingerprint;
(3.2) by the fingerprint of backup client to storage server transmission deblocking, whether the storage server end is inquired about this deblocking according to fingerprint and is existed;
(3.3) if this fingerprint does not exist, then backup client transmits this deblocking and gives storage server, and then this deblocking is new Backup Data piece, in storage server end memory space dynamic allocation, and finishes the write operation of this new Backup Data piece; If exist, then only need the pairing index information of updated stored this deblocking of server end, its reference count is added one;
(3.4) change step (2) over to;
When (4) recovering, check in the Hash tabulation for the treatment of that recovery file comprises by backup server, be positioned at logical place in the corresponding data space according to Hash tabulation visit storage space metadata information, read to treat that from the storage server end recovery file data are to core buffer successively then, pass to backup client by socket then, and synthetic required file set, change step (2) again over to;
(5) delete backup file by following process:
(5.1) check in the Hash tabulation for the treatment of that deleted file comprises by backup server in the standby system software;
(5.2) search the indexing head and the data directory mapping table of the meta-data region of storage server end according to hash value, if the suction parameter hash value does not exist, then return at once, rreturn value is false;
(5.3) otherwise the reference count of the object metadata of hash value correspondence is subtracted 1, rreturn value is true;
(5.4) change step (2) again over to.
Not only the many growths of kind are fast for present business data, and are high redundancies, a lot of identical files or data storing arranged in system and between the system, and the file that edits has a large amount of redundancies too, and these redundancies are present in the file version in the past.Traditional backup software backs up these redundant datas again and again, has amplified this redundancy.Present reasonable solution is to adopt data de-duplication technology.Data de-duplication technology not only can realize high compression rate, discharges storage space, also can reduce the cost based on Disk Backup, has also reduced the cost of data management.The present invention is data organization and the management method that a kind of data de-duplication technology based on the piece level realizes the storage server end, can efficiently carry out and client computer between the transmission of backup/restoration data, and carry out local storage space management and data organization by the strategy of backup server.The present invention can realize the data de-duplication of the overall situation under the prerequisite that does not influence the main users backup and recover, along with the growth of number of users and backup data quantity, the effect of data de-duplication will be obvious all the more.Can significantly reduce the required data volume of user ID, save the storage space that BACKUP TIME, the network bandwidth and backup need.
Description of drawings
Fig. 1 is the position fixing process figure of employed storage data organization of the inventive method and data item;
Fig. 2 is the FB(flow block) of the inventive method;
Fig. 3 is the process flow diagram of memory space dynamic allocation among the present invention;
Fig. 4 is the write operation process flow diagram in the backup operation of the present invention.
Embodiment
Below by by embodiment the present invention being described in further detail, but following examples only are illustrative, and protection scope of the present invention is not subjected to the restriction of these embodiment.
The backup services system is made up of backup server, storage server, backup client three parts based on tripartite framework.Wherein, backup client is responsible for accepting other relevant requests of data backup policy, recovery request or data management of customization.Backup server connects backup client and storage server, is the control center of whole data backup software.It is responsible for user right control, overall job scheduling and overall storage administration.When backup client is initiated the backup/restoration operation, guide the storage server of itself and appointment to connect and enter the execution link by backup server; On the other hand, backup server will be monitored calculating, transmission and the storage pressure of each storage server, and carry out the load balancing strategy.User profile, storage server state and other basic metadata that supports the backup server operation intend adopting database to store.Storage server be responsible for carrying out and client computer between the transmission of backup/restoration data, and carry out local storage space management and data organization by the strategy of backup server.
Below be 4 data structures that need this example use of explanation: master record district, indexing head, data directory and data field, its structure as shown in Figure 1.
The master record district: mainly describe the information of whole storage space, it deposits following information: indexing head information, data directory information, data field metadata information.
Indexing head: be an object Hash table, be used for realizing the mapping of object ID (by 160 hash value of data content generation) to data directory.Here to as if storage system in the elementary cell of data storage, be different from file and piece as basic module in the heritage storage system, to liking the combination of application data and definition memory attribute (metadata), wherein comprise data and permission data autonomy of other enough information and self-management.Its uses hash value represent object ID in the object-oriented storage, as storing foundation, sets up mapping relations content and object between by the hash value index with file content.Because hash value is that the overall situation is unique,, improved the manageability that system is shared so have the unique NameSpace of the overall situation on statistical significance.What system adopted is ripe MD5 algorithm, and the MD5Hash algorithm is transformed into the big integer of a 128bit (16byte), i.e. object ID with the data content of random length.
Data directory: be that a size is that (N represents index number in the data directory to N, and span is 2 20~2 30) array, each element in the array is the metadata structure of an object, information in the metadata structure has: object ID, (I represents the data space numbering to the start offset address of object in the data field, J represents the logical data block number in the data space of place, K represents the interior offset address of logic data block in the data space), object institute corresponding data size, the copy number of object institute corresponding data content, with the position of next object in Object table of this object map same position in the object Hash table, this just becomes a chained list to the object linking that is mapped to same position in the object Hash table.
Data field: the data that are used for depositing object, the data of object comprise object ID, data content length and data content, for the ease of storage space management, the data field is divided into several continuous data spaces (each data space is represented with an independent data file), and each data space is made up of some logic data blocks.
Deblocking: in the backup services system, when carrying out backup or recovery operation, all be the data that will handle according to the regular length piecemeal, each piecemeal is exactly a deblocking.
Backup Data piece: when the user utilizes the file of backup client backup appointment and file, backup client at first wants these backed up data according to regular length piecemeal (dividing block size in the actual backup services software systems is 4M), and each piecemeal is exactly the Backup Data piece
Logic data block: at the storage server end, manage and efficiently utilize the storage server storage space for convenience, each data space is divided into the experimental process storage unit, and each sub-storage unit is exactly a logic data block (each logical data block size is 1G in the actual backup services software systems)
Further specify the implementation procedure of this example below in conjunction with accompanying drawing.
Show that as Fig. 2 the inventive method comprises the steps:
(1) initialization:
Usually the storage data are divided into two parts: meta-data region and data field.The actual backed up data of user is stored in the data field, and the relevant information of describing these user data is stored in meta-data region.Beginning initialization metadata district mainly is that indexing head information, data directory information, the data field metadata information to meta-data region composed initial value.With indexing head is that the object Hash table all is changed to 0, represents all availablely, also each element in the data directory array is changed to simultaneously zero, represents that also write without any data this time.And prepare to accept user's backup request in data space of data field predistribution.The data space sum is defined as S, and S value maximum is no more than 1000.The data space number that the data field is current has used is V, V<=S.The preallocated logic data block of each data space (block) number is defined as P, and the P maximum is no more than 10.Each data space largest logical data block number is defined as W, and the W maximum is no more than 1024.In our present backup services Software deployment was implemented, each data space was made up of 1024 logic data blocks, and each logical data block size is 1G, and each data space is 1T to the maximum.
(2) receive user command and judge the user command type:
Judge the user command type, if backup operation enters step (3), if recovery operation then changes step (4) over to, if deletion action then changes step (5) over to;
(3) carry out back-up processing according to following process:
(3.1) (it is b to file block at first that the user is to be backed up that definition of data divides block size, b value size is 1M---4M, the b value is 4M during this backup services Software deployment of reality), then the content of data piecemeal is carried out Hash with the MD5 algorithm, obtain the fingerprint of a unique identification data piecemeal, deblocking is that index stores is in the indexing head and data directory of storage server end member data with the fingerprint;
(3.2) by the fingerprint of backup client to storage server transmission deblocking, whether the storage server end is inquired about this deblocking according to fingerprint and is existed;
(3.3) if this fingerprint does not exist, then backup client transmits this deblocking and gives storage server, in storage server end memory space dynamic allocation, and finishes the write operation of this deblocking; If exist, then need not to transmit data, only need the pairing index information of updated stored this deblocking of server end, reference count is added one.
(3.4) change step (2) over to;
In the above-mentioned steps (3.3), can be according to process memory space dynamic allocation shown in Figure 3, concrete steps are as follows:
(a1) judge whether the residue free space that can satisfy the big or small Backup Data piece of appointment is arranged,, enter step (a5) in P the logic data block in current data space if having, otherwise, step (a2) entered;
(a2) judge whether P<W sets up, enter step (a6) if set up, otherwise, step (a3) entered;
(a3) judge whether other data space in the storage server master record has the residue free space that can satisfy the Backup Data piece of specifying size, if having, enters step (a5), otherwise, step (a4) entered;
(a4) whether interpretation V<S sets up, if set up, then increases a data space on storage server, for Backup Data piece to be written in the new data space distributes a data index, changes step (a8) then over to, otherwise, enter step (a7);
(a5) for Backup Data piece to be written in the residue free space distributes a data index, change step (a8) then over to;
(a6) being the space of a Backup Data block size of this data space growth on storage server, is that this Backup Data piece distributes a data index again, changes step (a8) then over to;
(a7) because can not find the residue free space that can satisfy the Backup Data piece of specifying size, so announce the dynamic assignment failure;
(a8) finish dynamic allocation procedure.
Can also process as shown in Figure 4 finish write operation, its step is as follows:
(b1) dynamically seek free memory at the storage server end, search whether the logic data block that satisfies condition is arranged;
(b2) if do not have the utilogic data block then return failure;
(b3) if the free memory that satisfies new Backup Data block size is arranged, just create a new data index, new Backup Data piece is write the respective stored server location, then respective index head and data directory metadata are write the master record district.
When (4) recovering, check in the Hash tabulation for the treatment of that recovery file comprises by backup server in the standby system software, be positioned at logical place in the corresponding data space according to Hash tabulation visit storage space metadata information, read to treat that from the storage server end recovery file data are to core buffer successively then, pass to backup client by socket then, and synthetic required file set, change step (2) again over to.
As shown in Figure 2, it is as follows to be positioned at the process of the logical place in the corresponding data space according to Hash tabulation visit storage space metadata information:
(4.1) establish the figure place that m is predefined indexing head, by the preceding m position of data hash value indexing head is carried out index, the content of indexing head has constituted data directory number.
Usually, each indexing head accounts for two bytes, and one has 2 mIndividual, the span of m is generally 20~30.
(4.2) by indexing head to the data indexed addressing.Data directory then carries out addressing to the data item of single job, specifically comprises three partial contents:
(4.2.1) the structure member I by data directory (I is the data space numbering) finds concrete data space number;
(4.2.2) the structure member J by data directory (J is the logical data block number in the data space) finds the piece number in the data space;
(4.2.4) the structure member K (K is the interior offset address of logic data block in the data space) by data directory finds the offset address of data item in logic data block, is equivalent to three-level addressing.Data head and data entity that thus can the locator data item.
(4.4) obtain top three grades of logic data block address informations, just can navigate to corresponding data field reading of data.
(5) delete backup file by following process:
(5.1) check in the Hash tabulation that backup file to be deleted comprises by backup server in the standby system software;
(5.2) search the indexing head and the data directory mapping table of the meta-data region of storage server end according to hash value, if the suction parameter hash value does not exist, then return at once, rreturn value is false;
(5.3) otherwise the reference count of the object metadata of hash value correspondence is subtracted 1, rreturn value is true;
(5.4) change step (2) again over to.
Because we provide a kind of online backup service, thus backup server and storage server as finger daemon all the time at running background, therefore do not have the end situation, wait for the operation requests that receives the user all the time.And the operation interface of backup client to be the user use online backup service, the user can land the operation that backup client is carried out appointment arbitrarily the time, as backup, recovery and deletion etc.
Example:
The run time infrastructure of backup services system applies is:
1. hardware environment and support environment
Backup client requires main frame to possess 512M and above internal memory, 10Mbps and above network handling capacity.
Dispatch server requires main frame to possess 2GB and above internal memory, 1000Mbps and above network handling capacity.
Storage server requires main frame to possess 4GB and above internal memory and TB level external memory ability, the above network handling capacity of 1000Mbps level.
Possess GB level network exchange ability between dispatch server and the storage server software place main frame, possess the network-in-dialing ability between client and the service end software place main frame.Require server host place environment to possess the pacing items that redundant power guarantee, the guarantee of redundancy communication link, temperature control system, fire prevention system etc. guarantee that main frame runs well.
2. software runtime environment
The backup client program run is under Windows XP and later version operating system or the operating system platform based on Linux 2.6 kernels.
Dispatch server and storage server operate under Windows Server 2003 operating system platforms.
In the online backup service system that realizes at present and normally move, each data space size of storage server end is 1T, and the data space number is 20 to the maximum.Each data space is divided into 1024 logic data blocks, and each logical data block size is 1G.
The above is preferred embodiment of the present invention, but the present invention should not be confined to the disclosed content of this embodiment and accompanying drawing.So everyly do not break away from the equivalence of finishing under the spirit disclosed in this invention or revise, all fall into the scope of protection of the invention.

Claims (4)

1. a data organization method that is used for backup services is characterized in that, this method comprises the steps:
(1) initialization:
The metadata information partially-initialized comprises that indexing head information, data directory information, the data field metadata information to meta-data region composed initial value;
Prepare to accept user's backup request in data space of data field predistribution;
(2) receive user command and judge the user command type:
Judge the user command type, if backup operation enters step (3), if recovery operation then changes step (4) over to, if deletion action then changes step (5) over to;
(3) carry out back-up processing according to following process:
(3.1) file block at first that the user is to be backed up, then the content of data piecemeal is carried out Hash with the MD5 algorithm, obtain the fingerprint of a unique identification data piecemeal, deblocking is that index stores is in the indexing head and data directory of storage server end member data with the fingerprint;
(3.2) by the fingerprint of backup client to storage server transmission deblocking, whether the storage server end is inquired about this piecemeal according to fingerprint and is existed;
(3.3) if this fingerprint does not exist, then backup client transmits this deblocking and gives storage server, and then this deblocking is new Backup Data piece, in storage server end memory space dynamic allocation, and finishes the write operation of this new Backup Data piece; If exist, then only need the pairing index information of updated stored this deblocking of server end, its reference count is added one;
(3.4) change step (2) over to;
When (4) recovering, check in the Hash tabulation for the treatment of that recovery file comprises by backup server, the storage space metadata information was positioned at the logical place in the corresponding data space between tabulation was visited according to Hash, read to treat that from the storage server end recovery file data are to core buffer successively then, pass to backup client by socket then, and synthetic required file set, change step (2) again over to;
(5) delete backup file by following process:
(5.1) check in the Hash tabulation for the treatment of that deleted file comprises by backup server in the standby system software;
(5.2) search the indexing head and the data directory mapping table of the meta-data region of storage server end according to hash value, if the suction parameter hash value does not exist, then return at once, rreturn value is false;
(5.3) otherwise the reference count of the object metadata of hash value correspondence is subtracted 1, rreturn value is true;
(5.4) change step (2) again over to.
2. the data organization method that is used for backup services according to claim 1, it is characterized in that, in the above-mentioned steps (3.3), make P represent the preallocated logic data block number of each data space, W represents the largest logical data block number that can hold in each data space, the data space number that V has represented to use, S represents the data space sum;
The concrete steps of memory space dynamic allocation are as follows:
(a1) judge whether the residue free space that can satisfy the big or small Backup Data piece of appointment is arranged,, enter step (a5) in P the logic data block in current data space if having, otherwise, step (a2) entered;
(a2) judge whether P<W sets up, enter step (a6) if set up, otherwise, step (a3) entered;
(a3) judge whether other data space in the storage server master record has the residue free space that can satisfy the Backup Data piece of specifying size, if having, enters step (a5), otherwise, step (a4) entered;
(a4) whether interpretation V<S sets up, if set up, then increases a data space on storage server, for Backup Data piece to be written in the new data space distributes a data index, changes step (a8) then over to, otherwise, enter step (a7);
(a5) for Backup Data piece to be written in the residue free space distributes a data index, change step (a8) then over to.
(a6) on storage server, increase the space of a logical data block size, for data index of Backup Data piece distribution, change step (a8) then over to again for this data space;
(a7) announce the dynamic assignment failure;
(a8) finish dynamic allocation procedure.
3. the data organization method that is used for backup services according to claim 1 is characterized in that write operation comprises the steps:
(b1) dynamically seek free memory at the stores service end, search whether the logic data block that satisfies condition is arranged;
(b2) if do not have the utilogic data block then return failure;
(b3) if the free memory that satisfies new Backup Data block size is arranged, this creates a new data index, and new Backup Data piece is write the respective stored server location, then respective index head and data directory metadata is write the master record district.
4. the data organization method that is used for backup services according to claim 1 is characterized in that, the process that is positioned at the logical place in the corresponding data space according to storage space metadata information between Hash tabulation visit is as follows:
(4.1) according to the figure place m of predefined indexing head, indexing head is carried out index, the content composition data call number of indexing head by the preceding m position of data hash value;
(4.2) by indexing head to the data indexed addressing, by data directory the data item of single job is carried out addressing again, specifically comprise three partial contents:
(4.2.1) find concrete data space number by the data space among the structure member of data directory numbering;
(4.2.2) find piece number in the data space by the logical data block number in the data space among the structure member of data directory;
(4.2.4) find the offset address of data item in logic data block, the data head of locator data item and data entity by the offset address in the logic data block in the data space among the structure member of data directory;
(4.4) utilize the address information that obtains, navigate to corresponding data field reading of data.
CN2010101523978A 2010-04-22 2010-04-22 Data organization method for backup services Expired - Fee Related CN101814045B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2010101523978A CN101814045B (en) 2010-04-22 2010-04-22 Data organization method for backup services

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2010101523978A CN101814045B (en) 2010-04-22 2010-04-22 Data organization method for backup services

Publications (2)

Publication Number Publication Date
CN101814045A true CN101814045A (en) 2010-08-25
CN101814045B CN101814045B (en) 2011-09-14

Family

ID=42621306

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2010101523978A Expired - Fee Related CN101814045B (en) 2010-04-22 2010-04-22 Data organization method for backup services

Country Status (1)

Country Link
CN (1) CN101814045B (en)

Cited By (59)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101986276A (en) * 2010-10-21 2011-03-16 成都市华为赛门铁克科技有限公司 Methods and systems for storing and recovering files and server
CN101989929A (en) * 2010-11-17 2011-03-23 中兴通讯股份有限公司 Disaster recovery data backup method and system
CN102004769A (en) * 2010-11-12 2011-04-06 成都市华为赛门铁克科技有限公司 File management method, equipment and memory system
CN102012846A (en) * 2010-12-12 2011-04-13 成都东方盛行电子有限责任公司 Integrity check method for large video file
CN102364474A (en) * 2011-11-17 2012-02-29 中国科学院计算技术研究所 Metadata storage system for cluster file system and metadata management method
CN102385554A (en) * 2011-10-28 2012-03-21 华中科技大学 Method for optimizing duplicated data deletion system
CN102436408A (en) * 2011-10-10 2012-05-02 上海交通大学 Data storage cloud and cloud backup method based on Map/Dedup
CN102456059A (en) * 2010-10-21 2012-05-16 英业达股份有限公司 Data deduplication processing system
CN102467528A (en) * 2010-11-02 2012-05-23 英业达股份有限公司 Data de-duplication operation system
CN102469142A (en) * 2010-11-16 2012-05-23 英业达股份有限公司 Data transmission method for data deduplication program
CN102479245A (en) * 2010-11-30 2012-05-30 英业达集团(天津)电子技术有限公司 Data block segmentation method
CN102647399A (en) * 2011-02-17 2012-08-22 腾讯科技(深圳)有限公司 Software backup method and software backup system
CN102799659A (en) * 2012-07-05 2012-11-28 广州鼎鼎信息科技有限公司 Overall repeating data deleting system and method based on non-centre distribution system
CN102810108A (en) * 2011-06-02 2012-12-05 英业达股份有限公司 Method for processing repeated data
CN102810107A (en) * 2011-06-01 2012-12-05 英业达股份有限公司 Processing method for repeating data
CN102833298A (en) * 2011-06-17 2012-12-19 英业达集团(天津)电子技术有限公司 Distributed repeated data deleting system and processing method thereof
CN102890721A (en) * 2012-10-16 2013-01-23 苏州迈科网络安全技术股份有限公司 Database establishment method and database establishment system based on column storage technology
CN102915325A (en) * 2012-08-11 2013-02-06 深圳市极限网络科技有限公司 Md5 Hash list-based file decomposing and combining technique
CN103139300A (en) * 2013-02-05 2013-06-05 杭州电子科技大学 Virtual machine image management optimization method based on data de-duplication
CN103164431A (en) * 2011-12-13 2013-06-19 北京神州泰岳软件股份有限公司 Data storage method of relational database and storage system
WO2013107295A1 (en) * 2012-01-20 2013-07-25 腾讯科技(深圳)有限公司 Method for recovering hard drive data, server and distributed storage system
CN103238140A (en) * 2010-09-03 2013-08-07 赛门铁克公司 System and method for scalable reference management in a deduplication based storage system
CN103309873A (en) * 2012-03-09 2013-09-18 阿里巴巴集团控股有限公司 Method and device for processing data, and system
CN103348334A (en) * 2010-10-11 2013-10-09 Est软件公司 Cloud system and file compression and transmission method in a cloud system
WO2013163813A1 (en) * 2012-05-04 2013-11-07 华为技术有限公司 Data deduplication method and device
CN103412929A (en) * 2013-08-16 2013-11-27 蓝盾信息安全技术股份有限公司 Mass data storage method
TWI420306B (en) * 2010-12-22 2013-12-21 Inventec Corp A searching method of the blocks of the data deduplication
CN103530201A (en) * 2013-07-17 2014-01-22 华中科技大学 Safety data repetition removing method and system applicable to backup system
CN103559143A (en) * 2013-11-08 2014-02-05 华为技术有限公司 Data copying management device and data copying method of data copying management device
CN103873503A (en) * 2012-12-12 2014-06-18 鸿富锦精密工业(深圳)有限公司 Data block backup system and method
CN103944969A (en) * 2014-03-31 2014-07-23 中国电子科技集团公司第三十研究所 Secure transmission optimization method and device for narrow-band network
CN104166607A (en) * 2014-09-04 2014-11-26 北京国双科技有限公司 Data processing method and device for backup database
CN104317676A (en) * 2014-11-21 2015-01-28 四川智诚天逸科技有限公司 Data backup disaster tolerance method
CN104317735A (en) * 2014-09-24 2015-01-28 北京云巢动脉科技有限公司 High-capacity cache and method for data storage and readout as well as memory allocation and recovery
CN104536849A (en) * 2015-01-20 2015-04-22 成都携恩科技有限公司 Data backup method based on cloud computing
CN104537112A (en) * 2015-01-20 2015-04-22 成都携恩科技有限公司 Method for safe cloud computing
CN104778095A (en) * 2015-01-20 2015-07-15 成都携恩科技有限公司 Cloud platform data management method
CN104965772A (en) * 2015-07-29 2015-10-07 浪潮(北京)电子信息产业有限公司 Method and device for recovering files
CN105183400A (en) * 2015-10-23 2015-12-23 浪潮(北京)电子信息产业有限公司 Object storage method and system based on content addressing
CN105302675A (en) * 2015-11-25 2016-02-03 上海爱数信息技术股份有限公司 Method and device for data backup
CN106203154A (en) * 2016-06-27 2016-12-07 联想(北京)有限公司 A kind of file memory method and electronic equipment
CN106326397A (en) * 2016-08-19 2017-01-11 东软集团股份有限公司 Method and device for generating index file
CN106372170A (en) * 2016-08-30 2017-02-01 上海爱数信息技术股份有限公司 Database table recovery method and system and server with system
CN103810297B (en) * 2014-03-07 2017-02-01 华为技术有限公司 Writing method, reading method, writing device and reading device on basis of re-deleting technology
CN106877998A (en) * 2017-01-11 2017-06-20 裘羽 electronic evidence management method and system
CN107066352A (en) * 2017-03-02 2017-08-18 陈辉 With delete again and remote functionality portable intelligent device backup devices and methods therefor
CN107111460A (en) * 2015-03-30 2017-08-29 西部数据技术公司 Use the data de-duplication of block file
CN107340971A (en) * 2016-04-28 2017-11-10 上海优刻得信息科技有限公司 A kind of data storage is with recovering framework and method
CN109254786A (en) * 2018-09-30 2019-01-22 湖北华联博远科技有限公司 A kind of software backup restoring method and system
CN109271461A (en) * 2018-09-30 2019-01-25 广州鼎甲计算机科技有限公司 The increment synthesized backup method and device of SQL Server database
CN110471793A (en) * 2019-07-18 2019-11-19 维沃移动通信有限公司 Data back up method, data reconstruction method, first terminal and second terminal
CN111435331A (en) * 2019-01-14 2020-07-21 杭州宏杉科技股份有限公司 Data writing method and device for storage volume, electronic equipment and machine-readable storage medium
CN111694848A (en) * 2019-03-15 2020-09-22 阿里巴巴集团控股有限公司 Method and apparatus for updating data buffer using reference count
CN112000523A (en) * 2020-08-25 2020-11-27 浪潮云信息技术股份公司 Cloud backup system and method
CN112256194A (en) * 2020-09-30 2021-01-22 新华三技术有限公司成都分公司 Storage space distribution method and storage server
CN112328435A (en) * 2020-12-07 2021-02-05 武汉绿色网络信息服务有限责任公司 Method, device, equipment and storage medium for backing up and recovering target data
CN112394873A (en) * 2019-08-12 2021-02-23 深信服科技股份有限公司 Data management method, system, electronic equipment and storage medium
CN113111043A (en) * 2021-04-21 2021-07-13 北京大学 Method, device and system for processing source data file of middle station and storage medium
CN113422789A (en) * 2020-03-26 2021-09-21 山东管理学院 Service deployment method and system in network computing environment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005050386A2 (en) * 2003-11-13 2005-06-02 Commvault Systems, Inc. System and method for performing a snapshot and for restoring data
CN101183323A (en) * 2007-12-10 2008-05-21 华中科技大学 Data stand-by system based on finger print

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005050386A2 (en) * 2003-11-13 2005-06-02 Commvault Systems, Inc. System and method for performing a snapshot and for restoring data
CN101183323A (en) * 2007-12-10 2008-05-21 华中科技大学 Data stand-by system based on finger print

Cited By (91)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103238140B (en) * 2010-09-03 2016-02-17 赛门铁克公司 Based on the system and method quoting management in the storage system of deduplication for easily extensible
CN103238140A (en) * 2010-09-03 2013-08-07 赛门铁克公司 System and method for scalable reference management in a deduplication based storage system
CN103348334B (en) * 2010-10-11 2016-02-17 Est软件公司 Cloud system and the compressing file in cloud system and transfer approach
CN103348334A (en) * 2010-10-11 2013-10-09 Est软件公司 Cloud system and file compression and transmission method in a cloud system
CN101986276A (en) * 2010-10-21 2011-03-16 成都市华为赛门铁克科技有限公司 Methods and systems for storing and recovering files and server
CN102456059A (en) * 2010-10-21 2012-05-16 英业达股份有限公司 Data deduplication processing system
CN102467528A (en) * 2010-11-02 2012-05-23 英业达股份有限公司 Data de-duplication operation system
CN102004769A (en) * 2010-11-12 2011-04-06 成都市华为赛门铁克科技有限公司 File management method, equipment and memory system
CN102469142A (en) * 2010-11-16 2012-05-23 英业达股份有限公司 Data transmission method for data deduplication program
CN101989929B (en) * 2010-11-17 2014-07-02 中兴通讯股份有限公司 Disaster recovery data backup method and system
WO2012065408A1 (en) * 2010-11-17 2012-05-24 中兴通讯股份有限公司 Disaster tolerance data backup method and system
CN101989929A (en) * 2010-11-17 2011-03-23 中兴通讯股份有限公司 Disaster recovery data backup method and system
CN102479245A (en) * 2010-11-30 2012-05-30 英业达集团(天津)电子技术有限公司 Data block segmentation method
CN102479245B (en) * 2010-11-30 2013-07-17 英业达集团(天津)电子技术有限公司 Data block segmentation method
CN102012846A (en) * 2010-12-12 2011-04-13 成都东方盛行电子有限责任公司 Integrity check method for large video file
TWI420306B (en) * 2010-12-22 2013-12-21 Inventec Corp A searching method of the blocks of the data deduplication
CN102647399A (en) * 2011-02-17 2012-08-22 腾讯科技(深圳)有限公司 Software backup method and software backup system
CN102647399B (en) * 2011-02-17 2015-08-19 腾讯科技(深圳)有限公司 A kind of software backup method and system
CN102810107A (en) * 2011-06-01 2012-12-05 英业达股份有限公司 Processing method for repeating data
CN102810107B (en) * 2011-06-01 2015-10-07 英业达股份有限公司 The disposal route of repeating data
CN102810108A (en) * 2011-06-02 2012-12-05 英业达股份有限公司 Method for processing repeated data
CN102833298A (en) * 2011-06-17 2012-12-19 英业达集团(天津)电子技术有限公司 Distributed repeated data deleting system and processing method thereof
CN102436408A (en) * 2011-10-10 2012-05-02 上海交通大学 Data storage cloud and cloud backup method based on Map/Dedup
CN102436408B (en) * 2011-10-10 2014-02-19 上海交通大学 Data storage cloud and cloud backup method based on Map/Dedup
CN102385554A (en) * 2011-10-28 2012-03-21 华中科技大学 Method for optimizing duplicated data deletion system
CN102385554B (en) * 2011-10-28 2014-01-15 华中科技大学 Method for optimizing duplicated data deletion system
US9449005B2 (en) 2011-11-17 2016-09-20 Huawei Technologies Co., Ltd. Metadata storage system and management method for cluster file system
CN102364474A (en) * 2011-11-17 2012-02-29 中国科学院计算技术研究所 Metadata storage system for cluster file system and metadata management method
CN102364474B (en) * 2011-11-17 2014-08-20 中国科学院计算技术研究所 Metadata storage system for cluster file system and metadata management method
CN103164431A (en) * 2011-12-13 2013-06-19 北京神州泰岳软件股份有限公司 Data storage method of relational database and storage system
CN103164431B (en) * 2011-12-13 2016-04-20 北京神州泰岳软件股份有限公司 The date storage method of relevant database and storage system
WO2013107295A1 (en) * 2012-01-20 2013-07-25 腾讯科技(深圳)有限公司 Method for recovering hard drive data, server and distributed storage system
CN103309873A (en) * 2012-03-09 2013-09-18 阿里巴巴集团控股有限公司 Method and device for processing data, and system
WO2013163813A1 (en) * 2012-05-04 2013-11-07 华为技术有限公司 Data deduplication method and device
US8719237B2 (en) 2012-05-04 2014-05-06 Huawei Technologies Co., Ltd. Method and apparatus for deleting duplicate data
CN102799659B (en) * 2012-07-05 2015-01-21 广州鼎鼎信息科技有限公司 Overall repeating data deleting system and method based on non-centre distribution system
CN102799659A (en) * 2012-07-05 2012-11-28 广州鼎鼎信息科技有限公司 Overall repeating data deleting system and method based on non-centre distribution system
CN102915325A (en) * 2012-08-11 2013-02-06 深圳市极限网络科技有限公司 Md5 Hash list-based file decomposing and combining technique
CN102890721B (en) * 2012-10-16 2016-03-30 苏州迈科网络安全技术股份有限公司 Based on database building method and the system of row memory technology
CN102890721A (en) * 2012-10-16 2013-01-23 苏州迈科网络安全技术股份有限公司 Database establishment method and database establishment system based on column storage technology
CN103873503A (en) * 2012-12-12 2014-06-18 鸿富锦精密工业(深圳)有限公司 Data block backup system and method
CN103139300A (en) * 2013-02-05 2013-06-05 杭州电子科技大学 Virtual machine image management optimization method based on data de-duplication
CN103530201A (en) * 2013-07-17 2014-01-22 华中科技大学 Safety data repetition removing method and system applicable to backup system
CN103530201B (en) * 2013-07-17 2016-03-02 华中科技大学 A kind of secure data De-weight method and system being applicable to standby system
CN103412929A (en) * 2013-08-16 2013-11-27 蓝盾信息安全技术股份有限公司 Mass data storage method
CN103559143A (en) * 2013-11-08 2014-02-05 华为技术有限公司 Data copying management device and data copying method of data copying management device
WO2015067035A1 (en) * 2013-11-08 2015-05-14 华为技术有限公司 Data copy management device and data copy method thereof
CN103810297B (en) * 2014-03-07 2017-02-01 华为技术有限公司 Writing method, reading method, writing device and reading device on basis of re-deleting technology
CN103944969A (en) * 2014-03-31 2014-07-23 中国电子科技集团公司第三十研究所 Secure transmission optimization method and device for narrow-band network
CN104166607A (en) * 2014-09-04 2014-11-26 北京国双科技有限公司 Data processing method and device for backup database
CN104166607B (en) * 2014-09-04 2017-12-19 北京国双科技有限公司 Data processing method and device for backup database
CN104317735A (en) * 2014-09-24 2015-01-28 北京云巢动脉科技有限公司 High-capacity cache and method for data storage and readout as well as memory allocation and recovery
CN104317676A (en) * 2014-11-21 2015-01-28 四川智诚天逸科技有限公司 Data backup disaster tolerance method
CN104537112A (en) * 2015-01-20 2015-04-22 成都携恩科技有限公司 Method for safe cloud computing
CN104536849B (en) * 2015-01-20 2017-10-20 成都携恩科技有限公司 A kind of data back up method based on cloud computing
CN104536849A (en) * 2015-01-20 2015-04-22 成都携恩科技有限公司 Data backup method based on cloud computing
CN104778095A (en) * 2015-01-20 2015-07-15 成都携恩科技有限公司 Cloud platform data management method
CN104778095B (en) * 2015-01-20 2017-11-17 成都携恩科技有限公司 A kind of cloud platform data managing method
CN104537112B (en) * 2015-01-20 2017-07-14 成都携恩科技有限公司 A kind of method of safe cloud computing
CN107111460B (en) * 2015-03-30 2020-04-14 西部数据技术公司 Deduplication using chunk files
CN107111460A (en) * 2015-03-30 2017-08-29 西部数据技术公司 Use the data de-duplication of block file
CN104965772A (en) * 2015-07-29 2015-10-07 浪潮(北京)电子信息产业有限公司 Method and device for recovering files
CN105183400A (en) * 2015-10-23 2015-12-23 浪潮(北京)电子信息产业有限公司 Object storage method and system based on content addressing
CN105183400B (en) * 2015-10-23 2019-03-12 浪潮(北京)电子信息产业有限公司 It is a kind of based on content addressed object storage method and system
CN105302675A (en) * 2015-11-25 2016-02-03 上海爱数信息技术股份有限公司 Method and device for data backup
CN107340971A (en) * 2016-04-28 2017-11-10 上海优刻得信息科技有限公司 A kind of data storage is with recovering framework and method
CN107340971B (en) * 2016-04-28 2019-05-07 优刻得科技股份有限公司 A kind of storage of data with restore framework and method
CN106203154A (en) * 2016-06-27 2016-12-07 联想(北京)有限公司 A kind of file memory method and electronic equipment
CN106326397A (en) * 2016-08-19 2017-01-11 东软集团股份有限公司 Method and device for generating index file
CN106372170B (en) * 2016-08-30 2020-02-14 上海爱数信息技术股份有限公司 Method and system for recovering table in database and server with system
CN106372170A (en) * 2016-08-30 2017-02-01 上海爱数信息技术股份有限公司 Database table recovery method and system and server with system
CN106877998A (en) * 2017-01-11 2017-06-20 裘羽 electronic evidence management method and system
CN106877998B (en) * 2017-01-11 2020-06-19 裘羽 Electronic evidence management method and system
CN107066352A (en) * 2017-03-02 2017-08-18 陈辉 With delete again and remote functionality portable intelligent device backup devices and methods therefor
CN109271461A (en) * 2018-09-30 2019-01-25 广州鼎甲计算机科技有限公司 The increment synthesized backup method and device of SQL Server database
CN109254786A (en) * 2018-09-30 2019-01-22 湖北华联博远科技有限公司 A kind of software backup restoring method and system
CN111435331B (en) * 2019-01-14 2022-08-26 杭州宏杉科技股份有限公司 Data writing method and device for storage volume, electronic equipment and machine-readable storage medium
CN111435331A (en) * 2019-01-14 2020-07-21 杭州宏杉科技股份有限公司 Data writing method and device for storage volume, electronic equipment and machine-readable storage medium
CN111694848A (en) * 2019-03-15 2020-09-22 阿里巴巴集团控股有限公司 Method and apparatus for updating data buffer using reference count
CN111694848B (en) * 2019-03-15 2023-04-25 阿里巴巴集团控股有限公司 Method and apparatus for updating data buffering using reference counts
CN110471793B (en) * 2019-07-18 2022-05-06 维沃移动通信有限公司 Data backup method, data recovery method, first terminal and second terminal
CN110471793A (en) * 2019-07-18 2019-11-19 维沃移动通信有限公司 Data back up method, data reconstruction method, first terminal and second terminal
CN112394873A (en) * 2019-08-12 2021-02-23 深信服科技股份有限公司 Data management method, system, electronic equipment and storage medium
CN112394873B (en) * 2019-08-12 2024-05-24 深信服科技股份有限公司 Data management method, system, electronic equipment and storage medium
CN113422789A (en) * 2020-03-26 2021-09-21 山东管理学院 Service deployment method and system in network computing environment
CN113422789B (en) * 2020-03-26 2022-11-25 山东管理学院 Service deployment method and system in network computing environment
CN112000523A (en) * 2020-08-25 2020-11-27 浪潮云信息技术股份公司 Cloud backup system and method
CN112256194A (en) * 2020-09-30 2021-01-22 新华三技术有限公司成都分公司 Storage space distribution method and storage server
CN112328435A (en) * 2020-12-07 2021-02-05 武汉绿色网络信息服务有限责任公司 Method, device, equipment and storage medium for backing up and recovering target data
CN112328435B (en) * 2020-12-07 2023-09-12 武汉绿色网络信息服务有限责任公司 Method, device, equipment and storage medium for backing up and recovering target data
CN113111043A (en) * 2021-04-21 2021-07-13 北京大学 Method, device and system for processing source data file of middle station and storage medium

Also Published As

Publication number Publication date
CN101814045B (en) 2011-09-14

Similar Documents

Publication Publication Date Title
CN101814045B (en) Data organization method for backup services
US11526408B2 (en) Data recovery in a virtual storage system
US11550514B2 (en) Efficient transfers between tiers of a virtual storage system
US11093139B1 (en) Durably storing data within a virtual storage system
US10496442B2 (en) Job management and resource allocation in a data protection system
CN108804112B (en) Block chain settlement processing method and system
US11995336B2 (en) Bucket views
CN100419664C (en) Incremental backup operations in storage networks
CN100416508C (en) Copy operations in storage networks
US9372854B2 (en) Load balancing backup jobs in a virtualized storage system having a plurality of physical nodes
CN101676855B (en) Scalable secondary storage systems and methods
CN100462930C (en) Reconciliation method and system of backup data
CN101888405B (en) Cloud computing file system and data processing method
CN100524235C (en) Recovery operations in storage networks
US20230333781A1 (en) Modular data storage system with data resiliency
CN102436408B (en) Data storage cloud and cloud backup method based on Map/Dedup
US20070038822A1 (en) Copying storage units and related metadata to storage
CN106021016A (en) Virtual point in time access between snapshots
CN102906743A (en) Hybrid OLTP and OLAP high performance database system
CN100452046C (en) Storage method and system for mass file
CN102591982A (en) Method and system of performing incremental sql server database backups
US20210055885A1 (en) Enhanced data access using composite data views
US20220253255A1 (en) Scaling Storage Resources In A Storage Volume
US10783040B2 (en) Apparatus, system and method for data collection, import and modeling
US11327676B1 (en) Predictive data streaming in a virtual storage system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20110914

CF01 Termination of patent right due to non-payment of annual fee