CN106649556A - 基于分布式文件***的多层重复数据删除方法及装置 - Google Patents
基于分布式文件***的多层重复数据删除方法及装置 Download PDFInfo
- Publication number
- CN106649556A CN106649556A CN201610984188.7A CN201610984188A CN106649556A CN 106649556 A CN106649556 A CN 106649556A CN 201610984188 A CN201610984188 A CN 201610984188A CN 106649556 A CN106649556 A CN 106649556A
- Authority
- CN
- China
- Prior art keywords
- section
- file
- written
- digital finger
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/1827—Management specifically adapted to NAS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/164—File meta data generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610984188.7A CN106649556A (zh) | 2016-11-08 | 2016-11-08 | 基于分布式文件***的多层重复数据删除方法及装置 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610984188.7A CN106649556A (zh) | 2016-11-08 | 2016-11-08 | 基于分布式文件***的多层重复数据删除方法及装置 |
Publications (1)
Publication Number | Publication Date |
---|---|
CN106649556A true CN106649556A (zh) | 2017-05-10 |
Family
ID=58805866
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610984188.7A Pending CN106649556A (zh) | 2016-11-08 | 2016-11-08 | 基于分布式文件***的多层重复数据删除方法及装置 |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106649556A (zh) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109947731A (zh) * | 2017-07-31 | 2019-06-28 | 星辰天合(北京)数据科技有限公司 | 重复数据的删除方法和装置 |
WO2020215580A1 (zh) * | 2019-04-23 | 2020-10-29 | 平安科技(深圳)有限公司 | 一种分布式全局数据去重方法和装置 |
WO2023093091A1 (zh) * | 2021-11-25 | 2023-06-01 | 华为技术有限公司 | 数据存储***、智能网卡及计算节点 |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101216791A (zh) * | 2008-01-04 | 2008-07-09 | 华中科技大学 | 基于指纹的文件备份方法 |
CN102833298A (zh) * | 2011-06-17 | 2012-12-19 | 英业达集团(天津)电子技术有限公司 | 分布式的重复数据删除***及其处理方法 |
CN102915278A (zh) * | 2012-09-19 | 2013-02-06 | 浪潮(北京)电子信息产业有限公司 | 重复数据删除方法 |
CN103514250A (zh) * | 2013-06-20 | 2014-01-15 | 易乐天 | 一种全局重复数据删除的方法和***及存储装置 |
US8677132B1 (en) * | 2012-01-06 | 2014-03-18 | Narus, Inc. | Document security |
CN104408154A (zh) * | 2014-12-04 | 2015-03-11 | 华为技术有限公司 | 重复数据删除方法及装置 |
CN104932841A (zh) * | 2015-06-17 | 2015-09-23 | 南京邮电大学 | 一种云存储***中节约型重复数据删除方法 |
CN105320773A (zh) * | 2015-11-03 | 2016-02-10 | 中国人民解放军理工大学 | 一种基于Hadoop平台的分布式重复数据删除***和方法 |
CN105912268A (zh) * | 2016-04-12 | 2016-08-31 | 韶关学院 | 一种基于自匹配特征的分布式重复数据删除方法及其装置 |
-
2016
- 2016-11-08 CN CN201610984188.7A patent/CN106649556A/zh active Pending
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101216791A (zh) * | 2008-01-04 | 2008-07-09 | 华中科技大学 | 基于指纹的文件备份方法 |
CN102833298A (zh) * | 2011-06-17 | 2012-12-19 | 英业达集团(天津)电子技术有限公司 | 分布式的重复数据删除***及其处理方法 |
US8677132B1 (en) * | 2012-01-06 | 2014-03-18 | Narus, Inc. | Document security |
CN102915278A (zh) * | 2012-09-19 | 2013-02-06 | 浪潮(北京)电子信息产业有限公司 | 重复数据删除方法 |
CN103514250A (zh) * | 2013-06-20 | 2014-01-15 | 易乐天 | 一种全局重复数据删除的方法和***及存储装置 |
CN104408154A (zh) * | 2014-12-04 | 2015-03-11 | 华为技术有限公司 | 重复数据删除方法及装置 |
CN104932841A (zh) * | 2015-06-17 | 2015-09-23 | 南京邮电大学 | 一种云存储***中节约型重复数据删除方法 |
CN105320773A (zh) * | 2015-11-03 | 2016-02-10 | 中国人民解放军理工大学 | 一种基于Hadoop平台的分布式重复数据删除***和方法 |
CN105912268A (zh) * | 2016-04-12 | 2016-08-31 | 韶关学院 | 一种基于自匹配特征的分布式重复数据删除方法及其装置 |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109947731A (zh) * | 2017-07-31 | 2019-06-28 | 星辰天合(北京)数据科技有限公司 | 重复数据的删除方法和装置 |
WO2020215580A1 (zh) * | 2019-04-23 | 2020-10-29 | 平安科技(深圳)有限公司 | 一种分布式全局数据去重方法和装置 |
WO2023093091A1 (zh) * | 2021-11-25 | 2023-06-01 | 华为技术有限公司 | 数据存储***、智能网卡及计算节点 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10019184B2 (en) | Amortized snapshots | |
US10158483B1 (en) | Systems and methods for efficiently and securely storing data in a distributed data storage system | |
US7447839B2 (en) | System for a distributed column chunk data store | |
US7464247B2 (en) | System and method for updating data in a distributed column chunk data store | |
CN105190573B (zh) | 存储数据的减少冗余 | |
JP4648723B2 (ja) | データ価値に基づく階層型ストレージ管理の為の方法と装置 | |
US5819272A (en) | Record tracking in database replication | |
CN104281533B (zh) | 一种存储数据的方法及装置 | |
US7457935B2 (en) | Method for a distributed column chunk data store | |
US7587569B2 (en) | System and method for removing a storage server in a distributed column chunk data store | |
US11093387B1 (en) | Garbage collection based on transmission object models | |
US20070143369A1 (en) | System and method for adding a storage server in a distributed column chunk data store | |
US8768980B2 (en) | Process for optimizing file storage systems | |
US9462037B2 (en) | Dynamically sizing chunks in a partially loaded spreadsheet model | |
CN105556520A (zh) | 在存储器中镜像盘中的数据以提高查询性能 | |
JP2005539314A (ja) | データをネットワーク中で処理する装置および方法 | |
US7519636B2 (en) | Key sequenced clustered I/O in a database management system | |
CN106649556A (zh) | 基于分布式文件***的多层重复数据删除方法及装置 | |
CN109767274B (zh) | 一种对海量***数据进行关联存储的方法及*** | |
JP5241298B2 (ja) | 履歴上のファイル名およびロケーションをインデックス付きにすることによりファイル・サーチおよびファイル操作を支援するためのシステムおよび方法 | |
CN110008197A (zh) | 一种数据处理方法、***及电子设备和存储介质 | |
CN113282540A (zh) | 一种云对象存储同步方法、装置、计算机设备及存储介质 | |
CN108243207B (zh) | 一种网络云盘的数据存储方法 | |
Henry | Howard chu on lightning memory-mapped database | |
CN114417413A (zh) | 区块链文件***的文件处理方法、装置、设备及介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20171023 Address after: 518100 Guangdong city of Shenzhen province Nanshan District South Road Fiyta Technology Building Room 1402 Applicant after: Shenzhen Zhongbo Kechuang Information Technology Co., Ltd. Address before: 518000 Guangdong city of Shenzhen province Qianhai Shenzhen Hong Kong cooperation zone before Bay Road No. 1 building 201 room A (located in Shenzhen Qianhai business secretary Co. Ltd.) Applicant before: Shenzhen City Rui Bo Storage Technology Co. Ltd. |
|
TA01 | Transfer of patent application right | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20190902 Address after: 100089 Floor 1-4, No. 2 Building, No. 9 Courtyard, Dijin Road, Haidian District, Beijing Applicant after: Beijing Toyou Feiji Electronics Co., Ltd. Address before: 518100 Room 1402, Feiyada Science and Technology Building, Nanshan District, Shenzhen City, Guangdong Province Applicant before: Shenzhen Zhongbo Kechuang Information Technology Co., Ltd. |
|
TA01 | Transfer of patent application right | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170510 |
|
RJ01 | Rejection of invention patent application after publication |