CN110032549B - 分区***方法、装置、电子设备及可读存储介质 - Google Patents
分区***方法、装置、电子设备及可读存储介质 Download PDFInfo
- Publication number
- CN110032549B CN110032549B CN201910080122.9A CN201910080122A CN110032549B CN 110032549 B CN110032549 B CN 110032549B CN 201910080122 A CN201910080122 A CN 201910080122A CN 110032549 B CN110032549 B CN 110032549B
- Authority
- CN
- China
- Prior art keywords
- data
- splitting
- file
- partition
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005192 partition Methods 0.000 title claims abstract description 409
- 238000000034 method Methods 0.000 title claims abstract description 94
- 230000008569 process Effects 0.000 claims abstract description 39
- 238000012545 processing Methods 0.000 claims description 22
- 238000013479 data entry Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 9
- 230000008014 freezing Effects 0.000 claims description 9
- 238000007710 freezing Methods 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 11
- 230000000750 progressive effect Effects 0.000 description 8
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000001960 triggered effect Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 206010044625 Trichorrhexis Diseases 0.000 description 3
- 238000012217 deletion Methods 0.000 description 3
- 230000037430 deletion Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/278—Data partitioning, e.g. horizontal or vertical partitioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
- G06F16/322—Trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (14)
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910080122.9A CN110032549B (zh) | 2019-01-28 | 2019-01-28 | 分区***方法、装置、电子设备及可读存储介质 |
TW108129625A TWI738037B (zh) | 2019-01-28 | 2019-08-20 | 分區***方法、裝置、電子設備及可讀儲存媒體 |
US16/698,227 US10684787B1 (en) | 2019-01-28 | 2019-11-27 | Partition splitting method and apparatus, electronic device, and readable storage medium |
PCT/US2020/015472 WO2020160035A1 (en) | 2019-01-28 | 2020-01-28 | Partition splitting method and apparatus, electronic device, and readable storage medium |
EP20709375.8A EP3857402B1 (en) | 2019-01-28 | 2020-01-28 | Partition splitting method and apparatus, electronic device, and readable storage medium |
US16/899,251 US11016690B2 (en) | 2019-01-28 | 2020-06-11 | Partition splitting method and apparatus, electronic device, and readable storage medium |
US17/324,865 US11269535B2 (en) | 2019-01-28 | 2021-05-19 | Partition splitting method and apparatus, electronic device, and readable storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910080122.9A CN110032549B (zh) | 2019-01-28 | 2019-01-28 | 分区***方法、装置、电子设备及可读存储介质 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110032549A CN110032549A (zh) | 2019-07-19 |
CN110032549B true CN110032549B (zh) | 2023-10-20 |
Family
ID=67235598
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910080122.9A Active CN110032549B (zh) | 2019-01-28 | 2019-01-28 | 分区***方法、装置、电子设备及可读存储介质 |
Country Status (5)
Country | Link |
---|---|
US (3) | US10684787B1 (zh) |
EP (1) | EP3857402B1 (zh) |
CN (1) | CN110032549B (zh) |
TW (1) | TWI738037B (zh) |
WO (1) | WO2020160035A1 (zh) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110032549B (zh) | 2019-01-28 | 2023-10-20 | 北京奥星贝斯科技有限公司 | 分区***方法、装置、电子设备及可读存储介质 |
CN110716942A (zh) * | 2019-10-26 | 2020-01-21 | 南京录信软件技术有限公司 | 一种基于Lucene的大索引快速***方法 |
US11301210B2 (en) * | 2019-11-13 | 2022-04-12 | Cloudera, Inc. | Merging multiple sorted lists in a distributed computing system |
CN111352908B (zh) * | 2020-02-28 | 2023-10-10 | 北京奇艺世纪科技有限公司 | 基于lsm的数据存储方法、装置、存储介质及计算机设备 |
CN113094372A (zh) | 2021-04-16 | 2021-07-09 | 三星(中国)半导体有限公司 | 数据存取方法、数据存取控制装置及数据存取*** |
CN114253950B (zh) * | 2022-02-28 | 2022-06-03 | 北京奥星贝斯科技有限公司 | 管理数据库的方法和装置 |
CN116643300B (zh) * | 2023-07-25 | 2023-10-10 | 齐鲁空天信息研究院 | 基于地图映射的卫星导航数据分布式实时处理方法和*** |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7917494B2 (en) * | 2008-07-11 | 2011-03-29 | Adobe Software Trading Company Limited | System and method for a log-based data storage |
CN102073697A (zh) * | 2010-12-28 | 2011-05-25 | 中兴通讯股份有限公司 | 一种数据处理方法及装置 |
CN105468642A (zh) * | 2014-09-09 | 2016-04-06 | 杭州海康威视数字技术股份有限公司 | 数据的存储方法及装置 |
CN107436738A (zh) * | 2017-08-17 | 2017-12-05 | 北京理工大学 | 一种数据存储方法及*** |
US9852139B1 (en) * | 2012-07-02 | 2017-12-26 | Veritas Technologies Llc | Directory partitioning with concurrent directory access |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7085936B1 (en) * | 1999-08-30 | 2006-08-01 | Symantec Corporation | System and method for using login correlations to detect intrusions |
JP3640580B2 (ja) * | 1999-11-11 | 2005-04-20 | 富士通株式会社 | ソート・マージ処理装置 |
US7958096B2 (en) * | 2000-09-20 | 2011-06-07 | Ndsu-Research Foundation | System and method for organizing, compressing and structuring data for data mining readiness |
US7174343B2 (en) * | 2002-05-10 | 2007-02-06 | Oracle International Corporation | In-database clustering |
US7423656B2 (en) * | 2003-03-11 | 2008-09-09 | National University Of Singapore | Method and apparatus for generating morphing sequence |
US7533363B2 (en) * | 2004-03-29 | 2009-05-12 | Takumi Technology Corporation | System for integrated circuit layout partition and extraction for independent layout processing |
US7461091B2 (en) * | 2005-06-09 | 2008-12-02 | Sap Aktiengesellschaft | Controlling data transition between business processes in a computer application |
EP2016694B1 (en) * | 2006-05-09 | 2019-03-20 | Cognio, Inc. | System and method for identifying wireless devices using pulse fingerprinting and sequence analysis |
US9641615B1 (en) * | 2014-03-31 | 2017-05-02 | EMC IP Holding Company LLC | Allocating RAID storage volumes across a distributed network of storage elements |
CN105528347B (zh) * | 2014-09-28 | 2019-03-26 | 北京古盘创世科技发展有限公司 | 数据块储存方法、数据查询方法和数据修改方法 |
US10031935B1 (en) * | 2015-08-21 | 2018-07-24 | Amazon Technologies, Inc. | Customer-requested partitioning of journal-based storage systems |
US10346434B1 (en) * | 2015-08-21 | 2019-07-09 | Amazon Technologies, Inc. | Partitioned data materialization in journal-based storage systems |
CN107943412B (zh) * | 2016-10-12 | 2021-10-22 | 阿里巴巴集团控股有限公司 | 一种分区***、删除分区中数据文件的方法、装置及*** |
US10706106B2 (en) * | 2017-02-09 | 2020-07-07 | Micron Technology, Inc. | Merge tree modifications for maintenance operations |
US10725988B2 (en) * | 2017-02-09 | 2020-07-28 | Micron Technology, Inc. | KVS tree |
CN110032549B (zh) | 2019-01-28 | 2023-10-20 | 北京奥星贝斯科技有限公司 | 分区***方法、装置、电子设备及可读存储介质 |
-
2019
- 2019-01-28 CN CN201910080122.9A patent/CN110032549B/zh active Active
- 2019-08-20 TW TW108129625A patent/TWI738037B/zh active
- 2019-11-27 US US16/698,227 patent/US10684787B1/en active Active
-
2020
- 2020-01-28 EP EP20709375.8A patent/EP3857402B1/en active Active
- 2020-01-28 WO PCT/US2020/015472 patent/WO2020160035A1/en unknown
- 2020-06-11 US US16/899,251 patent/US11016690B2/en active Active
-
2021
- 2021-05-19 US US17/324,865 patent/US11269535B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7917494B2 (en) * | 2008-07-11 | 2011-03-29 | Adobe Software Trading Company Limited | System and method for a log-based data storage |
CN102073697A (zh) * | 2010-12-28 | 2011-05-25 | 中兴通讯股份有限公司 | 一种数据处理方法及装置 |
US9852139B1 (en) * | 2012-07-02 | 2017-12-26 | Veritas Technologies Llc | Directory partitioning with concurrent directory access |
CN105468642A (zh) * | 2014-09-09 | 2016-04-06 | 杭州海康威视数字技术股份有限公司 | 数据的存储方法及装置 |
CN107436738A (zh) * | 2017-08-17 | 2017-12-05 | 北京理工大学 | 一种数据存储方法及*** |
Also Published As
Publication number | Publication date |
---|---|
WO2020160035A1 (en) | 2020-08-06 |
EP3857402B1 (en) | 2023-03-08 |
TW202044064A (zh) | 2020-12-01 |
TWI738037B (zh) | 2021-09-01 |
US20200301596A1 (en) | 2020-09-24 |
US11016690B2 (en) | 2021-05-25 |
EP3857402A1 (en) | 2021-08-04 |
US10684787B1 (en) | 2020-06-16 |
US11269535B2 (en) | 2022-03-08 |
US20210271404A1 (en) | 2021-09-02 |
CN110032549A (zh) | 2019-07-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110032549B (zh) | 分区***方法、装置、电子设备及可读存储介质 | |
US10275184B2 (en) | Framework for volatile memory query execution in a multi node cluster | |
US10002148B2 (en) | Memory-aware joins based in a database cluster | |
EP3637280B1 (en) | Data storage method and device, and storage medium | |
US9875259B2 (en) | Distribution of an object in volatile memory across a multi-node cluster | |
CN102419770B (zh) | 文件共享***及实现文件共享的方法、文件索引服务设备 | |
CN108984639B (zh) | 服务器集群的数据处理方法和装置 | |
US9792185B2 (en) | Directed backup for massively parallel processing databases | |
CN112163048A (zh) | 基于ClickHouse实现OLAP分析的方法、装置 | |
CN113742135A (zh) | 数据备份方法、装置及计算机可读存储介质 | |
CN104932986A (zh) | 一种数据重分布方法及装置 | |
US11151111B2 (en) | Redistributing table data in a database cluster | |
CN110597912B (zh) | 一种区块存储方法及装置 | |
CN107391508B (zh) | 数据加载方法和*** | |
KR20210024751A (ko) | 이종 메모리 장치를 포함하는 그래프 처리 시스템 및 그래프 처리 시스템의 동작방법 | |
CN105469173A (zh) | 一种静态内存进行优化管理的方法 | |
US11429311B1 (en) | Method and system for managing requests in a distributed system | |
CN103905512A (zh) | 一种数据处理方法和设备 | |
CN112579550A (zh) | 一种分布式文件***的元数据信息同步方法及*** | |
CN117056303A (zh) | 适用于军事行动大数据的数据存储方法及装置 | |
CN114969165B (zh) | 数据查询请求的处理方法、装置、设备及存储介质 | |
CN114153395B (zh) | 一种对象存储数据生命周期管理方法、装置及设备 | |
CN110427390B (zh) | 数据查询方法及装置、存储介质、电子装置 | |
CN112783835A (zh) | 索引管理方法、装置及电子设备 | |
CN115840756B (zh) | 主备调***的数据更新方法、主备调***、设备和介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
TA01 | Transfer of patent application right | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20200925 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200925 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
TA01 | Transfer of patent application right | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20210208 Address after: 801-10, Section B, 8th floor, 556 Xixi Road, Xihu District, Hangzhou City, Zhejiang Province Applicant after: Ant financial (Hangzhou) Network Technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Innovative advanced technology Co.,Ltd. |
|
TA01 | Transfer of patent application right | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20210908 Address after: 100000 unit 02, floor 901, unit 1, building 1, No. 1, Middle East Third Ring Road, Chaoyang District, Beijing Applicant after: Beijing Aoxing Beisi Technology Co.,Ltd. Address before: 801-10, Section B, 8th floor, 556 Xixi Road, Xihu District, Hangzhou City, Zhejiang Province 310000 Applicant before: Ant financial (Hangzhou) Network Technology Co.,Ltd. |
|
GR01 | Patent grant | ||
GR01 | Patent grant |