CN103140840A - 数据管理的方法及装置 - Google Patents

数据管理的方法及装置 Download PDF

Info

Publication number
CN103140840A
CN103140840A CN2011800019002A CN201180001900A CN103140840A CN 103140840 A CN103140840 A CN 103140840A CN 2011800019002 A CN2011800019002 A CN 2011800019002A CN 201180001900 A CN201180001900 A CN 201180001900A CN 103140840 A CN103140840 A CN 103140840A
Authority
CN
China
Prior art keywords
block
record
storage pool
storage
key
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN2011800019002A
Other languages
English (en)
Other versions
CN103140840B (zh
Inventor
王道辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Li Shaohua
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of CN103140840A publication Critical patent/CN103140840A/zh
Application granted granted Critical
Publication of CN103140840B publication Critical patent/CN103140840B/zh
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G06F3/0613—Improving I/O performance in relation to throughput
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0644—Management of space entities, e.g. partitions, extents, pools
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)

Abstract

一种数据管理的方法及装置。所述方法将磁盘按逻辑上划分成多个存储池，且同一存储池中的块大小相等；在内存中设置入口索引数组，通过该入口索引数组来表示各个存储池的信息；将键值对Key-Value中的Key，所述磁盘中所划分出的存储池的池索引，以及存储池中块的块索引组成三元组，存放在内存中；通过内存中所设置的入口索引数组和存放的三元组信息来定位某一数据记录在所述磁盘中的存储位置，并根据该存储位置进行该数据的读、写或删除操作。通过该技术方案就能够有效的管理海量数据，而不需要进行磁盘碎片整理，使磁盘利用率可以大幅提高，并大大的提高索引和读取K-V的速度，从而提供高效的数据存储和管理操作。

Description

PCT国内申请，说明书已公开。

Claims (16)

  1. PCT国内申请，权利要求书已公开。
CN201180001900.2A 2011-09-30 2011-09-30 数据管理的方法及装置 Expired - Fee Related CN103140840B (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2011/080446 WO2013044502A1 (zh) 2011-09-30 2011-09-30 数据管理的方法及装置

Publications (2)

Publication Number Publication Date
CN103140840A true CN103140840A (zh) 2013-06-05
CN103140840B CN103140840B (zh) 2016-08-03

Family

ID=47994167

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201180001900.2A Expired - Fee Related CN103140840B (zh) 2011-09-30 2011-09-30 数据管理的方法及装置

Country Status (2)

Country Link
CN (1) CN103140840B (zh)
WO (1) WO2013044502A1 (zh)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104182508A (zh) * 2014-08-19 2014-12-03 华为技术有限公司 一种数据处理方法和数据处理装置
CN104360819A (zh) * 2014-09-29 2015-02-18 华为技术有限公司 数据写入方法及装置
CN106294603A (zh) * 2016-07-29 2017-01-04 北京奇虎科技有限公司 文件存储方法及装置
CN106708427A (zh) * 2016-11-17 2017-05-24 华中科技大学 一种适用于键值对数据的存储方法
CN107783732A (zh) * 2017-10-30 2018-03-09 郑州云海信息技术有限公司 一种数据读写方法、***、设备及计算机存储介质
CN108614734A (zh) * 2016-12-13 2018-10-02 迈普通信技术股份有限公司 安全参数索引管理方法及装置
CN109933291A (zh) * 2019-03-20 2019-06-25 浪潮商用机器有限公司 一种sram数据的处理方法、装置、设备及存储介质
CN110908996A (zh) * 2018-09-18 2020-03-24 北京京东尚科信息技术有限公司 一种数据处理的方法和装置
CN112214468A (zh) * 2020-10-18 2021-01-12 苏州浪潮智能科技有限公司 一种分布式存储***小文件加速方法、装置、设备及介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111897666B (zh) * 2020-08-05 2024-02-06 北京图森未来科技有限公司 用于多进程之间通信的方法、设备及***
CN113961153B (zh) * 2021-12-21 2022-11-04 杭州趣链科技有限公司 一种索引数据写入磁盘的方法、装置及终端设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030074341A1 (en) * 2001-10-11 2003-04-17 International Business Machines Corporation Method and system for dynamically managing hash pool data structures
CN1432922A (zh) * 2001-12-27 2003-07-30 日本电气株式会社 能够有效使用存储器的存储池管理方法和***
CN101997918A (zh) * 2010-11-11 2011-03-30 清华大学 异构san环境中的海量存储资源按需分配的实现方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030074341A1 (en) * 2001-10-11 2003-04-17 International Business Machines Corporation Method and system for dynamically managing hash pool data structures
CN1432922A (zh) * 2001-12-27 2003-07-30 日本电气株式会社 能够有效使用存储器的存储池管理方法和***
CN101997918A (zh) * 2010-11-11 2011-03-30 清华大学 异构san环境中的海量存储资源按需分配的实现方法

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104182508A (zh) * 2014-08-19 2014-12-03 华为技术有限公司 一种数据处理方法和数据处理装置
CN104182508B (zh) * 2014-08-19 2018-10-30 华为技术有限公司 一种数据处理方法和数据处理装置
CN104360819A (zh) * 2014-09-29 2015-02-18 华为技术有限公司 数据写入方法及装置
CN104360819B (zh) * 2014-09-29 2018-04-27 华为技术有限公司 数据写入方法及装置
CN106294603A (zh) * 2016-07-29 2017-01-04 北京奇虎科技有限公司 文件存储方法及装置
CN106294603B (zh) * 2016-07-29 2019-07-23 北京奇虎科技有限公司 文件存储方法及装置
CN106708427B (zh) * 2016-11-17 2019-05-10 华中科技大学 一种适用于键值对数据的存储方法
CN106708427A (zh) * 2016-11-17 2017-05-24 华中科技大学 一种适用于键值对数据的存储方法
CN108614734A (zh) * 2016-12-13 2018-10-02 迈普通信技术股份有限公司 安全参数索引管理方法及装置
CN107783732A (zh) * 2017-10-30 2018-03-09 郑州云海信息技术有限公司 一种数据读写方法、***、设备及计算机存储介质
CN110908996A (zh) * 2018-09-18 2020-03-24 北京京东尚科信息技术有限公司 一种数据处理的方法和装置
CN109933291A (zh) * 2019-03-20 2019-06-25 浪潮商用机器有限公司 一种sram数据的处理方法、装置、设备及存储介质
CN112214468A (zh) * 2020-10-18 2021-01-12 苏州浪潮智能科技有限公司 一种分布式存储***小文件加速方法、装置、设备及介质
CN112214468B (zh) * 2020-10-18 2023-01-06 苏州浪潮智能科技有限公司 一种分布式存储***小文件加速方法、装置、设备及介质

Also Published As

Publication number Publication date
CN103140840B (zh) 2016-08-03
WO2013044502A1 (zh) 2013-04-04

Similar Documents

Publication Publication Date Title
CN103140840A (zh) 数据管理的方法及装置
CN104850598B (zh) 一种实时数据库备份恢复方法
US9043540B2 (en) Systems and methods for tracking block ownership
US8799601B1 (en) Techniques for managing deduplication based on recently written extents
CN103902623B (zh) 用于在存储***上存取文件的方法和***
Easton Key-sequence data sets on indelible storage
CN101286127B (zh) 一种多叉日志存储的连续数据保护和恢复方法
WO2012012968A1 (zh) 一种分布式并行数据库***的数据分区方法
CN107391306A (zh) 一种异构数据库备份文件恢复方法
CN103714163B (zh) 一种NoSQL数据库的模式管理方法及***
EP3495964B1 (en) Apparatus and program for data processing
WO2017101505A1 (zh) 一种基于PostgreSQL块存储设备的迁移方法
WO2013075306A1 (zh) 数据访问方法和装置
CN105912675B (zh) 一种针对小文件合并的批量删除/查询方法与装置
WO2021107211A1 (ko) 인메모리 데이터베이스 기반의 시계열 데이터 관리시스템
Lee et al. Video indexing: an approach based on moving object and track
CN110309233A (zh) 数据存储的方法、装置、服务器和存储介质
CN104054071A (zh) 访问存储设备的方法和存储设备
CN100433019C (zh) 一种数据存储与检索的方法及***
CN110275838A (zh) Kv存储设备的地址转换及其加速器
CN100347705C (zh) 一种合并文件的方法
CN103548004A (zh) 在文件***中实现多级存储的方法和装置
WO2014025097A1 (ko) 비휘발성 램 기반의 ｂ＋ 트리 데이터베이스화 방법
WO2005066835A1 (fr) Procede d'extraction d'un enregistrement d'une page de donnees d'une base de donnees
CN1797397A (zh) 一种电力自动化***实时数据库管理***及其实现方法

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20170515

Address after: 510640 Guangdong City, Tianhe District Province, No. five, road, public education building, unit 371-1, unit 2401

Patentee after: Guangdong Gaohang Intellectual Property Operation Co., Ltd.

Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen

Patentee before: Huawei Technologies Co., Ltd.

TR01 Transfer of patent right
CB03 Change of inventor or designer information
CB03 Change of inventor or designer information

Inventor after: Li Shaohua

Inventor before: Wang Daohui

TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20170911

Address after: 054000 No. 185, Jia Jia Cun, Da Tun Tun, Renxian County, Hebei, Xingtai

Patentee after: Li Shaohua

Address before: 510640 Guangdong City, Tianhe District Province, No. five, road, public education building, unit 371-1, unit 2401

Patentee before: Guangdong Gaohang Intellectual Property Operation Co., Ltd.

CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20160803

Termination date: 20170930