CN111475507A - Key value data indexing method for workload self-adaptive single-layer L SMT - Google Patents

Key value data indexing method for workload self-adaptive single-layer L SMT Download PDF

Info

Publication number
CN111475507A
CN111475507A CN202010244527.4A CN202010244527A CN111475507A CN 111475507 A CN111475507 A CN 111475507A CN 202010244527 A CN202010244527 A CN 202010244527A CN 111475507 A CN111475507 A CN 111475507A
Authority
CN
China
Prior art keywords
sub
key
space
run
memory table
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010244527.4A
Other languages
Chinese (zh)
Other versions
CN111475507B (en
Inventor
陈珂
周信静
寿黎但
骆歆远
伍赛
江大伟
陈刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202010244527.4A priority Critical patent/CN111475507B/en
Publication of CN111475507A publication Critical patent/CN111475507A/en
Application granted granted Critical
Publication of CN111475507B publication Critical patent/CN111475507B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a key value data index method of a working load self-adaptive single-layer L SMT, which optimizes a traditional log structure merged Tree (L g-Structured-Merge Tree, L SMT), removes multi-layer design and fixed memory table capacity design, and introduces the single-layer L SMT and dynamic capacity memory table design.

Description

Key value data indexing method for workload self-adaptive single-layer L SMT
Technical Field
The invention belongs to the technical field of database systems, and particularly relates to a key value data indexing method of a workload self-adaptive single-layer L SMT.
Background
A key-value storage system based on a log-Structured Merge Tree (L SMT) is widely applied to data-intensive Internet applications due to excellent processing-intensive writing capacity, but the conventional storage system based on the L SMT generally has the problems of amplification and no perception of workload
Figure BDA0002433639280000011
To quantify this issue, the problem of workload unaware refers to the inability of existing L SMT systems to make more appropriate optimizations to the storage structure based on the distribution of reads and writes in the workload.
In order to solve the problem of read-write amplification, researchers have proposed many methods, but these methods generally sacrifice read amplification for reduction of write amplification (such as WiscKey, PebblesDB), and cannot guarantee high efficiency of both read and write. The problem of no perception of workload is also very rare to study and solve.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides a workload adaptive key value data indexing method on a block storage device. The method can effectively reduce read-write amplification, and meanwhile, self-adaptive storage structure optimization can be performed according to the working load, so that the read delay is further reduced.
The invention aims to realize the technical scheme that a workload self-adaptive single-layer L SMT key value data indexing method specifically comprises the following steps:
(1) the L SMT storage structure is designed in a modified mode, and comprises the following sub-steps:
(1.1) removing L intermediate layer of SMT multilayer structure, retaining the last layer, and using the last layer as storage layer L0(ii) a The original memory table with fixed capacity is replaced by a dynamic capacity memory table, the capacity value of the dynamic capacity memory table is M, a real number parameter R is introduced, R is more than 1, and the requirement of meeting the requirement
Figure BDA0002433639280000012
|L0And | is the data amount of the current storage layer.
(1.2) storage layer L0Partitioning into N sub-key spaces l according to key range1,l2,…,lNThe sub-key spaces do not overlap, and each sub-key space liThe data (i is more than or equal to 1 and less than or equal to N) are stored in independent storage files. Each sub-key space li(1 ≦ i ≦ N) storing at most T updated Run's from the dynamic volumetric memory table for data in the sub-key space, and noting γ (l)i) Is the sub-bond space liThe contained Run set, | gamma (l)i) I is the size of the set; the Run key value data are sorted according to the key sequence, and the Run of one sub-key space can be overlapped; and T is more than or equal to 1.
(2) Merging the dynamic capacity memory tables, specifically, when the capacity value of the dynamic capacity memory table exceeds M, converting the dynamic capacity memory table into a read-only memory table, starting a merging process in a background thread, and merging the read-only memory table into a storage layer L0At the same time, a new active memory list is established, and the front-end read-write request is continuously processed, and the merging process is specifically carried out according to the storage layer L0The read-only memory table is divided into N Run, which is recorded as r1，r2，...，rNWherein r isiBelong toi. Then r isiWrite the corresponding sub-key space liIn the corresponding storage file. When a sub-key space liExceeds a threshold value β or gamma (l)i) When | is greater than T, gamma (l)i) Merging into a Run, i.e. | γ (l) after mergingi) And (3) equally dividing the data into two sub-key spaces according to the data quantity, and finishing the index writing step after the merging is finished.
Further, the threshold β takes the value of 64 MB.
Further, the indexing method also comprises an adaptive reading optimization method, which comprises the following sub-steps:
(a) will be provided with
Figure BDA0002433639280000021
The sub-key space l as time tiThe heat reading statistics of (a) for the time t are obtained as:
Figure BDA0002433639280000022
wherein the content of the first and second substances,
Figure BDA0002433639280000023
α is an attenuation factor between t-1 and t, a is more than 0 and less than 1,
Figure BDA0002433639280000024
and is
Figure BDA0002433639280000026
(b) Will be provided with
Figure BDA0002433639280000027
The sub-key space l as time tiThe write heat statistics of (1) for the write heat at time t are:
Figure BDA0002433639280000028
wherein the content of the first and second substances,
Figure BDA0002433639280000029
the number of times of writing into the sub-key space li between the times t-1 and t,
Figure BDA00024336392800000210
and is
Figure BDA00024336392800000211
I(t)The time elapsed between the current time t and the last time t-1.
(c) The following process was run periodically: firstly, clustering is carried out on sub-key spaces according to the writing heat degree, and the sub-key spaces are divided into four types, namely: cold, Warm, WriteBalanced, WriteHeavy. Selecting the child key space of the Cold class with the lowest writing heat degree, then sorting the child key spaces of the Cold class in a descending order according to the reading heat degree, and filtering out the child key space l meeting the following conditionsi：
Figure BDA00024336392800000212
Then, the first P, l in the sorted and filtered Cold-type sub-key space set are selected1，l2，...，lPSpace l of each sub-keyiGamma (l) ofi) (i is more than or equal to 1 and less than or equal to P) are combined into a Run.
Compared with the prior art, the formula has the advantages that the indexing method provided by the invention removes the traditional L SMT multilayer design, and introduces a dynamic capacity memory table and a self-adaptive reading optimization mechanism:
1) compared with the conventional L SMT, the indexing method provided by the invention has the advantages that the read-write amplification factor is lower, the read-write throughput of the system is higher, and the service life of the storage device is prolonged.
2) The indexing method provided by the invention can automatically make structure optimization according to the read-write distribution in the workload, thereby further reducing the system delay.
Drawings
FIG. 1 is a flow chart of an indexing method proposed by the present invention.
Detailed Description
The technical solutions of the present invention are further described below with reference to the accompanying drawings, and it should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.
Fig. 1 is a flowchart of a workload adaptive key value data indexing method on a block storage device according to the present invention, which specifically includes the following steps:
(1) the L SMT storage structure is designed in a modified mode, and comprises the following sub-steps:
(1.1) removing L intermediate layers of the SMT multilayer structure, reserving the last layer, using the last layer as a storage layer L0, changing the original memory table with fixed capacity into a dynamic capacity memory table, keeping other structures basically unchanged, reserving a log file for restoring the memory table after downtime, wherein the capacity value of the dynamic capacity memory table is M, and a real number parameter R is introduced, wherein R is more than 1 to meet the requirement that the real number parameter R is larger than 1
Figure BDA0002433639280000031
R is used for balancing the memory usage amount and the read-write amplification, and M is increased along with the increase of the data amount.
(1.2) to facilitate adaptive read optimization of subsequent sequences, for the storage layer L0A dynamic partitioning mechanism is introduced, the storage layer L0Partitioning into N sub-key spaces l according to key range1，l2，...，lNThe sub-key spaces do not overlap, and each sub-key space liThe data (i is more than or equal to 1 and less than or equal to N) are stored in independent storage files. Each sub-key space li(1 ≦ i ≦ N) storing at most T updated Run's from the dynamic volumetric memory table for data in the sub-key space, and noting γ (l)i) Is the sub-bond space liThe contained Run set, | gamma (l)i) I is the size of the set; the Run key value data are sorted according to the key sequence, and the Run of one sub-key space can be overlapped; and T is more than or equal to 1. Each sub-key space holds metadata: max _ key, num _ runs, index _ data; the max _ key is the maximum key for describing the sub-key interval and is used for describing the range of the sub-key space; num runs describes how many runs the sub-key space has; index _ data is index data that stores each Run of the sub-key space. WhereinThe index data of each Run in turn contains: the bloom _ filter _ data is bloom _ filter _ data established for the Run; the block _ index _ data divides the data stored by the Run by blocks, wherein the blocks are 4KB, and the block _ index _ data stores the maximum key of each block and is used for quickly locating the block where one query key is possibly located. Finally, the metadata is further processed<max_key，metadata>To store key-value data in another independent conventional L SMT key-value storage system, such as L evelDB/RocskDB, since these metadata are much smaller than the data, these metadata can generally be cached in memory without the need to execute IOs.
(2) Merging the dynamic capacity memory tables, specifically, when the capacity value of the dynamic capacity memory table exceeds M, converting the dynamic capacity memory table into a read-only memory table, starting a merging process in a background thread, and merging the read-only memory table into a storage layer L0And meanwhile, establishing a new active memory table, and continuously processing the front-end read-write request, wherein the merging process specifically comprises the step of dividing the read-only memory table into N Runs which are recorded as r according to the range partition of the sub-key space of the storage layer L01，r2，...，rNWherein r isiBelong toiI.e. riMaximum key of middle key value data is less than or equal to liThe largest key of the stored key-value data (available by querying the original data store L SMT), and riMinimum key > l of medium-key value datai-1The largest key of the key-value data stored in (obtained by querying the original data store L SMT)iWrite the corresponding sub-key space liIn the corresponding storage file. The metadata is updated at the same time, and this updating process includes modifying num _ runs and index _ data, etc. To limit the number of queries IO within a sub-key space, when a sub-key space liExceeds a threshold value β (64MB) or | γ (l)i) When | is greater than T, gamma (l)i) Merging into a Run, i.e. | γ (l) after mergingi) 1 and equally divided into two sub-key spaces according to the data amount, the process is a split operation. By introducing split operations, each sub-key is guaranteed to be emptyThe split operation involves writing in a sub-key space file, updating original data L SMT, when all Runs finish writing and the split operation is triggered, the whole merging is finished, when the merging is finished, the index writing step is finished.
One embodiment of the invention also comprises an adaptive read optimization method, which comprises the following sub-steps:
(a) will be provided with
Figure BDA0002433639280000041
The sub-key space l as time tiThe reading heat value at the time t is obtained by the following exponential decay formula:
Figure BDA0002433639280000042
wherein the content of the first and second substances,
Figure BDA0002433639280000043
between t-1 and t, the sub-key space liα is an attenuation factor, a is more than 0 and less than 1,
Figure BDA0002433639280000044
and is
Figure BDA0002433639280000045
The exponential decay technique is used here to allow the system to capture temporal locality, i.e., reads that are executed recently are more reflective of the current state of the workload than older reads, giving these reads more weight. While these read heat information are periodically updated in the system.
(b) Will be provided with
Figure BDA0002433639280000046
The sub-key space l as time tiThe writing heat value at the time t can be obtained by the following exponential decay formula:
Figure BDA0002433639280000047
wherein the content of the first and second substances,
Figure BDA0002433639280000048
between t-1 and t, the sub-key space liThe number of times of writing of (a),
Figure BDA0002433639280000049
and is
Figure BDA00024336392800000410
I(t)The time elapsed between the current time t and the last time t-1. The time is defined as the starting time of each merging, namely, the updating of the writing heat information is carried out at the merging time, so that the updating overhead can be reduced. Introduction of I(t)This is to deal with the situation where the system has no write requests for a long time, in which case the weight of these writes should also be reduced.
(c) With the read-write heat, we introduce an adaptive read optimization mechanism. Firstly, clustering is carried out on sub-key spaces according to the writing heat degree, and the sub-key spaces are divided into four types, namely: cold, Warm, WriteBalanced, WriteHeavy. Optimizing the child key spaces of the Cold type, wherein the child key spaces of the other three types are always subjected to more writing operations, so that the writing operation performance of the writing child key spaces is not influenced; sorting the child key spaces of the Cold class in a descending order according to the reading heat degree, and filtering out the child key spaces l meeting the following conditionsi：
Figure BDA0002433639280000051
Then, the first P, l in the sorted and filtered Cold-type sub-key space set are selected1,l2,…,lPSpace l of each sub-keyiGamma (l) ofi) (i is more than or equal to 1 and less than or equal to P) are combined into a Run.
The method is used for balancing the memory usage amount and the read-write amplification, and increases along with the increase of the data amount. Formula one embodiment of the present invention further includes read operations that are divided into a point read operation and a range read operation. For the point reading operation, firstly, inquiring the dynamic capacity memory table, if not, inquiring the read-only memory table, and if so, finishing the point reading operation. If all dynamic capacity memory tables do not find corresponding data, determining a sub-key space possibly containing a query key through a metadata file, then searching Run according to the reverse order of the writing sequence of the Run in the sub-key space, if the Run is found, finishing the query, and if the Run is found, declaring that the query cannot find the query key. In order to optimize the point reading operation, the invention also allocates a bloom filter for the Run of each sub-key space, before executing the Run query, the bloom filter is queried to judge whether the Run may contain the query key, and invalid IO is avoided with low cost. And for the range reading operation, simultaneously inquiring Run of the dynamic capacity memory table and the sub-key space, and obtaining an inquiry result through merging and merging.
Compared with WiscKey and L evenDB, the indexing method disclosed by the invention has the advantages that the write amplification required for completing the same operation is reduced by 4 times at most, the read operation performance is kept excellent, and the click-to-read delay is reduced by 30% again under a self-adaptive read optimization mechanism.

Claims (3)

1. A method for indexing key value data of a workload adaptive single layer L SMT is characterized by comprising the following steps:
(1) the L SMT storage structure is designed in a modified mode, and comprises the following sub-steps:
(1.1) removing L intermediate layer of SMT multilayer structure, retaining the last layer, and using the last layer as storage layer L0(ii) a The original memory table with fixed capacity is replaced by a dynamic capacity memory table, the capacity value of the dynamic capacity memory table is M, a real number parameter R is introduced, R is more than 1, and the requirement of meeting the requirement
Figure FDA0002433639270000011
|L0And | is the data amount of the current storage layer.
(1.2) storage layer L0Partitioning into N sub-key spaces l according to key range1，l2，...，lNThe sub-key spaces do not overlap, and each sub-key space liThe data (i is more than or equal to 1 and less than or equal to N) are stored in independent storage files. Each sub-key space li(1 ≦ i ≦ N) storing at most T updated Run's from the dynamic volumetric memory table for data in the sub-key space, and noting γ (l)i) Is the sub-bond space liThe contained Run set, | gamma (l)i) I is the size of the set; the Run key value data are sorted according to the key sequence, and the Run of one sub-key space can be overlapped; and T is more than or equal to 1.
(2) Merging the dynamic capacity memory tables, specifically, when the capacity value of the dynamic capacity memory table exceeds M, converting the dynamic capacity memory table into a read-only memory table, starting a merging process in a background thread, and merging the read-only memory table into a storage layer L0At the same time, a new active memory list is established, and the front-end read-write request is continuously processed, and the merging process is specifically carried out according to the storage layer L0The read-only memory table is divided into N Run, which is recorded as r1，r2，...，rNWherein r isiBelong toi. Then r isiWrite the corresponding sub-key space liIn the corresponding storage file. When a sub-key space liExceeds a threshold value β or gamma (l)i) When | is greater than T, gamma (l)i) Merging into a Run, i.e. | γ (l) after mergingi) And (3) equally dividing the data into two sub-key spaces according to the data quantity, and finishing the index writing step after the merging is finished.
2. The key-value data indexing method of claim 1, wherein the threshold β takes on a value of 64 MB.
3. The key-value data indexing method of claim 1, further comprising an adaptive read optimization method comprising the sub-steps of:
(a) will be provided with
Figure FDA0002433639270000012
The sub-key space l as time tiThe heat reading statistics of (a) for the time t are obtained as:
Figure FDA0002433639270000013
wherein the content of the first and second substances,
Figure FDA0002433639270000014
between t-1 and t, the sub-key space liα is an attenuation factor, a is more than 0 and less than 1,
Figure FDA0002433639270000015
and (b) and (c).
Figure FDA0002433639270000016
(b) Will be provided with
Figure FDA0002433639270000017
The sub-key space l as time tiThe write heat statistics of (1) for the write heat at time t are:
Figure FDA0002433639270000018
wherein the content of the first and second substances,
Figure FDA0002433639270000021
between t-1 and t, the sub-key space liThe number of times of writing of (a),
Figure FDA0002433639270000022
and (b) and (c).
Figure FDA0002433639270000023
I(t)The time elapsed between the current time t and the last time t-1.
(c) The following process was run periodically: root of firstClustering the sub-key space according to the writing heat degree, and dividing the sub-key space into four types, namely: cold, Warm, WriteBalanced, WriteHeavy. Selecting the child key space of the Cold class with the lowest writing heat degree, then sorting the child key spaces of the Cold class in a descending order according to the reading heat degree, and filtering out the child key space l meeting the following conditionsi：
Figure FDA0002433639270000024
Then, the first P, l in the sorted and filtered Cold-type sub-key space set are selected1，l2，...，lPSpace l of each sub-keyiGamma (l) ofi) (i is more than or equal to 1 and less than or equal to P) are combined into a Run.
CN202010244527.4A 2020-03-31 2020-03-31 Key value data indexing method for workload adaptive single-layer LSMT Active CN111475507B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010244527.4A CN111475507B (en) 2020-03-31 2020-03-31 Key value data indexing method for workload adaptive single-layer LSMT

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010244527.4A CN111475507B (en) 2020-03-31 2020-03-31 Key value data indexing method for workload adaptive single-layer LSMT

Publications (2)

Publication Number Publication Date
CN111475507A true CN111475507A (en) 2020-07-31
CN111475507B CN111475507B (en) 2022-06-21

Family

ID=71749332

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010244527.4A Active CN111475507B (en) 2020-03-31 2020-03-31 Key value data indexing method for workload adaptive single-layer LSMT

Country Status (1)

Country Link
CN (1) CN111475507B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112115113A (en) * 2020-09-25 2020-12-22 北京百度网讯科技有限公司 Data storage system, method, device, equipment and storage medium
CN112527804A (en) * 2021-01-27 2021-03-19 中智关爱通(南京)信息科技有限公司 File storage method, file reading method and data storage system
CN117725095A (en) * 2024-02-07 2024-03-19 北京四维纵横数据技术有限公司 Data storage and query method, device, equipment and medium for data set
WO2024066597A1 (en) * 2022-09-29 2024-04-04 华为云计算技术有限公司 Data storage method and apparatus
CN118092812A (en) * 2024-04-18 2024-05-28 华侨大学 Key value storage and read-write method based on memory table index and iterator reduction mechanism

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105159915A (en) * 2015-07-16 2015-12-16 中国科学院计算技术研究所 Dynamically adaptive LSM (Log-structured merge) tree combination method and system
US20180349095A1 (en) * 2017-06-06 2018-12-06 ScaleFlux, Inc. Log-structured merge tree based data storage architecture
CN110168532A (en) * 2017-05-09 2019-08-23 华为技术有限公司 Data-updating method and storage device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105159915A (en) * 2015-07-16 2015-12-16 中国科学院计算技术研究所 Dynamically adaptive LSM (Log-structured merge) tree combination method and system
CN110168532A (en) * 2017-05-09 2019-08-23 华为技术有限公司 Data-updating method and storage device
US20180349095A1 (en) * 2017-06-06 2018-12-06 ScaleFlux, Inc. Log-structured merge tree based data storage architecture

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112115113A (en) * 2020-09-25 2020-12-22 北京百度网讯科技有限公司 Data storage system, method, device, equipment and storage medium
CN112527804A (en) * 2021-01-27 2021-03-19 中智关爱通(南京)信息科技有限公司 File storage method, file reading method and data storage system
CN112527804B (en) * 2021-01-27 2022-09-16 中智关爱通（上海）科技股份有限公司 File storage method, file reading method and data storage system
WO2024066597A1 (en) * 2022-09-29 2024-04-04 华为云计算技术有限公司 Data storage method and apparatus
CN117725095A (en) * 2024-02-07 2024-03-19 北京四维纵横数据技术有限公司 Data storage and query method, device, equipment and medium for data set
CN117725095B (en) * 2024-02-07 2024-05-03 北京四维纵横数据技术有限公司 Data storage and query method, device, equipment and medium for data set
CN118092812A (en) * 2024-04-18 2024-05-28 华侨大学 Key value storage and read-write method based on memory table index and iterator reduction mechanism

Also Published As

Publication number Publication date
CN111475507B (en) 2022-06-21

Similar Documents

Publication Publication Date Title
CN111475507B (en) Key value data indexing method for workload adaptive single-layer LSMT
US9449005B2 (en) Metadata storage system and management method for cluster file system
CN110825748B (en) High-performance and easily-expandable key value storage method by utilizing differentiated indexing mechanism
CN108139968B (en) Method and equipment for determining quantity of threads of garbage collector and managing activities
CN109299113B (en) Range query method with storage-aware mixed index
Teng et al. LSbM-tree: Re-enabling buffer caching in data management for mixed reads and writes
CN107888687B (en) Proxy client storage acceleration method and system based on distributed storage system
CN113553476B (en) Key value storage method for reducing write pause by utilizing hash
CN114356877A (en) Log structure merged tree hierarchical storage method and system based on persistent memory
CN113268457B (en) Self-adaptive learning index method and system supporting efficient writing
CN114416646A (en) Data processing method and device of hierarchical storage system
CN114020779B (en) Self-adaptive optimization retrieval performance database and data query method
CN111857582B (en) Key value storage system
CN110990340A (en) Big data multi-level storage framework
CN108897618B (en) Resource allocation method based on task perception under heterogeneous memory architecture
CN111444036B (en) Data relevance perception erasure code memory replacement method, equipment and memory system
CN114996275A (en) Key value storage method based on multi-tree conversion mechanism
CN115328387A (en) Data compression method and device
CN113760171A (en) Metadata storage method and device
CN110321349B (en) Self-adaptive data merging and storing method for data origin system
US20240086362A1 (en) Key-value store and file system
US20240070135A1 (en) Hash engine for conducting point queries
US20150278117A1 (en) Method and apparatus method and apparatus for controlling access to a hash-based disk
CN117370414A (en) Application method and device of time sequence database
CN111694847B (en) Update access method with high concurrency and low delay for extra-large LOB data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant