WO2023108360A1 - Procédé et appareil de gestion de données dans un système de stockage - Google Patents

Procédé et appareil de gestion de données dans un système de stockage Download PDF

Info

Publication number
WO2023108360A1
WO2023108360A1 PCT/CN2021/137522 CN2021137522W WO2023108360A1 WO 2023108360 A1 WO2023108360 A1 WO 2023108360A1 CN 2021137522 W CN2021137522 W CN 2021137522W WO 2023108360 A1 WO2023108360 A1 WO 2023108360A1
Authority
WO
WIPO (PCT)
Prior art keywords
target data
data
data blocks
length
window
Prior art date
Application number
PCT/CN2021/137522
Other languages
English (en)
Chinese (zh)
Inventor
张海波
郭小东
唐飞龙
李旭
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2021/137522 priority Critical patent/WO2023108360A1/fr
Priority to CN202180102494.2A priority patent/CN118043799A/zh
Publication of WO2023108360A1 publication Critical patent/WO2023108360A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation

Definitions

  • the embodiments of the present application relate to the field of data storage, and in particular, to a data management method and device in a storage system.
  • Merkle-DAG Merkle directed acyclic graph
  • Merkle-DAG is composed of at least one tree, that is, data management is based on the tree structure.
  • a method for managing data in a storage system based on Merkle-DAG is: divide the target data (that is, the object to be managed) into multiple data blocks according to a fixed size (such as a fixed byte), and calculate the The hash value of the block; then, store the hash values of the multiple data blocks and the multiple data blocks according to the corresponding relationship between the data blocks and the hash values of the data blocks.
  • the Merkle-DAG of the target data is called an index tree), and this index tree is saved.
  • the application layer updates the target data (such as data insertion, data modification, and data deletion)
  • the total data volume of the target data may change. If the target data is divided into blocks according to the above-mentioned fixed size, it may cause The number of divided data blocks changes, which in turn leads to a large change in the number of groups of data blocks, so that the index tree needs to be rebuilt. Therefore, more resources need to be consumed in the process of data management.
  • Embodiments of the present application provide a data management method and device in a storage system, which can save resources required for data management.
  • an embodiment of the present application provides a data management method and device in a storage system, the method comprising: dividing the target data into M candidate data blocks based on the content of the target data, where M is an integer greater than or equal to 2;
  • the M candidate data blocks are divided into N target data blocks, N is a positive integer less than or equal to M, and each target data block includes at least one candidate data block; store N target data blocks and the fingerprint features of the N target data blocks, the target data block and the fingerprint features of the target data blocks have a one-to-one correspondence; according to the N target data blocks, an index tree of the target data is generated, and the index tree Used to address the contents of object data.
  • the data management device divides the target data into M candidate data blocks based on the content of the target data, and then divides the M candidate data blocks according to the respective fingerprint features of the M candidate data blocks.
  • the candidate data blocks are divided into N target data blocks, and finally, an index tree for addressing the target data is generated according to the N target data blocks.
  • the target data is first divided into multiple candidate data blocks based on the content of the target data, and then the candidate data blocks are divided into target data blocks according to the fingerprint characteristics of the candidate data blocks, instead of dividing the target data according to a fixed size
  • the data is divided into multiple target data blocks, so the size of the target data block in the embodiment of the present application is not specifically limited.
  • the number of target data blocks may not necessarily change after the target data after the inserted data is divided into target data blocks. Therefore, To a certain extent, it can save the resources consumed by data management.
  • the above-mentioned target data is divided into M candidate data blocks based on the content of the target data, which specifically includes: determining M-1 division points of the target data according to the fingerprint characteristics of the target data; The M-1 dividing points divide the target data into M candidate data blocks.
  • the determination of the M-1 division points of the target data according to the fingerprint features of the target data includes: for any one of the above-mentioned M-1 division points, the first division point in the sliding window When the fingerprint feature of a data satisfies the first preset condition, the end position of the sliding window is determined as the division point, the first data is part of the data in the target data, and the first preset condition is the target in the sliding window
  • the modulo value of the fingerprint feature of the data and the first threshold is equal to the second threshold.
  • the determination of the M-1 division points of the target data according to the fingerprint features of the target data includes: for any one of the above-mentioned M-1 division points, the first division point in the sliding window When the fingerprint feature of a data does not meet the first preset condition, slide the sliding window along the preset direction for a preset length, and when the fingerprint feature of the second data in the sliding window satisfies the first preset condition, slide the sliding window
  • the end position is determined as a division point
  • the second data is part of the target data, and the second data is different from the above-mentioned first data; wherein, the preset length is less than or equal to the length of the sliding window, and the above-mentioned first preset condition
  • a modulo value between the fingerprint feature of the target data in the sliding window and the first threshold is equal to the second threshold.
  • the above-mentioned division of the target data into M candidate data blocks based on the content of the target data includes: determining M-1 division points of the target data according to the transformation value of the above-mentioned target data, and the transformation The value is a value converted into a digital form of each data in the preset window based on a preset rule; the target data is divided into M candidate data blocks according to the M-1 division points.
  • the above-mentioned M-1 division points of the target data are determined according to the transformation value of the target data, including: the preset window includes a first fixed-length window, a variable-length window, and a second Fixed-length window; the length of the second fixed-length window is the length of a data in the target data or the length of the transformation value of a data in the target data; for any one of the above-mentioned M-1 division points,
  • the end position of the second fixed-length window is determined as the division point, and the second preset condition is The transformation value of the data is greater than the maximum value of the transformation values of each data in the first fixed-length window, and greater than the maximum value of the transformation values of each data in the variable-length window.
  • the above-mentioned M-1 division points of the target data are determined according to the transformation value of the target data, including: the above-mentioned preset window includes the first fixed-length window, the variable-length window and the first Two fixed-length windows; the length of the second fixed-length window is the length of a data in the above-mentioned target data or the length of the transformation value of a data in the target data; for any division point, in the second fixed-length window
  • the conversion value of the data does not meet the second preset condition, increase the length of the variable-length window in the preset window, and when the conversion value of the data in the second fixed-length window satisfies the second preset condition, the The end position of the second fixed-length window is determined as a division point, and the above-mentioned second preset condition is that the conversion value of the data in the second fixed-length window is greater than the maximum value of the conversion values of each data in the first fixed-length window, and Greater than the maximum value of the transformation
  • the M candidate data blocks are divided into N target data blocks, which specifically includes: the fingerprint features of the M candidate data blocks satisfy the third predetermined Assume that the end position of the candidate data block of the condition is determined as the N-1 division points of the candidate data block, wherein the third preset condition is that the fingerprint feature of the data of the candidate data block and the modulo value of the third threshold are within the preset within the range; according to the N-1 dividing points, the M candidate data blocks are divided into N target data blocks.
  • the fingerprint feature of the data in the candidate data block is the fingerprint feature of all the data in the candidate data block; or, the fingerprint feature of the data in the candidate data block is the fingerprint feature of some data in the candidate data block.
  • the above-mentioned generation of the index tree of the target data based on the N target data blocks specifically includes: dividing the N target data blocks into at least one data block according to the respective fingerprint features of the N target data blocks group; generate an index tree of target data based on the respective fingerprint characteristics of multiple data groups.
  • the N target data blocks are divided into at least one data group, which specifically includes: among the fingerprint features of the N target data blocks satisfying the fourth A target data block with preset conditions is determined as at least one division point of the N target data blocks; and the N target data blocks are divided into multiple data groups according to at least one division point of the target data block.
  • the fourth preset condition is: a modulo value between the fingerprint feature of the target data block and the fourth threshold is greater than or equal to the fifth threshold.
  • the above fourth preset condition is: the modulo value between the fingerprint feature of the target data block and the fourth threshold is greater than or equal to the fifth threshold, and the number of target data blocks in the data group is within the first number Between the threshold and the second threshold number, the first number threshold is greater than the second number threshold.
  • the fingerprint feature is a hash value.
  • the embodiment of the present application provides a data management device in a storage system
  • the data management device includes: a processing module, a storage module and a generation module; the processing module is used to divide the target data into M based on the content of the target data Candidate data blocks, M is an integer greater than or equal to 2; the processing module is also used to divide the M candidate data blocks into N target data blocks according to the respective fingerprint characteristics of the M candidate data blocks, and N is less than or equal to M is a positive integer, each target data block includes at least one candidate data block; the storage module is used to store N target data blocks and the fingerprint features of the N target data blocks, and the target data block and the fingerprint feature of the target data block have the same One-to-one relationship; the generation module is used to generate an index tree of the target data according to the N target data blocks, and the index tree is used to address the contents of the target data.
  • the determination module is configured to determine M-1 division points of the target data according to the fingerprint characteristics of the target data; the processing module is specifically configured to divide the target data according to the M-1 division points are M candidate data blocks.
  • the data management device in the above-mentioned storage system further includes: a determination module; the determination module is configured to, when the fingerprint feature of the first data in the sliding window meets the first preset condition, set the The end position is determined as a division point, the first data is part of the target data, and the first preset condition is that the modulo value between the fingerprint feature of the target data in the sliding window and the first threshold is equal to the second threshold.
  • the data management device in the above-mentioned storage system further includes: a sliding module; the sliding module is used for any one of the above-mentioned M-1 dividing points, the fingerprint of the first data in the sliding window When the feature does not meet the first preset condition, slide the sliding window along the preset direction for a preset length; when the fingerprint feature of the second data in the sliding window meets the first preset condition, the determination module will slide the sliding window The end position is determined as a division point, the second data is part of the target data, and the second data is different from the above-mentioned first data; wherein, the preset length is less than or equal to the length of the sliding window, and the above-mentioned first preset condition A modulo value between the fingerprint feature of the target data in the sliding window and the first threshold is equal to the second threshold.
  • the data management device in the above-mentioned storage system further includes: a determination module; the determination module is configured to determine M-1 division points of the target data according to the transformation value of the above-mentioned target data, and the transformation value is based on the predetermined A rule is set to convert each data in the preset window into a value in digital form; the processing module is used to divide the target data into M candidate data blocks according to the M-1 division points.
  • the preset window includes a first fixed-length window, a variable-length window, and a second fixed-length window that are sequentially adjacent;
  • the length of the second fixed-length window is the length of one of the target data length or the length of the converted value of a data in the target data;
  • the above-mentioned processing module is used to convert the converted value of the data included in the second fixed-length window to the second preset condition.
  • the end position is determined as the division point, and the second preset condition is that the transformation value of the data in the second fixed-length window is greater than the maximum value of the transformation values of each data in the first fixed-length window, and greater than the maximum value of the transformation values of the data in the variable-length window. The maximum value of the transformation value of each data.
  • the preset window includes a first fixed-length window, a variable-length window, and a second fixed-length window that are sequentially adjacent;
  • the length of the second fixed-length window is one of the above-mentioned target data length or the length of the conversion value of a data in the target data;
  • the processing module is used to increase the available data in the preset window when the conversion value of the data in the second fixed-length window does not meet the second preset condition
  • the length of the variable-length window when the conversion value of the data in the second fixed-length window satisfies the second preset condition, the end position of the second fixed-length window is determined as the division point, and the above-mentioned second preset condition is the second
  • the transformation value of the data in the fixed-length window is greater than the maximum value of the transformation values of each data in the first fixed-length window, and greater than the maximum value of the transformation values of each data in the variable-length window.
  • the determination module is used to determine the end positions of the candidate data blocks whose fingerprint features meet the third preset condition among the M candidate data blocks as the N-1 division points of the candidate data blocks, where the The three preset conditions are that the fingerprint feature of the data of the candidate data block and the modulo value of the third threshold are within the preset range; the processing module is used to divide the M candidate data blocks into three according to the N-1 dividing points N target data blocks.
  • the above-mentioned processing module is used to divide the N target data blocks into at least one data group according to the respective fingerprint features of the above-mentioned N target data blocks; Fingerprint features to generate an index tree of the target data.
  • the determination module is configured to determine a target data block satisfying the fourth preset condition among the respective fingerprint features of the N target data blocks as at least one division point of the N target data blocks;
  • the module is used for dividing the N target data blocks into multiple data groups according to at least one dividing point of the target data blocks.
  • the fourth preset condition is: a modulo value between the fingerprint feature of the target data block and the fourth threshold is greater than or equal to the fifth threshold.
  • the above fourth preset condition is: the modulo value between the fingerprint feature of the target data block and the fourth threshold is greater than or equal to the fifth threshold, and the number of target data blocks in the data group is within the first number Between the threshold and the second threshold number, the first number threshold is greater than the second number threshold.
  • the fingerprint feature is a hash value.
  • an embodiment of the present application provides a data management device in a storage system, wherein the memory is coupled to the processor; the memory is used to store computer program codes, wherein the computer program codes include computer instructions; when the computer instructions are executed by the processor , make the data management device in the storage system execute the method described in any one of the first aspect and its possible implementation manners.
  • an embodiment of the present application provides a computer storage medium, including computer instructions.
  • the computing device is made to execute the above-mentioned method described in any one of the first aspect and its possible implementations. method.
  • the embodiments of the present application provide a computer program product, which, when run on a computer, causes the computer to execute the method described in any one of the above first aspect and possible implementations thereof.
  • FIG. 1 is a schematic diagram of a block and group flow process of target data provided by an embodiment of the present application
  • FIG. 2 is a schematic structural diagram of an index tree provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a construction process of an index tree provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a hardware structure of a storage system provided by an embodiment of the present application.
  • FIG. 5 is a first schematic flowchart of a data management method in a storage system provided by an embodiment of the present application
  • FIG. 6 is a second schematic flow diagram of a data management method in a storage system provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a division process of a candidate data block provided in an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a method for dividing candidate data blocks provided by an embodiment of the present application.
  • FIG. 9 is a third schematic flowchart of a data management method in a storage system provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of another candidate data block division process provided by the embodiment of the present application.
  • FIG. 11 is a schematic flowchart of another method for dividing candidate data blocks provided by the embodiment of the present application.
  • FIG. 12 is a fourth schematic flowchart of a data management method in a storage system provided by an embodiment of the present application.
  • FIG. 13 is a schematic flow diagram V of a data management method in a storage system provided by an embodiment of the present application.
  • FIG. 14 is a sixth schematic flow diagram of a data management method in a storage system provided by an embodiment of the present application.
  • FIG. 15 is a schematic diagram of a data management device in a storage system provided by an embodiment of the present application.
  • FIG. 16 is a schematic diagram of another data management device in a storage system provided by an embodiment of the present application.
  • first and second in the description and claims of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order of objects.
  • first threshold and the second threshold are used to distinguish different thresholds, but not to describe a specific order of the thresholds.
  • words such as “exemplary” or “for example” are used as examples, illustrations or illustrations. Any embodiment or design scheme described as “exemplary” or “for example” in the embodiments of the present application shall not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Rather, the use of words such as “exemplary” or “such as” is intended to present related concepts in a concrete manner.
  • plurality means two or more.
  • a plurality of data groups refers to two or more data groups.
  • Merkle-DAG is composed of at least one tree, that is, data management is based on the tree structure.
  • the method of managing data in the storage system based on Merkle-DAG is as follows: first, divide the target data into multiple data blocks according to a fixed size, assuming that the size of the target data is 1024 bytes, as shown in Figure 1 As shown in (A) in Figure 1, the target data is divided into 4 data blocks according to the size of each data block is 256 bytes, that is, data block 1-data block shown in Figure (B) in Figure 1 4; Secondly, calculate the hash value of each data block separately, for example, the hash value of the above-mentioned data block 1-data block 4 is hash value 1-hash value 4, and according to the corresponding relationship between the data block and the hash value Store the data block and the hash value of the data block in the preset table, and the following table 1 is an example of the corresponding relationship between the above-mentioned data block 1-data block 4 and its hash
  • the generation method of the first index tree of the above target data is as follows: divide the target data into hash values corresponding to data block 1-data block 4 1-Hash value 4 is used as the leaf node of the first index tree, which can be recorded as leaf node 1, leaf node 2, leaf node 3, and leaf node 4; then, calculate the two data groups divided by these 4 leaf nodes
  • the hash value of the leaf node that is, calculate the hash value of hash value 1 and hash value 2, and the hash value of hash value 3 and hash value 4, and combine hash value 1 and hash value 2
  • the parent node of leaf node 1 and leaf node 2 (called the first parent node, the first parent node is a child node in the first index tree)
  • hash value 3 and hash value 4 as the parent node of leaf node 3 and leaf node 4 (called the second parent node, the second parent node is another child node in the first index tree)
  • the target data When the target data is updated, it may cause the total data volume of the target data to change. For example, with reference to FIG.
  • the data volume of the target data is increased from 1024 bytes to 1280 bytes. If the size of each data block is fixed at 256 bytes, the target data is divided into blocks, and 5 data blocks can be obtained, and then the After the 5 data blocks are grouped according to the above grouping method, 3 data groups can be obtained.
  • Data group 1 of the 3 data groups includes data block 1 and data block 5
  • data group 2 includes data block 2 and data block 3.
  • Data group 3 includes data block 4, and before new data is inserted, the target data is divided into two data groups, data group 1 includes data block 2 and data block 2, data group 2 includes data block 3 and data block 4, It can be seen that the total data volume of the target data has changed, resulting in a large change in the grouping of the target data. In this way, the second index tree shown in Figure 3 is generated based on the hash values of data blocks 1-5.
  • the second index tree has a large change, such as: the shaded part in Figure 3 is the part that has changed, therefore, it takes a lot of time to generate the index tree for the target data after inserting the data
  • the amount of calculation leads to the consumption of more resources in the process of data management in the above-mentioned prior art.
  • the embodiment of the present application provides a data management method and device in a storage system.
  • the data management device divides the target data into M candidate data blocks based on the content of the target data, and M is greater than or an integer equal to 2; according to the respective fingerprint characteristics of the M candidate data blocks, the M candidate data blocks are divided into N target data blocks, N is a positive integer less than or equal to M, and each target data block includes at least one candidate Data block; store N target data blocks and fingerprint features of N target data blocks, and target data blocks have a one-to-one correspondence with the fingerprint features of target data blocks; generate an index tree of target data according to N target data blocks, The index tree is used to address the content of the target data.
  • the data management method and device in the storage system provided by the embodiment of the present application can be applied to the storage system shown in Figure 4.
  • the storage system can be a storage system composed of a solid-state hard disk, or a storage system composed of other types of storage media. system.
  • the storage system includes a controller (abbreviation: main control) 401 and a plurality of hard disks 405, wherein the main control 401 includes: a processor 402, optionally, the controller 401 also includes a host interface 404, and n (n>0) channel controllers 403 .
  • the above-mentioned master control 401 is used to issue executable commands to multiple hard disks 405 , so as to read or update data on the hard disks 405 .
  • the above-mentioned host interface 404 is used to communicate with the host, and then receive the command request sent by the host, and forward the command request to the processor 402, wherein the above-mentioned host is not limited to any device such as server, personal computer or array controller.
  • the above-mentioned processor 402 sends executable commands to the above-mentioned multiple hard disks 405 according to the command request forwarded by the host interface 404.
  • the above-mentioned processor 402 is used to execute the data management method in the storage system provided by the embodiment of the present application, for example, processing
  • the implementer 402 is used to block target data, group data blocks, and generate an index tree.
  • the processor 402 may include one or more CPUs, and the CPUs may be single-core CPUs (single-CPU) or multi-core CPUs (multi-CPU).
  • the channel controller 403 is used to carry the executable commands issued by the processor 402 to the hard disk 405 .
  • the storage system further includes a bus 406, and the processor 402, the channel controller 403, the host interface 404, and the hard disk 405 are generally connected to each other through the bus 406, or are connected to each other in other ways.
  • the host interface 404 in the main control 401 forwards the target data to the processor 402 in the main control 401, and the processor 402 divides the target data into M candidate data blocks, According to the respective fingerprint features of the M candidate data blocks, the M candidate data blocks are divided into N target data blocks, and then, the processor 402 divides the N target data blocks according to the corresponding relationship between the target data blocks and the fingerprint features of the target data blocks.
  • the target data block and the fingerprint features of the N target data blocks are sent to n hard disks 405 through the channel controller 403 to be stored in the hard disks 405; finally, the processor 402 generates the target data according to the N target data blocks. index tree, and store the index tree in the hard disk 405.
  • the device for executing the data management method in the storage system may be the processor 402 in the controller in the storage system shown in FIG. 4 above.
  • the data management method in the storage system may include S510-S540.
  • the data management device divides the target data into M candidate data blocks based on the content of the target data.
  • M is an integer greater than or equal to 2.
  • the above M candidate data blocks may be M data blocks that the data management device determines M-1 division points based on the content of the target data, and then cuts the target data into M data blocks according to the M-1 division points;
  • the content of the target data determines M-1 division points in the content of the target data, and the M-1 division points divide the target data into M intervals, and there is no need to cut the target data according to the M-1 division points , each interval is a candidate data block, and the specific embodiment of the present application does not limit the division method of the above M candidate data blocks.
  • the content of the above-mentioned target data refers to all elements that make up the target data, and the sizes of the above-mentioned M candidate data blocks may be all the same, may be partly the same, or may be different from each other.
  • the data management device divides the M candidate data blocks into N target data blocks according to the respective fingerprint features of the M candidate data blocks.
  • N is a positive integer less than or equal to M.
  • the fingerprint feature of the above candidate block may be the hash value of the candidate block, or other features that can uniquely identify a data block, which are determined according to actual needs. To limit.
  • the above S520 is specifically: the data management device determines N-1 division points according to the respective fingerprint characteristics of the M candidate data blocks, and then, the data management device divides the M candidate data blocks into N according to the N-1 division points. target data blocks, and each target data block includes at least one candidate data block.
  • the data management device stores N target data blocks and fingerprint features of the N target data blocks.
  • the above steps are specifically: storing the fingerprint feature of a certain target data block and the content of the target data block in the same row of the preset table, or storing the fingerprint feature of the target data block and the target data block ( index of the target data block), the index of the target data block and the content of the target data block are stored in the preset table 2, and the application does not specify the storage method of the N target data blocks and the fingerprint features of the N target data blocks To limit.
  • the data management device generates an index tree of the target data according to the N target data blocks, and the index tree is used for addressing content of the target data.
  • the corresponding index tree of the target data needs to be determined, and then the leaf nodes of the index tree are searched recursively according to the root node of the index tree, and then according to the hash in the leaf node Hash the content of the data block corresponding to the leaf node to obtain the target data
  • the data management device searches the child nodes (the first parent node and the second parent node) of the index tree according to the root node of the index tree. node), and then find 4 leaf nodes according to the child nodes, and finally, query the content of the data block corresponding to the 4 hash values in the preset table according to the hash values in the 4 leaf nodes, so as to obtain the target data.
  • the data management device searches the child nodes (the first parent node and the second parent node) of the index tree according to the root node of the index tree. node), and then find 4 leaf nodes according to the child nodes, and finally, query the content of the data block corresponding to the 4 hash values in the preset table according to the hash values in the 4 leaf nodes, so as to obtain the target data.
  • the data management device divides the target data into M candidate data blocks based on the content of the target data, and then divides the M candidate data blocks according to the respective fingerprint features of the M candidate data blocks.
  • the candidate data blocks are divided into N target data blocks, and finally, an index tree for addressing the target data is generated according to the N target data blocks.
  • the target data is first divided into multiple candidate data blocks based on the content of the target data, and then the candidate data blocks are divided into target data blocks according to the fingerprint characteristics of the candidate data blocks, instead of dividing the target data according to a fixed size
  • the data is divided into multiple target data blocks, so the size of the target data block in the embodiment of the present application is not specifically limited.
  • the number of target data blocks may not necessarily change after the target data after the inserted data is divided into target data blocks. Therefore, To a certain extent, it can save the resources consumed by data management.
  • the method for dividing the target data into M candidate data blocks based on the content of the target data may specifically include: S610-S620.
  • the data management device determines M ⁇ 1 division points of the target data according to the fingerprint feature of the target data.
  • the method of determining each division point is the same. As shown in FIG. 8 , the method of determining a division point of the target data includes S810-S830.
  • the data management device judges whether the fingerprint feature of the data in the sliding window satisfies a first preset condition.
  • the size of the above-mentioned sliding window is fixed, the sliding step of the sliding window is a preset length, and the sliding step of the sliding window can be set according to actual needs, for example, the sliding step is 1 byte or 2 bytes, etc. This embodiment of the present application does not limit it.
  • the above-mentioned first preset condition is that the modulo value between the fingerprint feature of the data in the sliding window and the first threshold is equal to the second threshold, where the first threshold and the second threshold may be pre-configured.
  • the modulo value of the hash value 2113 of the data in the sliding window and the first threshold 20 is 13, and the second threshold is 9, that is: the hash value of the data in the sliding window and the first threshold
  • the modulo value is not equal to the second threshold; therefore, the hash value of the data in the sliding window does not satisfy the first preset condition.
  • the data management device determines an end position of the sliding window as a division point.
  • the above-mentioned first data is part of the target data.
  • the end position of the sliding window is the position closest to the sliding direction on the sliding window.
  • the end position of the sliding window at this time is the letters "C" and "D”. "The junction position.
  • the data management device slides the sliding window along a preset direction for a preset length.
  • the preset length is the sliding step of the sliding window, and the preset length is less than or equal to the length of the sliding window.
  • the sliding direction of the above-mentioned sliding window is pre-configured, and the sliding direction can be from right to left, or from left to right. In the embodiment of this application, the sliding direction of the sliding window is rightward An example is used for description, and details will not be described later.
  • the data management device executes S830, it continues to execute the above S810.
  • the end position of the sliding window is determined as the division point, and the second data is part of the data in the target data; that is, after the data management device executes S830, it continues to execute the above S810 until the fingerprint feature of the data in the sliding window satisfies the first preset condition, and determines the end position of the sliding window as dividing point.
  • the hash value of the data (that is, BC) in the sliding window does not meet the first preset condition , slide the sliding window to the right by one letter, currently, the data in the sliding window is "CD", as shown in (B) in Figure 7; at this time, determine the hash of the data "CD" in the sliding window Whether the value satisfies the first preset condition, and if so, determine the boundary position between "C" and "D” as the dividing point.
  • the data management device divides the target data into M candidate data blocks according to the M-1 division points.
  • the target data as "ABCDEF...XYZ" as an example, assuming that according to the above S610, 4 division points of the target data are determined, and the target data is divided into 5 intervals according to these 4 division points, then the The 5 intervals correspond to 5 candidate data blocks, and the 5 candidate data blocks are respectively ⁇ ABCDEF ⁇ , ⁇ GHIJK ⁇ , ⁇ LMNOP ⁇ , ⁇ QRSTU ⁇ and ⁇ VWXYZ ⁇ .
  • the target data is updated (such as inserting data into the target data)
  • the content of the target data has changed, but the position in the updated target data that satisfies the first preset condition may not occur Changes, and then after the updated target data is divided into candidate blocks, the number of candidate blocks will not change, and the content of most candidate blocks may also remain unchanged.
  • the first threshold is 20 and the second threshold is 9; when the data "AALMXXWX" is inserted into the target data as shown in (A) or (B) , assuming that the insertion position is between "C” and "D” in the above candidate data block ⁇ ABCDEF ⁇ , then for the new target data, execute the above S610-S620, because the target data after inserting the data satisfies the first preset The position of the condition has not changed, so the number and position of the division points are determined based on the above-mentioned technical solution of S810-S830, and the target data is divided into 5 candidate data blocks according to the 4 division points, and the 5 candidate data blocks
  • the blocks are ⁇ ABCAALMXXWXDEF ⁇ , ⁇ GHIJK ⁇ , ⁇ LMNOP ⁇ , ⁇ QRSTU ⁇ , and ⁇ VWXYZ ⁇ .
  • the above-mentioned method of dividing the target data into M candidate data blocks (that is, S510) based on the content of the target data may include: S910- S920.
  • the data management device determines M-1 division points of the target data according to the conversion value of the target data.
  • the above-mentioned conversion value is based on a preset rule to convert each data in the preset window into a value in digital form.
  • the transformation value of each data above is the value obtained after transforming each data in the target data by using a transformation method.
  • the conversion value of a letter can be the ASCII (American standard code for information interchange, ASCII) corresponding to the letter, or the hash value corresponding to the letter or other values represented in digital form .
  • ASCII American standard code for information interchange
  • FIG. 10 it is the conversion value text of the target data obtained by converting the content of the target data into numbers according to preset rules.
  • each division point of the target data is determined by using a preset window
  • the method is specifically shown in Figure 11, including S1110-S1130.
  • the data management device judges whether the conversion value of the data in the second fixed-length window in the preset window satisfies a second preset condition.
  • the above-mentioned preset window includes the first fixed-length window, the variable-length window and the second fixed-length window adjacent in sequence; wherein, the length of the first fixed-length window is an integer greater than 0; the variable-length window The initial length is a preset value, and the preset value is an integer greater than or equal to 0; the length of the second fixed-length window is the length of a data in the target data or the length of a transformed value of a data in the target data, That is to say, when the length of the variable-length window is greater than 0, the length of the second fixed-length window is the smallest unit of data corresponding to the variable-length window and the first fixed-length window, for example: when the length of the first fixed-length window is 4 bytes, and when the length of the variable-length window is 2 bytes, the length of the second fixed-length window is 1 byte.
  • the above-mentioned second preset condition is that the conversion value of the data in the second fixed-length window is greater than the maximum value of the conversion value of each data in the first fixed-length window, and is greater than the maximum value of the conversion value of each data in the variable-length window. value.
  • the data management device determines the maximum value of each transformation value in the first fixed-length window, and the maximum value is called the first maximum value; the data management device then determines the maximum value of each transformation value in the variable-length window , that is: the second maximum value; then, the data management device judges whether the transformation value in the second fixed-length window is greater than the first maximum value and also greater than the second maximum value.
  • the data management device determines an end position of the second fixed-length window as a division point.
  • the data management device increases the length of the variable-length window in the preset window.
  • the above-mentioned data management device may increase the length of the variable-length window in the preset window according to a pre-configured length, and the pre-configured length may be determined according to actual conditions, for example, configured as 2 bytes.
  • variable-length window is increased by a preset length
  • second fixed-length window is adjacent to the variable-length window, the second fixed-length window will move backward by a preset length.
  • the data management device judges whether the transformation value of the data in the current second fixed-length window satisfies the second preset condition, and if so, determines the end position of the current second fixed-length window as the dividing point ; If not satisfied, then continue to increase the length of the variable-length window in the preset window until the conversion value of the data in the second fixed-length window meets the second preset condition, and determine the end position of the second fixed-length window as the dividing point.
  • the data management device determines the end position of the second fixed-length window as the division point.
  • the initial position of the preset window for determining the next division point is conversion value 5
  • the transformation values in the first fixed-length window in the preset window include 5 and 9
  • the transformation values in the variable-length window include 36
  • the transformation values in the second fixed-length window include 5.
  • the data management device divides the target data into M candidate data blocks according to the M-1 division points.
  • the conversion value text of the target data can be divided into four candidate data blocks, for example, respectively: ⁇ 12,18,2,6,45 ⁇ , ⁇ 5,9,36,5, 5,65 ⁇ , ⁇ 56,5,9,7,62 ⁇ , and ⁇ 8,8,432,9,81,20 ⁇ .
  • the target data is updated (such as: inserting data in the target data)
  • the content of the target data has changed, but the position in the updated target data that satisfies the second preset condition may not occur Changes, and then after the updated target data is divided into candidate blocks, the number of candidate blocks will not change, and the content of most candidate blocks may also remain unchanged.
  • the conversion value set of the inserted data is ⁇ 9,10,12,1,-40 ⁇ , assuming the position of the inserted data is the middle of 18 and 2 in the candidate data block ⁇ 12, 18, 2, 6, 45 ⁇ , since the position in the target data after inserting the data that satisfies the second preset condition has not changed, so based on the above S1110-S1130
  • the technical solution determines the number and position of the division points, and divides the target data into 4 candidate data blocks according to the 3 division points, and the 4 candidate data blocks are respectively ⁇ 12, 18, 9, 10, 12, 1 ,-40,2,6,45 ⁇ , ⁇ 5,9,36,5,5,65 ⁇ , ⁇ 56,5,9,7,62 ⁇ , and ⁇ 8,8,432,9,81,20 ⁇ .
  • the M candidate data blocks are divided into N target data blocks according to the respective fingerprint features of the M candidate data blocks (ie: S520), including: S1210-S1220.
  • the data management device determines, among the M candidate data blocks, the end positions of the candidate data blocks whose fingerprint features meet the third preset condition as N ⁇ 1 division points of the candidate data blocks.
  • the fingerprint feature of the above candidate data block can be a fingerprint feature corresponding to all the data in the candidate data block. For example, if the data in the candidate data block is "WANHH", then the hash value of the candidate data block is "WANHH".
  • the overall hash value can also be the fingerprint feature corresponding to some data in the candidate data block; for example, if the data in the candidate data block is "WANHH", then the hash value of the candidate data block corresponds to "NHH".
  • the fingerprint features of the candidate data blocks are described by taking the fingerprint features corresponding to all the data in the candidate data blocks as an example, and will not be described in detail later.
  • the above-mentioned third preset condition is that the modulo value between the fingerprint feature of the data of the candidate data block and the third threshold is within a preset range.
  • the data management device divides the M candidate data blocks into N target data blocks according to the N-1 division points.
  • the target data is divided into 5 candidate data blocks, and the 5 candidate data blocks are respectively ⁇ ABCAALMXXWXDEF ⁇ , ⁇ GHIJK ⁇ , ⁇ LMNOP ⁇ , ⁇ QRSTU ⁇ , and ⁇ VWXYZ ⁇
  • the calculated hash values of the five candidate data blocks are: -1130721247, 67787465, 72558990, 77330515, and 82102040; then, calculate the hash values of the five candidate data blocks and the first
  • the modulo values of the three thresholds are: -7, 5, 30, 55 and 20 respectively.
  • the candidate data block ⁇ LMNOP ⁇ satisfies the third preset condition, so the candidate data block ⁇ LMNOP ⁇ is used as the above
  • the division points of the 5 candidate data blocks can divide the candidate data blocks into 2 target data blocks, namely ⁇ ABCAALMXXWXDEFGHIJKLMNOP ⁇ and ⁇ QRSTU ⁇ .
  • the method for dividing the target data block in the above S520 may be similar to the method for dividing candidate data blocks S610-S620 and S910-S920.
  • the method for dividing candidate data blocks S610-S620 and S910-S920 may be similar to the method for dividing candidate data blocks S610-S620 and S910-S920.
  • S610-S620 and S910-S920 refer to the relevant descriptions of the above-mentioned S610-S620 and S910-S920, which will not be repeated here.
  • the data management device divides the target data into M candidate data blocks, and then, the data management device divides the candidate data whose fingerprint characteristics satisfy the third preset condition in the M candidate data blocks
  • the end positions of the data blocks are determined as N-1 division points of the candidate data blocks, and then N target data blocks are determined.
  • the index tree to be constructed is compared to the target data before inserting data. In the index tree, only the hash values of individual nodes have changed, thereby saving the resources consumed by data management.
  • the above S540 includes: S1310-S1320.
  • the data management device divides the N target data blocks into at least one data group according to the respective fingerprint features of the N target data blocks.
  • the above-mentioned fingerprint feature of the target data block can be a hash value of all data in the target data block, or a hash value corresponding to some data in the target data block.
  • the target data The fingerprint features of a block are illustrated by taking the hash value corresponding to all data in the target data block as an example.
  • the data in the target data block is ⁇ ABCAALMXXWXDEFGHIJKLMNOP ⁇
  • the fingerprint feature of the target data block is the hash value of "ABCAALMXXWXDEFGHIJKLMNOP”.
  • the foregoing S1310 specifically includes: S1410-S1420.
  • the data management device determines a target data block satisfying a fourth preset condition among the fingerprint features of each of the N target data blocks as at least one division point of the N target data blocks.
  • the fourth preset condition above is:
  • a modulo value between the fingerprint feature of the target data block and the fourth threshold is greater than or equal to the fifth threshold.
  • the number of target data blocks in the data group is between the first number threshold and the second number threshold, and the modulo value of the fingerprint feature of the target data block and the fourth threshold is greater than or equal to the fifth threshold, wherein the first number The threshold is greater than the second number threshold.
  • the data management device divides the N target data blocks into multiple data groups according to at least one division point of the target data blocks.
  • the target data is divided into 4 target data blocks, namely: ⁇ ALMXXWXDEFGHIJK ⁇ , ⁇ ABRFGRTGRTRGE ⁇ , ⁇ DEWFRTNEBJ ⁇ and ⁇ JDIEOFJDEJFOEW ⁇ ; the fourth threshold is 80, and the fifth threshold is 70; obtained by calculation
  • the modulo values of the hash values of the above four target data blocks and the fourth threshold 80 are respectively 25, 75, 33 and 55. It can be seen that the target data block ⁇ ABRFGRTGRTRGE ⁇ satisfies the fourth preset condition.
  • the target data blocks are divided into two groups: ⁇ ALMXXWXDEFGHIJK ⁇ and ⁇ ABRFGRTGRTRGE ⁇ , and ⁇ DEWFRTNEBJ ⁇ and ⁇ JDIEOFJDEJFOEW ⁇ .
  • the above S1310 may also use the above methods S610-S620 and/or S910-S920 for dividing candidate blocks, for details, refer to the relevant descriptions of S610-S620 and/or S910-S920, which will not be repeated here.
  • the data management device generates an index tree of the target data based on the respective fingerprint features of the multiple data groups.
  • the data management device determines the target data block that satisfies the fourth preset condition among the respective fingerprint features of the N target data blocks as The at least one division point of the target data block is used to divide the N target data blocks into multiple data groups according to the at least one division point of the target data block.
  • the target data when the target data is updated, even if the number of target data blocks of the target data changes, but when the updated data of the target data (such as inserted data) does not meet the fourth preset condition, then the The division points for grouping the target data block remain unchanged, so the number of groups of the target data block is exactly the same as the number of groups of the target data block before updating the target data.
  • the index tree of the updated target data only the Part of the leaf nodes, part of the hash values of child nodes and the hash value of the root node of the index tree of the target data before the data is inserted can be modified, thereby saving resources required for data management.
  • the embodiment of the present application provides a data management device in the storage system, the data management device in the storage system is used to execute each step in the above-mentioned fingerprint verification method, the embodiment of the present application can use the example of the above-mentioned method for the storage system
  • the data management device divides the functional modules.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules.
  • the division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.
  • FIG. 15 shows a possible structural diagram of the data management device in the storage system involved in the above embodiment.
  • the data management device in the storage system includes: a processing module 1501 , a storage module 1502 and a generating module 1503 .
  • the processing module 1501 is configured to divide the target data into M candidate data blocks based on the content of the target data, for example, execute step S510 in the above method embodiment.
  • the processing module 1501 is further configured to divide the M candidate data blocks into N target data blocks according to their respective fingerprint features, for example, execute step S520 in the above method embodiment.
  • the storage module 1502 is configured to store N target data blocks and fingerprint features of the N target data blocks, for example, execute step S530 in the above method embodiment.
  • the generating module 1503 is configured to generate an index tree of the target data according to the N target data blocks, and the index tree is used to address the content of the target data, for example, execute step S540 in the above method embodiment.
  • the data management device in the storage system provided in the embodiment of the present application further includes a determination module 1504;
  • the determining module 1504 is configured to determine M-1 division points of the target data according to the fingerprint feature of the target data, for example, execute step S610 in the above method embodiment.
  • the processing module 1501 is specifically configured to divide the target data into M candidate data blocks according to the M-1 division points, for example, execute step S620 in the above method embodiment.
  • the data management device in the storage system provided in the embodiment of the present application further includes a sliding module 1505 .
  • the determining module 1504 is configured to determine the end position of the sliding window as the division point when the fingerprint feature of the first data in the sliding window satisfies the first preset condition, for example, execute step S820 in the above method embodiment.
  • the sliding module 1505 is configured to slide the sliding window along a preset direction for a preset length when the fingerprint feature of the first data in the sliding window does not satisfy the first preset condition, for example, execute step S830 in the above method embodiment.
  • the determination module 1504 is further configured to determine M-1 division points of the target data according to the transformation value of the target data, for example, execute step S910 in the above method embodiment.
  • the processing module 1501 is specifically configured to divide the target data into M candidate data blocks according to the M-1 division points, for example, execute step S920 in the above method embodiment.
  • the determination module 1504 is further configured to determine the end position of the second fixed-length window as the dividing point when the transformation value of the data included in the second fixed-length window satisfies the second preset condition, for example, execute the above-mentioned Step S1120 in the method embodiment.
  • the above-mentioned processing module 1501 is also used to increase the length of the variable-length window in the preset window when the conversion value of the data in the second fixed-length window does not meet the second preset condition, and within the second fixed-length window When the transformation value of the data of the above-mentioned data satisfies the second preset condition, the end position of the second fixed-length window is determined as the division point, for example, step S1130 in the above method embodiment is executed.
  • the determination module 1504 is configured to determine the end positions of the candidate data blocks whose fingerprint features meet the third preset condition among the M candidate data blocks as the N-1 division points of the candidate data blocks, for example, execute the above method to implement Step S1210 in the example.
  • the processing module 1501 divides the M candidate data blocks into N target data blocks according to the N-1 division points, for example, executes step S1220 in the above method embodiment.
  • the above processing module 1501 is further configured to divide the N target data blocks into at least one data group according to their respective fingerprint features, for example, perform step S1310 in the above method embodiment.
  • the generating module 1503 is further configured to generate an index tree of target data based on the respective fingerprint features of multiple data groups, for example, execute step S1320 in the above method embodiment.
  • the determination module 1504 is configured to determine the target data block that satisfies the fourth preset condition among the fingerprint features of each of the N target data blocks as at least one division point of the N target data blocks, for example, execute the above method to implement Step S1410 in the example.
  • the processing module 1501 is configured to divide the N target data blocks into multiple data groups according to at least one division point of the target data blocks, for example, execute step S1420 in the above method embodiment.
  • Each module of the data management device in the above-mentioned storage system can also be used to perform other actions in the above-mentioned method embodiment. All relevant content of each step involved in the above-mentioned method embodiment can be referred to the function description of the corresponding functional module, which is not described here. Let me repeat.
  • FIG. 16 a schematic structural diagram of a data management device in a storage system provided by an embodiment of the present application is shown in FIG. 16 .
  • the electronic device includes: a processing module 1601 and a communication module 1602 .
  • the processing module 1601 is used to control and manage the actions of the data management device in the storage system, for example, to execute the steps performed by the processing module 1501, the generation module 1503, the determination module 1504, and the sliding module 1505, and/or to execute the steps described herein. other processes of the technology.
  • the communication module 1602 is used to support the interaction between the data management device and other devices in the storage system.
  • the data management device in the storage system may also include a storage module 1603, which is used to store the program code of the data management device in the storage system and to relationship etc.
  • the processing module 1601 may be a processor or a controller, for example, the controller 401 or the processor 402 in FIG. 4 .
  • the communication module 1602 may be a transceiver, an RF circuit, or a communication interface, etc., such as the bus 406 and/or the channel controller 403 in FIG. 4 .
  • the storage module 1603 may be a memory, such as the hard disk 405 in FIG. 4 .
  • all or part of them may be implemented by software, hardware, firmware or any combination thereof.
  • a software program When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computer, all or part of the processes or functions according to the embodiments of the present application will be generated.
  • the computer can be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device including a server, a data center, and the like integrated with one or more available media.
  • the available medium may be a magnetic medium (for example, a floppy disk, a magnetic disk, a magnetic tape), an optical medium (for example, a digital video disc (digital video disc, DVD)), or a semiconductor medium (for example, a solid state drive (solid state drives, SSD)), etc. .
  • the disclosed system, device and method can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage medium includes: flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk, and other various media capable of storing program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Des modes de réalisation de la présente demande concernent un procédé et un appareil de gestion de données dans un système de stockage et se rapportent au domaine du stockage de données. Le procédé et l'appareil peuvent économiser des ressources pour la gestion de données. Le procédé consiste : sur la base d'un contenu de données cibles, à diviser les données cibles en M blocs de données candidats, M étant un nombre entier supérieur ou égal à 2 ; puis, selon des caractéristiques d'empreintes digitales respectives des M blocs de données candidats, à diviser les M blocs de données candidats en N blocs de données cibles, N étant un nombre entier positif inférieur ou égal à M, chaque bloc de données cibles comprenant au moins un bloc de données candidat ; à stocker les N blocs de données cibles et les caractéristiques d'empreintes digitales des N blocs de données cibles, les blocs de données cibles et les caractéristiques d'empreintes digitales des blocs de données cibles étant dans une correspondance biunivoque ; et enfin, à générer un arbre d'index des données cibles selon les N blocs de données cibles, l'arbre d'index étant utilisé pour adresser le contenu des données cibles.
PCT/CN2021/137522 2021-12-13 2021-12-13 Procédé et appareil de gestion de données dans un système de stockage WO2023108360A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2021/137522 WO2023108360A1 (fr) 2021-12-13 2021-12-13 Procédé et appareil de gestion de données dans un système de stockage
CN202180102494.2A CN118043799A (zh) 2021-12-13 2021-12-13 一种存储***中数据管理方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/137522 WO2023108360A1 (fr) 2021-12-13 2021-12-13 Procédé et appareil de gestion de données dans un système de stockage

Publications (1)

Publication Number Publication Date
WO2023108360A1 true WO2023108360A1 (fr) 2023-06-22

Family

ID=86775264

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/137522 WO2023108360A1 (fr) 2021-12-13 2021-12-13 Procédé et appareil de gestion de données dans un système de stockage

Country Status (2)

Country Link
CN (1) CN118043799A (fr)
WO (1) WO2023108360A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011097887A1 (fr) * 2010-02-10 2011-08-18 北京播思软件技术有限公司 Procédé de subdivision d'un fichier sur base du contenu
CN111309523A (zh) * 2020-02-16 2020-06-19 西安奥卡云数据科技有限公司 数据读写方法、数据远程复制方法及装置、分布式存储***
CN113126879A (zh) * 2019-12-30 2021-07-16 ***通信集团四川有限公司 一种数据存储方法、装置和电子设备
CN113495901A (zh) * 2021-04-20 2021-10-12 河海大学 一种面向可变长数据块的快速检索方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011097887A1 (fr) * 2010-02-10 2011-08-18 北京播思软件技术有限公司 Procédé de subdivision d'un fichier sur base du contenu
CN113126879A (zh) * 2019-12-30 2021-07-16 ***通信集团四川有限公司 一种数据存储方法、装置和电子设备
CN111309523A (zh) * 2020-02-16 2020-06-19 西安奥卡云数据科技有限公司 数据读写方法、数据远程复制方法及装置、分布式存储***
CN113495901A (zh) * 2021-04-20 2021-10-12 河海大学 一种面向可变长数据块的快速检索方法

Also Published As

Publication number Publication date
CN118043799A (zh) 2024-05-14

Similar Documents

Publication Publication Date Title
US10628449B2 (en) Method and apparatus for processing database data in distributed database system
US20200301850A1 (en) Data processing method and nvme storage device
US9189506B2 (en) Database index management
KR20200027413A (ko) 데이터 저장 방법, 장치 및 시스템
US9378155B2 (en) Method for processing and verifying remote dynamic data, system using the same, and computer-readable medium
US11681679B2 (en) Systems and methods for performing tree-structured dataset operations
US20150293958A1 (en) Scalable data structures
CN105989015B (zh) 一种数据库扩容方法和装置以及访问数据库的方法和装置
US10984050B2 (en) Method, apparatus, and computer program product for managing storage system
WO2021129354A1 (fr) Procédé et dispositif de gestion d'indice de données dans un système de stockage
CN112035413B (zh) 元数据信息查询方法、装置及存储介质
WO2023108360A1 (fr) Procédé et appareil de gestion de données dans un système de stockage
WO2012114402A1 (fr) Dispositif de gestion de base de données et procédé de gestion de base de données
US11287993B2 (en) Method, device, and computer program product for storage management
US9146694B2 (en) Distribution processing unit of shared storage
WO2018210178A1 (fr) Procédé de mémorisation de fichier et dispositif de mémorisation
CN113934361B (zh) 用于管理存储***的方法、设备和计算机程序产品
CN111290700A (zh) 分布式数据读写方法和***
CN115221360A (zh) 树形结构配置方法和***
CN110895520B (zh) 一种文件迁移方法、相关装置及设备
CN112307266B (zh) 一种索引模型的构建方法及装置
CN111597146B (zh) 视频文件处理方法及装置、存储介质、电子装置
US20240211143A1 (en) Method, device and computer program product for transferring data
WO2023116428A1 (fr) Procédé et appareil de comparaison de la migration de données entre des bases de données, et dispositif et support de stockage
Nunome et al. Enhancing the Performance of an Autonomous Distributed Storage System in a Large-Scale Network

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21967502

Country of ref document: EP

Kind code of ref document: A1