JP2015519674A - ストレージクラスタにおける消失符号付加および複製 - Google Patents
ストレージクラスタにおける消失符号付加および複製 Download PDFInfo
- Publication number
- JP2015519674A JP2015519674A JP2015517293A JP2015517293A JP2015519674A JP 2015519674 A JP2015519674 A JP 2015519674A JP 2015517293 A JP2015517293 A JP 2015517293A JP 2015517293 A JP2015517293 A JP 2015517293A JP 2015519674 A JP2015519674 A JP 2015519674A
- Authority
- JP
- Japan
- Prior art keywords
- cluster
- segment
- storage cluster
- digital object
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000010076 replication Effects 0.000 title claims abstract description 32
- 238000000034 method Methods 0.000 claims abstract description 87
- 230000002085 persistent effect Effects 0.000 claims abstract description 5
- 238000006243 chemical reaction Methods 0.000 claims description 13
- 230000009466 transformation Effects 0.000 claims description 5
- 230000003362 replicative effect Effects 0.000 claims description 4
- 230000000717 retained effect Effects 0.000 claims description 4
- 230000001172 regenerating effect Effects 0.000 claims 1
- 238000011426 transformation method Methods 0.000 claims 1
- 238000007792 addition Methods 0.000 description 86
- 238000012545 processing Methods 0.000 description 19
- 230000036541 health Effects 0.000 description 18
- 230000008569 process Effects 0.000 description 9
- 230000004044 response Effects 0.000 description 8
- 230000008901 benefit Effects 0.000 description 7
- 238000007726 management method Methods 0.000 description 6
- 238000012546 transfer Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000011084 recovery Methods 0.000 description 4
- 239000007787 solid Substances 0.000 description 4
- 238000011144 upstream manufacturing Methods 0.000 description 3
- 235000004443 Ricinus communis Nutrition 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 230000000153 supplemental effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0605—Improving or facilitating administration, e.g. storage management by facilitating the interaction with a user or administrator
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0632—Configuration or reconfiguration of storage systems by initialisation or re-initialisation of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/065—Replication mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0662—Virtualisation aspects
- G06F3/0667—Virtualisation aspects at data level, e.g. file, record or object virtualisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
- H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
- H03M13/05—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
- H03M13/25—Error detection or forward error correction by signal space coding, i.e. adding redundancy in the signal constellation, e.g. Trellis Coded Modulation [TCM]
- H03M13/251—Error detection or forward error correction by signal space coding, i.e. adding redundancy in the signal constellation, e.g. Trellis Coded Modulation [TCM] with block coding
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
- H03M13/27—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes using interleaving techniques
- H03M13/2703—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes using interleaving techniques the interleaver involving at least two directions
- H03M13/2707—Simple row-column interleaver, i.e. pure block interleaving
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
- H03M13/35—Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics
- H03M13/353—Adaptation to the channel
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
- H03M13/35—Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics
- H03M13/356—Unequal error protection [UEP]
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
- H03M13/37—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
- H03M13/373—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35 with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
- H03M13/37—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
- H03M13/3761—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
- H03M13/25—Error detection or forward error correction by signal space coding, i.e. adding redundancy in the signal constellation, e.g. Trellis Coded Modulation [TCM]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Probability & Statistics with Applications (AREA)
- Quality & Reliability (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
1つの実施形態では、ある方法でデジタルオブジェクトをストレージクラスタに格納する。まず、ストレージクラスタは、ストレージクラスタのコンピュータノードで、デジタルオブジェクトを格納する要求をクライアントアプリケーションから受け取る。するとストレージクラスタは、デジタルオブジェクトをストレージクラスタに複製を用いて格納するか消失符号付加を用いて格納するかを判断する。この判断は、クライアントからの命令、オブジェクトの固有の特性、オブジェクトのメタデータ、クラスタの設定を参照して行われてもよいし、他の手段によって行われてもよい。消失符号付加を用いてデジタルオブジェクトを格納すると判断された場合、ストレージクラスタは、消失符号付加を用いてストレージクラスタのいくつかのコンピュータノードにデジタルオブジェクトを書き込み、デジタルオブジェクトは、複数のセグメントとして格納される。さらに、各々のセグメントに対して消失符号付加を示すものと一意識別子とをストレージクラスタ内に含んでいるマニフェストコンピュータファイルが作成される。次にストレージクラスタは、クラスタの1つ以上のノードにマニフェストコンピュータファイルを格納し、マニフェストコンピュータファイルの一意識別子をクライアントアプリケーションに返す。マニフェストは、他のデジタルオブジェクトとは区別できるものである。
1つの実施形態では、本発明により、複製または消失符号付加のいずれかを用いてオブジェクトを格納できる。クラスタは、クライアントアプリケーション、オブジェクトのカテゴリ、オブジェクトのサイズ、オブジェクトのメタデータなどからの命令に応じて、オブジェクト単位で切り替わることができる。例えば、比較的大きいオブジェクトはすべて消失符号付加を用いて格納でき、比較的小さいオブジェクトはすべて複製を用いて格納できる。通常の複製は、K=1で、Mが所与のオブジェクトの合計レプリカ数に等しいという消失符号付加の特別なケースと見なしてよい。また、異なるオブジェクトには異なる消失符号付加を割り当ててよい。極めて大きいオブジェクトの場合、このような消失セットを複数用いてオブジェクトを表現してよい。通常の複製を含む様々な符号化を様々なオブジェクトに割り当てるようにすることによって、本発明は、様々な処理費用およびストレージフットプリントで、データ損失からの保護を様々なレベルで可能にする。
・ストレージクラスタの例
・消失セットの例
・マニフェストの例
・クラスタへのデジタルオブジェクトの書き込み
・クラスタからのデジタルオブジェクトの読み出し
・ボリューム障害からの回復
・消失符号付加されたセグメントの再配置
・消失符号付加したオブジェクトを複製に変換、およびその逆
・クラスタ全体のデジタルオブジェクトの管理
・コンピュータシステムの実施形態
Claims (28)
- ストレージクラスタにデジタルオブジェクトを格納する方法であって、
前記ストレージクラスタのコンピュータノードでクライアントアプリケーションから、前記デジタルオブジェクトを格納するための要求を受け取り、
前記デジタルオブジェクトを前記ストレージクラスタに、複製を用いて格納するか消失符号付加を用いて格納するかを判断し、
消失符号付加を用いて前記デジタルオブジェクトを格納すると判断された場合、消失符号付加を用いて前記デジタルオブジェクトを前記ストレージクラスタの複数のコンピュータノードに書き込み、前記デジタルオブジェクトは複数のセグメントとして格納され、
消失符号付加の表示と前記ストレージクラスタ内における各前記セグメントの一意識別子とを含むマニフェストコンピュータファイルを作成し、
前記ストレージクラスタのコンピュータノードに前記マニフェストコンピュータファイルを格納し、
前記マニフェストコンピュータファイルの一意識別子を前記クライアントアプリケーションに返すこと
を備える、方法。 - 前記デジタルオブジェクトの固有の特性、前記クライアントアプリケーションからの命令、または前記デジタルオブジェクトのメタデータを参照して、複製または消失符号付加を用いて前記デジタルオブジェクトを格納するか否かを判断すること
をさらに備える、請求項1に記載の方法。 - 前記ストレージクラスタ内に前記マニフェストコンピュータファイルを複製し、消失符号付加を用いて前記マニフェストコンピュータファイルを格納しないこと
をさらに備える、請求項1に記載の方法。 - 前記デジタルオブジェクトを前記ストレージクラスタ内に複製しないこと
をさらに備える、請求項1に記載の方法。 - 前記ストレージクラスタのディスクに格納されている各セグメントについて、前記セグメントに関連付けられている前記ディスクに、前記デジタルオブジェクトの別のセグメントを格納する次のディスクの一意識別子を格納すること
をさらに備える、請求項1に記載の方法。 - 前記ディスクの前記セグメントについて前記一意識別子をジャーナルエントリに格納することによって、前記セグメントに関連付けられている前記次のディスクの前記一意識別子を格納すること
をさらに備える、請求項5に記載の方法。 - 複数のコンピュータノードを有するストレージクラスタからデジタルオブジェクトを読み出す方法であって、
前記ストレージクラスタ内にある前記コンピュータノードのうちの1つにおいて、前記デジタルオブジェクトについての一意識別子を含むクライアントアプリケーションから要求を受け取り、
複製または消失符号付加を用いて前記ストレージクラスタ内に前記デジタルオブジェクトを格納するか否かを判断し、
消失符号付加を用いて前記デジタルオブジェクトを格納すると判断された場合に、前記コンピュータノードのうちの1つに格納されているマニフェストを読み出し、前記マニフェストは前記一意識別子によって識別され、
前記マニフェスト内で発見された一意的なセグメント識別子を用いて前記ストレージクラスタ内の複数のセグメントを識別し、
前記セグメントおよび消失符号付加アルゴリズムを用いて前記デジタルオブジェクトを再構築し、
前記デジタルオブジェクトを前記クライアントアプリケーションに返すこと
とを備える、方法。 - 前記マニフェストを参照することにより、消失符号付加を用いて前記デジタルオブジェクトを格納することを判断すること
をさらに備える、請求項7に記載の方法。 - 前記セグメントのうちの1つが前記ストレージクラスタ内に存在しないことを判断し、
他の前記セグメントおよび消失符号付加アルゴリズムを用いて、存在しない前記セグメントを再生すること
とをさらに備える、請求項7に記載の方法。 - 前記セグメントのうちの1つが格納されている第1のディスクを識別し、
もう1つの前記セグメントが格納されている第2のディスクについてのディスクの識別子を読み出し、前記ディスクの識別子は、前記第1のディスクにある前記セグメントのうちの前記1つと関連付けられて格納されること
とをさらに備える、請求項7に記載の方法。 - 前記マニフェスト内で第2の消失セットを識別し、前記第2の消失セットは複数の第2の一意的なセグメント識別子を含み、
前記セグメントを用いて前記デジタルオブジェクトを再構築し、複数の第2のセグメントは、前記第2の一意的なセグメント識別子、および前記消失符号付加アルゴリズムによって識別されること
とをさらに備える、請求項7に記載の方法。 - 前記マニフェストは、前記ストレージクラスタ内で複製され、前記マニフェストは、消失符号付加を用いて前記ストレージクラスタ内には格納されない、請求項7に記載の方法。
- 前記デジタルオブジェクトは、前記ストレージクラスタ内で複製されない、請求項7に記載の方法。
- 障害のあるディスクから回復する方法であって、
複数のコンピュータノードを有するストレージクラスタ内で、前記ノードのうちの1つの第1のディスクに障害があることを検知し、
前記ストレージクラスタの第2のディスクの永続的なストレージ領域をスキャンして前記障害のあるディスクの一意識別子を発見し、前記一意識別子は前記ストレージクラスタのデジタルストリーム関連付けられており、
複製または消失符号付加を用いて、前記デジタルストリームを前記ストレージクラスタ内に格納するか否かを判断し、
消失符号付加を用いて前記デジタルストリームを格納すると判断された場合、前記障害のあるディスクに以前格納された喪失セグメントを識別し、
前記ストレージクラスタ内において複数の他のセグメントを見つけ出し、前記複数のセグメントは前記デジタルストリームを含み、
前記複数の他のセグメントおよび消失符号付加アルゴリズムを用いて、前記障害のあるディスクに以前格納された前記喪失セグメントを再生し、
前記再生されたセグメントを前記ストレージクラスタのコンピュータノードに格納すること
を備える、方法。 - 前記デジタルストリームのメタデータセクションをスキャンして、前記複数の他のセグメントに対する一意識別子を発見すること
をさらに備える、請求項14に記載の方法。 - 前記第2のディスクの前記永続的なストレージ領域をスキャンすることは、前記デジタルストリームに対するジャーナルエントリをスキャンすることを含むこと
をさらに備える、請求項14に記載の方法。 - 前記ジャーナルエントリを参照することにより、消失符号付加を用いて前記デジタルストリームを格納するか否かを判断すること
をさらに備える、請求項16に記載の方法。 - 前記喪失セグメントを含む消失符号付加を用いて、ストレージクラスタの外部から前記ストレージクラスタ内に格納されているデジタルオブジェクトについての要求を受け取る前に、前記障害のあるディスクに以前格納された前記喪失セグメントを識別すること
をさら備える、請求項14に記載の方法。 - 前記喪失セグメントは、前記ストレージクラスタ内では複製されない、請求項14に記載の方法。
- セグメントをストレージクラスタ内で再配置する方法であって、
コンピュータノードの第1のディスク上にあるセグメントをストレージクラスタ内で識別し、前記セグメントは、前記ストレージクラスタ内で格納されているデジタルオブジェクトを表す複数のセグメントのうちの1つであり、
前記ストレージクラスタの前記第1のディスクから第2のディスクへ前記セグメントを再配置し、前記第2のディスクは一意的なディスク識別子によって識別され、
前記セグメントのメタデータから前記複数のセグメントのシブリングセグメントについての一意識別子を回収し、前記シブリングセグメントは、そのメタデータ内に前記第1のディスクに対する一意的なディスク識別子を含み、
前記一意識別子を用いて前記シブリングセグメントを前記ストレージクラスタ内で見つけ出し、
前記シブリングセグメントの前記メタデータ内で、前記第1のディスクについての前記一意的なディスク識別子を前記第2のディスクについての一意的なディスク識別子に入れ替え、前記シブリングセグメントの前記メタデータは、前記セグメントが再配置された先のディスクを示すこと
とをさらに備える、方法。 - ストレージクラスタ内でデジタルオブジェクトを変換する方法であって、
前記ストレージクラスタ内の前記デジタルオブジェクトを、ビットの連続ストリームとしてコンピュータノードの単一のディスク上に格納し、前記デジタルオブジェクトは、前記ストレージクラスタ内で一意識別子を有し、
前記格納の後に、前記デジタルオブジェクトを消失符号付加ストレージ方式に変換する要求を示す前記ストレージクラスタのメタデータを識別し、
前記ストレージクラスタのコンピュータノードを用いて前記単一のディスクから前記デジタルオブジェクトを読み出し、
前記消失符号付加ストレージ方式を用いて、前記デジタルオブジェクトを前記ストレージクラスタの複数のディスクに書き込み、
前記消失符号付加ストレージ方式で書き込まれた前記デジタルオブジェクトについての前記一意識別子を保持し、これにより、クライアントアプリケーションは、前記一意識別子を用いて前記消失符号付加ストレージ方式で書き込まれた前記デジタルオブジェクトを検索可能であること
を備える、方法。 - ストレージクラスタ内でデジタルオブジェクトを変換する方法であって、
前記ストレージクラスタ内の前記デジタルオブジェクトを第1の消失符号付加ストレージ方式で格納し、前記デジタルオブジェクトは、前記ストレージクラスタ内で一意識別子を有し
前記格納の後に、前記デジタルオブジェクトを第2の消失符号付加ストレージ方式に変換する要求を示す前記ストレージクラスタのメタデータを識別し、
前記ストレージクラスタのコンピュータノードを用いて前記ストレージクラスタから前記デジタルオブジェクトを読み出し、
前記第2の消失符号付加ストレージ方式を用いて、前記デジタルオブジェクトを前記ストレージクラスタに書き込み、
前記第2の消失符号付加ストレージ方式で書き込まれた前記デジタルオブジェクトについての前記一意識別子を保持し、クライアントアプリケーションは、前記一意識別子を用いて前記第2の消失符号付加ストレージ方式で書き込まれた前記デジタルオブジェクトを検索可能であること
を含備える、方法。 - ストレージクラスタ内でデジタルオブジェクトを変換する方法であって、
前記ストレージクラスタ内の前記デジタルオブジェクトを消失符号付加ストレージ方式で格納し、前記デジタルオブジェクトが、前記ストレージクラスタ内で一意識別子を有し、
前記格納の後に、複製を用いて前記デジタルオブジェクトをストレージ方式に変換する要求を示す前記ストレージクラスタのメタデータを識別し、
前記ストレージクラスタのコンピュータノードを用いて前記ストレージクラスタから前記デジタルオブジェクトを読み出し、
前記ストレージクラスタのコンピュータノードの単一のディスクに前記デジタルオブジェクトをビットの連続ストリームとして書き込み、
前記ビットの連続ストリームとして書き込まれた前記デジタルオブジェクトについての前記一意識別子を保持し、クライアントアプリケーションは、前記一意識別子を用いて前記ビットの連続ストリームとして書き込まれた前記デジタルオブジェクトを検索可能であること
とを備える、方法。 - ソースストレージクラスタからターゲットストレージクラスタにデジタルオブジェクトをコピーする方法であって、
前記ターゲットクラスタのターゲットノードにいて、前記ソースクラスタから前記ターゲットクラスタに前記デジタルオブジェクトをコピーするための命令を受け取り、前記命令は、前記ソースクラスタのソースノードのアドレスを含み、
前記デジタルオブジェクトを前記ターゲットクラスタに格納する先のターゲット変換方式を決定し、
前記ソースクラスタから前記デジタルオブジェクトを読み出し、前記デジタルオブジェクトは、ソース変換方式で格納され、前記ソースクラスタ内に一意識別子を有し、
前記ターゲット変換方式を用いて前記デジタルオブジェクトを前記ターゲットクラスタに格納し、前記一意識別子を用いて前記デジタルオブジェクトは前記一意識別子を用いて格納されること
を備える、方法。 - 前記ターゲット変換方式は、前記ソース変換方式とは異なる、請求項24に記載の方法。
- 前記ソース変換方式は、複製または消失符号付加である、請求項24に記載の方法。
- 前記デジタルオブジェクトのメタデータを参照するか前記ターゲットクラスタの設定を参照して、前記ターゲット変換方式を決定することを更に備える、請求項24に記載の方法。
- 前記ターゲットクラスタは、前記ソースクラスタとは異なるストレージ方式を実装する、請求項24に記載の方法。
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/517,527 | 2012-06-13 | ||
US13/517,527 US8799746B2 (en) | 2012-06-13 | 2012-06-13 | Erasure coding and replication in storage clusters |
PCT/US2013/044045 WO2013188169A1 (en) | 2012-06-13 | 2013-06-04 | Erasure coding and replication in storage clusters |
Publications (3)
Publication Number | Publication Date |
---|---|
JP2015519674A true JP2015519674A (ja) | 2015-07-09 |
JP2015519674A5 JP2015519674A5 (ja) | 2016-07-07 |
JP6280108B2 JP6280108B2 (ja) | 2018-02-14 |
Family
ID=49757125
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2015517293A Active JP6280108B2 (ja) | 2012-06-13 | 2013-06-04 | ストレージクラスタにおける消失符号付加および複製 |
Country Status (7)
Country | Link |
---|---|
US (4) | US8799746B2 (ja) |
EP (1) | EP2862085A4 (ja) |
JP (1) | JP6280108B2 (ja) |
CN (1) | CN104541251B (ja) |
AU (1) | AU2013274626A1 (ja) |
CA (1) | CA2876638A1 (ja) |
WO (1) | WO2013188169A1 (ja) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2017142605A (ja) * | 2016-02-09 | 2017-08-17 | 株式会社日立製作所 | バックアップリストアシステム及びリストア方法 |
JP2018116704A (ja) * | 2017-01-20 | 2018-07-26 | 三星電子株式会社Samsung Electronics Co.,Ltd. | ストレージシステム及びその動作方法 |
JP2018156656A (ja) * | 2017-03-20 | 2018-10-04 | 三星電子株式会社Samsung Electronics Co.,Ltd. | 仮想装置階層を利用した複数のメモリ装置を含む仮想装置に対する客体の格納及び読み取り方法とこれを用いたストレージ装置 |
KR20190088873A (ko) * | 2018-01-19 | 2019-07-29 | 삼성전자주식회사 | 대형 키 밸류 객체들을 저장하는 시스템 및 방법 |
JP2019128959A (ja) * | 2018-01-19 | 2019-08-01 | 三星電子株式会社Samsung Electronics Co.,Ltd. | キーバリューデータ信頼性システム、そのデータ格納方法、及び、その方法を具現するコンピュータコードを含む非一時的なコンピュータ読出し可能な媒体 |
JP2020518934A (ja) * | 2017-06-08 | 2020-06-25 | ヒタチ ヴァンタラ コーポレーションHitachi Vantara Corporation | 地理的に分散したオブジェクトデータの高速呼び戻し |
JP2022512064A (ja) * | 2018-10-15 | 2022-02-02 | ネットアップ,インコーポレイテッド | 様々なデータ冗長性スキームを備えた、システムにおける利用可能なストレージ空間を改善すること |
US11275762B2 (en) | 2017-03-20 | 2022-03-15 | Samsung Electronics Co., Ltd. | System and method for hybrid data reliability for object storage devices |
JP2022169072A (ja) * | 2021-04-27 | 2022-11-09 | 株式会社日立製作所 | ストレージシステム、ストレージノード仮想計算機復旧方法、及び復旧プログラム |
Families Citing this family (280)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11614893B2 (en) | 2010-09-15 | 2023-03-28 | Pure Storage, Inc. | Optimizing storage device access based on latency |
US12008266B2 (en) | 2010-09-15 | 2024-06-11 | Pure Storage, Inc. | Efficient read by reconstruction |
US8510267B2 (en) * | 2011-03-08 | 2013-08-13 | Rackspace Us, Inc. | Synchronization of structured information repositories |
US8589640B2 (en) | 2011-10-14 | 2013-11-19 | Pure Storage, Inc. | Method for maintaining multiple fingerprint tables in a deduplicating storage system |
US9146969B2 (en) * | 2012-11-26 | 2015-09-29 | The Boeing Company | System and method of reduction of irrelevant information during search |
US9558206B2 (en) * | 2013-02-25 | 2017-01-31 | Datadirect Networks, Inc. | Asymmetric distributed data storage system |
US9354991B2 (en) * | 2013-06-25 | 2016-05-31 | Microsoft Technology Licensing, Llc | Locally generated simple erasure codes |
US9313021B2 (en) * | 2013-08-29 | 2016-04-12 | National Chiao Tung University | Secret communication method with self-authentication capability |
KR102238590B1 (ko) * | 2013-11-29 | 2021-04-09 | 삼성전자주식회사 | 데이터 패킷 전송의 인증 및 검증 방법, 및 상기 방법을 수행하는 장치들 |
US10685037B2 (en) * | 2013-12-18 | 2020-06-16 | Amazon Technology, Inc. | Volume cohorts in object-redundant storage systems |
US10620830B2 (en) * | 2013-12-18 | 2020-04-14 | Amazon Technologies, Inc. | Reconciling volumelets in volume cohorts |
US8874835B1 (en) | 2014-01-16 | 2014-10-28 | Pure Storage, Inc. | Data placement based on data properties in a tiered storage device system |
US9344120B2 (en) * | 2014-02-13 | 2016-05-17 | Quantum Corporation | Adjusting redundancy in an erasure code object store to account for varying data value |
US9003144B1 (en) | 2014-06-04 | 2015-04-07 | Pure Storage, Inc. | Mechanism for persisting messages in a storage system |
US9612952B2 (en) | 2014-06-04 | 2017-04-04 | Pure Storage, Inc. | Automatically reconfiguring a storage memory topology |
US11068363B1 (en) | 2014-06-04 | 2021-07-20 | Pure Storage, Inc. | Proactively rebuilding data in a storage cluster |
US11652884B2 (en) | 2014-06-04 | 2023-05-16 | Pure Storage, Inc. | Customized hash algorithms |
US9213485B1 (en) * | 2014-06-04 | 2015-12-15 | Pure Storage, Inc. | Storage system architecture |
EP3152662B1 (en) * | 2014-06-04 | 2019-04-24 | Pure Storage, Inc. | Mechanism for persisting messages in a storage system |
AU2015269370B2 (en) * | 2014-06-04 | 2019-06-06 | Pure Storage, Inc. | Automatically reconfiguring a storage memory topology |
US9367243B1 (en) | 2014-06-04 | 2016-06-14 | Pure Storage, Inc. | Scalable non-uniform storage sizes |
US9218244B1 (en) * | 2014-06-04 | 2015-12-22 | Pure Storage, Inc. | Rebuilding data across storage nodes |
US8850108B1 (en) | 2014-06-04 | 2014-09-30 | Pure Storage, Inc. | Storage cluster |
US10574754B1 (en) | 2014-06-04 | 2020-02-25 | Pure Storage, Inc. | Multi-chassis array with multi-level load balancing |
US11399063B2 (en) | 2014-06-04 | 2022-07-26 | Pure Storage, Inc. | Network authentication for a storage system |
US11960371B2 (en) | 2014-06-04 | 2024-04-16 | Pure Storage, Inc. | Message persistence in a zoned system |
US9836234B2 (en) | 2014-06-04 | 2017-12-05 | Pure Storage, Inc. | Storage cluster |
US11886308B2 (en) | 2014-07-02 | 2024-01-30 | Pure Storage, Inc. | Dual class of service for unified file and object messaging |
US9836245B2 (en) | 2014-07-02 | 2017-12-05 | Pure Storage, Inc. | Non-volatile RAM and flash memory in a non-volatile solid-state storage |
US8868825B1 (en) | 2014-07-02 | 2014-10-21 | Pure Storage, Inc. | Nonrepeating identifiers in an address space of a non-volatile solid-state storage |
US11604598B2 (en) | 2014-07-02 | 2023-03-14 | Pure Storage, Inc. | Storage cluster with zoned drives |
US10114757B2 (en) | 2014-07-02 | 2018-10-30 | Pure Storage, Inc. | Nonrepeating identifiers in an address space of a non-volatile solid-state storage |
US9021297B1 (en) | 2014-07-02 | 2015-04-28 | Pure Storage, Inc. | Redundant, fault-tolerant, distributed remote procedure call cache in a storage system |
US10853311B1 (en) * | 2014-07-03 | 2020-12-01 | Pure Storage, Inc. | Administration through files in a storage system |
US9817750B2 (en) * | 2014-07-03 | 2017-11-14 | Pure Storage, Inc. | Profile-dependent write placement of data into a non-volatile solid-state storage |
US9811677B2 (en) | 2014-07-03 | 2017-11-07 | Pure Storage, Inc. | Secure data replication in a storage grid |
US8874836B1 (en) | 2014-07-03 | 2014-10-28 | Pure Storage, Inc. | Scheduling policy for queues in a non-volatile solid-state storage |
US9747229B1 (en) | 2014-07-03 | 2017-08-29 | Pure Storage, Inc. | Self-describing data format for DMA in a non-volatile solid-state storage |
US10983859B2 (en) | 2014-08-07 | 2021-04-20 | Pure Storage, Inc. | Adjustable error correction based on memory health in a storage unit |
US9495255B2 (en) | 2014-08-07 | 2016-11-15 | Pure Storage, Inc. | Error recovery in a storage cluster |
US9483346B2 (en) | 2014-08-07 | 2016-11-01 | Pure Storage, Inc. | Data rebuild on feedback from a queue in a non-volatile solid-state storage |
US9558069B2 (en) * | 2014-08-07 | 2017-01-31 | Pure Storage, Inc. | Failure mapping in a storage array |
US9082512B1 (en) | 2014-08-07 | 2015-07-14 | Pure Storage, Inc. | Die-level monitoring in a storage cluster |
US9766972B2 (en) | 2014-08-07 | 2017-09-19 | Pure Storage, Inc. | Masking defective bits in a storage array |
US10079711B1 (en) | 2014-08-20 | 2018-09-18 | Pure Storage, Inc. | Virtual file server with preserved MAC address |
US9923970B2 (en) * | 2014-08-22 | 2018-03-20 | Nexenta Systems, Inc. | Multicast collaborative erasure encoding and distributed parity protection |
US20160219120A1 (en) * | 2015-01-23 | 2016-07-28 | Netapp, Inc. | Methods for providing a staging area for objects prior to erasure coding and devices thereof |
US9665428B2 (en) * | 2015-02-05 | 2017-05-30 | Netapp, Inc. | Distributing erasure-coded fragments in a geo-distributed storage system |
US9948615B1 (en) | 2015-03-16 | 2018-04-17 | Pure Storage, Inc. | Increased storage unit encryption based on loss of trust |
US11294893B2 (en) | 2015-03-20 | 2022-04-05 | Pure Storage, Inc. | Aggregation of queries |
US9940234B2 (en) | 2015-03-26 | 2018-04-10 | Pure Storage, Inc. | Aggressive data deduplication using lazy garbage collection |
US10082985B2 (en) | 2015-03-27 | 2018-09-25 | Pure Storage, Inc. | Data striping across storage nodes that are assigned to multiple logical arrays |
US10178169B2 (en) | 2015-04-09 | 2019-01-08 | Pure Storage, Inc. | Point to point based backend communication layer for storage processing |
US9672125B2 (en) * | 2015-04-10 | 2017-06-06 | Pure Storage, Inc. | Ability to partition an array into two or more logical arrays with independently running software |
US10140149B1 (en) | 2015-05-19 | 2018-11-27 | Pure Storage, Inc. | Transactional commits with hardware assists in remote memory |
US9817576B2 (en) | 2015-05-27 | 2017-11-14 | Pure Storage, Inc. | Parallel update to NVRAM |
US10846275B2 (en) | 2015-06-26 | 2020-11-24 | Pure Storage, Inc. | Key management in a storage device |
US9734028B2 (en) * | 2015-06-29 | 2017-08-15 | International Business Machines Corporation | Reverse resynchronization by a secondary data source when a data destination has more recent data |
US10983732B2 (en) | 2015-07-13 | 2021-04-20 | Pure Storage, Inc. | Method and system for accessing a file |
US11232079B2 (en) | 2015-07-16 | 2022-01-25 | Pure Storage, Inc. | Efficient distribution of large directories |
US9898470B2 (en) * | 2015-08-05 | 2018-02-20 | Bank Of America Corporation | Transferring archived data |
US10929431B2 (en) | 2015-08-28 | 2021-02-23 | Hewlett Packard Enterprise Development Lp | Collision handling during an asynchronous replication |
US10108355B2 (en) | 2015-09-01 | 2018-10-23 | Pure Storage, Inc. | Erase block state detection |
US11269884B2 (en) | 2015-09-04 | 2022-03-08 | Pure Storage, Inc. | Dynamically resizable structures for approximate membership queries |
US11341136B2 (en) | 2015-09-04 | 2022-05-24 | Pure Storage, Inc. | Dynamically resizable structures for approximate membership queries |
CN107850983B (zh) * | 2015-09-11 | 2020-11-03 | 株式会社日立制作所 | 计算机***、存储装置和数据的管理方法 |
US9768953B2 (en) | 2015-09-30 | 2017-09-19 | Pure Storage, Inc. | Resharing of a split secret |
US10762069B2 (en) | 2015-09-30 | 2020-09-01 | Pure Storage, Inc. | Mechanism for a system where data and metadata are located closely together |
US10853266B2 (en) | 2015-09-30 | 2020-12-01 | Pure Storage, Inc. | Hardware assisted data lookup methods |
US10360119B2 (en) | 2015-10-06 | 2019-07-23 | Netapp, Inc. | Data recovery in a distributed storage system |
US9843453B2 (en) | 2015-10-23 | 2017-12-12 | Pure Storage, Inc. | Authorizing I/O commands with I/O tokens |
US10007457B2 (en) | 2015-12-22 | 2018-06-26 | Pure Storage, Inc. | Distributed transactions with token-associated execution |
EP3430515B1 (en) | 2016-03-15 | 2021-09-22 | Datomia Research Labs Ou | Distributed storage system data management and security |
US10931402B2 (en) | 2016-03-15 | 2021-02-23 | Cloud Storage, Inc. | Distributed storage system data management and security |
US9916478B2 (en) * | 2016-03-24 | 2018-03-13 | Netapp, Inc. | Data protection enhancement using free space |
US10261690B1 (en) | 2016-05-03 | 2019-04-16 | Pure Storage, Inc. | Systems and methods for operating a storage system |
US10552038B2 (en) * | 2016-05-13 | 2020-02-04 | International Business Machines Corporation | Object storage architecture based on file_heat |
US11231858B2 (en) | 2016-05-19 | 2022-01-25 | Pure Storage, Inc. | Dynamically configuring a storage system to facilitate independent scaling of resources |
AU2017274321B2 (en) * | 2016-06-01 | 2020-01-02 | Archive360, Inc. | Methods and systems for archiving and retrieving data |
US10691567B2 (en) | 2016-06-03 | 2020-06-23 | Pure Storage, Inc. | Dynamically forming a failure domain in a storage system that includes a plurality of blades |
US10547681B2 (en) | 2016-06-30 | 2020-01-28 | Purdue Research Foundation | Functional caching in erasure coded storage |
US11706895B2 (en) | 2016-07-19 | 2023-07-18 | Pure Storage, Inc. | Independent scaling of compute resources and storage resources in a storage system |
US11861188B2 (en) | 2016-07-19 | 2024-01-02 | Pure Storage, Inc. | System having modular accelerators |
US9672905B1 (en) | 2016-07-22 | 2017-06-06 | Pure Storage, Inc. | Optimize data protection layouts based on distributed flash wear leveling |
US10768819B2 (en) | 2016-07-22 | 2020-09-08 | Pure Storage, Inc. | Hardware support for non-disruptive upgrades |
US11449232B1 (en) | 2016-07-22 | 2022-09-20 | Pure Storage, Inc. | Optimal scheduling of flash operations |
US11604690B2 (en) | 2016-07-24 | 2023-03-14 | Pure Storage, Inc. | Online failure span determination |
US10216420B1 (en) | 2016-07-24 | 2019-02-26 | Pure Storage, Inc. | Calibration of flash channels in SSD |
US11080155B2 (en) | 2016-07-24 | 2021-08-03 | Pure Storage, Inc. | Identifying error types among flash memory |
US10366004B2 (en) | 2016-07-26 | 2019-07-30 | Pure Storage, Inc. | Storage system with elective garbage collection to reduce flash contention |
US11886334B2 (en) | 2016-07-26 | 2024-01-30 | Pure Storage, Inc. | Optimizing spool and memory space management |
US11734169B2 (en) | 2016-07-26 | 2023-08-22 | Pure Storage, Inc. | Optimizing spool and memory space management |
US10203903B2 (en) | 2016-07-26 | 2019-02-12 | Pure Storage, Inc. | Geometry based, space aware shelf/writegroup evacuation |
US11797212B2 (en) | 2016-07-26 | 2023-10-24 | Pure Storage, Inc. | Data migration for zoned drives |
US10348807B2 (en) * | 2016-07-28 | 2019-07-09 | Caringo Inc. | Multi-part upload |
AT518910B1 (de) * | 2016-08-04 | 2018-10-15 | Ait Austrian Inst Tech Gmbh | Verfahren zur Prüfung der Verfügbarkeit und Integrität eines verteilt gespeicherten Datenobjekts |
US10191809B2 (en) * | 2016-08-17 | 2019-01-29 | International Business Machines Corporation | Converting a data chunk into a ring algebraic structure for fast erasure coding |
US11422719B2 (en) | 2016-09-15 | 2022-08-23 | Pure Storage, Inc. | Distributed file deletion and truncation |
US9747039B1 (en) | 2016-10-04 | 2017-08-29 | Pure Storage, Inc. | Reservations over multiple paths on NVMe over fabrics |
US12039165B2 (en) | 2016-10-04 | 2024-07-16 | Pure Storage, Inc. | Utilizing allocation shares to improve parallelism in a zoned drive storage system |
US10756816B1 (en) | 2016-10-04 | 2020-08-25 | Pure Storage, Inc. | Optimized fibre channel and non-volatile memory express access |
US10481798B2 (en) | 2016-10-28 | 2019-11-19 | Pure Storage, Inc. | Efficient flash management for multiple controllers |
CN106685426B (zh) * | 2016-11-28 | 2021-02-09 | 北京航天自动控制研究所 | 一种目标信息的编码方法 |
US11269888B1 (en) * | 2016-11-28 | 2022-03-08 | Amazon Technologies, Inc. | Archival data storage for structured data |
US10268538B2 (en) | 2016-11-28 | 2019-04-23 | Alibaba Group Holding Limited | Efficient and enhanced distributed storage clusters |
US10740288B2 (en) * | 2016-12-02 | 2020-08-11 | International Business Machines Corporation | Accessing objects in an erasure code supported object storage environment |
US11550481B2 (en) | 2016-12-19 | 2023-01-10 | Pure Storage, Inc. | Efficiently writing data in a zoned drive storage system |
US11307998B2 (en) | 2017-01-09 | 2022-04-19 | Pure Storage, Inc. | Storage efficiency of encrypted host system data |
US9747158B1 (en) | 2017-01-13 | 2017-08-29 | Pure Storage, Inc. | Intelligent refresh of 3D NAND |
US11955187B2 (en) | 2017-01-13 | 2024-04-09 | Pure Storage, Inc. | Refresh of differing capacity NAND |
US10979223B2 (en) | 2017-01-31 | 2021-04-13 | Pure Storage, Inc. | Separate encryption for a solid-state drive |
CN108628539B (zh) * | 2017-03-17 | 2021-03-26 | 杭州海康威视数字技术股份有限公司 | 数据存储、分散、重构、回收方法、装置及数据处理*** |
US10621055B2 (en) | 2017-03-28 | 2020-04-14 | Amazon Technologies, Inc. | Adaptive data recovery for clustered data devices |
US11356445B2 (en) | 2017-03-28 | 2022-06-07 | Amazon Technologies, Inc. | Data access interface for clustered devices |
US10530752B2 (en) * | 2017-03-28 | 2020-01-07 | Amazon Technologies, Inc. | Efficient device provision |
US10528488B1 (en) | 2017-03-30 | 2020-01-07 | Pure Storage, Inc. | Efficient name coding |
US11016667B1 (en) | 2017-04-05 | 2021-05-25 | Pure Storage, Inc. | Efficient mapping for LUNs in storage memory with holes in address space |
US10944671B2 (en) | 2017-04-27 | 2021-03-09 | Pure Storage, Inc. | Efficient data forwarding in a networked device |
US10141050B1 (en) | 2017-04-27 | 2018-11-27 | Pure Storage, Inc. | Page writes for triple level cell flash memory |
US10516645B1 (en) | 2017-04-27 | 2019-12-24 | Pure Storage, Inc. | Address resolution broadcasting in a networked device |
US10275180B2 (en) * | 2017-05-08 | 2019-04-30 | Samsung Electronics Co., Ltd. | Ethernet SSD system including storage offload engine (SoE) controller and ethernet switch |
US10579284B2 (en) | 2017-06-02 | 2020-03-03 | International Business Machines Corporation | Filesystem enhancements for unified file and object access in an object storage cloud |
US11467913B1 (en) | 2017-06-07 | 2022-10-11 | Pure Storage, Inc. | Snapshots with crash consistency in a storage system |
US11947814B2 (en) | 2017-06-11 | 2024-04-02 | Pure Storage, Inc. | Optimizing resiliency group formation stability |
US11782625B2 (en) | 2017-06-11 | 2023-10-10 | Pure Storage, Inc. | Heterogeneity supportive resiliency groups |
US11138103B1 (en) | 2017-06-11 | 2021-10-05 | Pure Storage, Inc. | Resiliency groups |
JP6807457B2 (ja) * | 2017-06-15 | 2021-01-06 | 株式会社日立製作所 | ストレージシステム及びストレージシステムの制御方法 |
US10310943B2 (en) | 2017-06-16 | 2019-06-04 | Microsoft Technology Licensing, Llc | Distributed data object management system |
US11003532B2 (en) | 2017-06-16 | 2021-05-11 | Microsoft Technology Licensing, Llc | Distributed data object management system operations |
US10425473B1 (en) | 2017-07-03 | 2019-09-24 | Pure Storage, Inc. | Stateful connection reset in a storage cluster with a stateless load balancer |
JP6733620B2 (ja) * | 2017-07-12 | 2020-08-05 | 京セラドキュメントソリューションズ株式会社 | 画像読取装置、及び画像読取方法 |
US10761743B1 (en) | 2017-07-17 | 2020-09-01 | EMC IP Holding Company LLC | Establishing data reliability groups within a geographically distributed data storage environment |
US10402266B1 (en) | 2017-07-31 | 2019-09-03 | Pure Storage, Inc. | Redundant array of independent disks in a direct-mapped flash storage system |
US10831935B2 (en) | 2017-08-31 | 2020-11-10 | Pure Storage, Inc. | Encryption management with host-side data reduction |
US10877827B2 (en) | 2017-09-15 | 2020-12-29 | Pure Storage, Inc. | Read voltage optimization |
US10210926B1 (en) | 2017-09-15 | 2019-02-19 | Pure Storage, Inc. | Tracking of optimum read voltage thresholds in nand flash devices |
US10880040B1 (en) | 2017-10-23 | 2020-12-29 | EMC IP Holding Company LLC | Scale-out distributed erasure coding |
US10515701B1 (en) | 2017-10-31 | 2019-12-24 | Pure Storage, Inc. | Overlapping raid groups |
US10496330B1 (en) | 2017-10-31 | 2019-12-03 | Pure Storage, Inc. | Using flash storage devices with different sized erase blocks |
US10545687B1 (en) | 2017-10-31 | 2020-01-28 | Pure Storage, Inc. | Data rebuild when changing erase block sizes during drive replacement |
US10884919B2 (en) | 2017-10-31 | 2021-01-05 | Pure Storage, Inc. | Memory management in a storage system |
US11024390B1 (en) | 2017-10-31 | 2021-06-01 | Pure Storage, Inc. | Overlapping RAID groups |
US10700711B1 (en) * | 2017-11-03 | 2020-06-30 | Caringo Inc. | Multi-part upload and editing of erasure-coded objects |
US10860475B1 (en) | 2017-11-17 | 2020-12-08 | Pure Storage, Inc. | Hybrid flash translation layer |
US10990566B1 (en) | 2017-11-20 | 2021-04-27 | Pure Storage, Inc. | Persistent file locks in a storage system |
US10929053B2 (en) | 2017-12-08 | 2021-02-23 | Pure Storage, Inc. | Safe destructive actions on drives |
US10719265B1 (en) | 2017-12-08 | 2020-07-21 | Pure Storage, Inc. | Centralized, quorum-aware handling of device reservation requests in a storage system |
US10929031B2 (en) | 2017-12-21 | 2021-02-23 | Pure Storage, Inc. | Maximizing data reduction in a partially encrypted volume |
US10382554B1 (en) | 2018-01-04 | 2019-08-13 | Emc Corporation | Handling deletes with distributed erasure coding |
US11861025B1 (en) | 2018-01-08 | 2024-01-02 | Rankin Labs, Llc | System and method for receiving and processing a signal within a TCP/IP protocol stack |
CN110018783B (zh) | 2018-01-09 | 2022-12-20 | 阿里巴巴集团控股有限公司 | 一种数据存储方法、装置及*** |
US10467527B1 (en) | 2018-01-31 | 2019-11-05 | Pure Storage, Inc. | Method and apparatus for artificial intelligence acceleration |
US10976948B1 (en) | 2018-01-31 | 2021-04-13 | Pure Storage, Inc. | Cluster expansion mechanism |
US10733053B1 (en) | 2018-01-31 | 2020-08-04 | Pure Storage, Inc. | Disaster recovery for high-bandwidth distributed archives |
US11036596B1 (en) | 2018-02-18 | 2021-06-15 | Pure Storage, Inc. | System for delaying acknowledgements on open NAND locations until durability has been confirmed |
US11494109B1 (en) | 2018-02-22 | 2022-11-08 | Pure Storage, Inc. | Erase block trimming for heterogenous flash memory storage devices |
US10901845B2 (en) * | 2018-04-16 | 2021-01-26 | Microsoft Technology Licensing, Llc | Erasure coding for a single-image memory |
US11995336B2 (en) | 2018-04-25 | 2024-05-28 | Pure Storage, Inc. | Bucket views |
US12001688B2 (en) | 2019-04-29 | 2024-06-04 | Pure Storage, Inc. | Utilizing data views to optimize secure data access in a storage system |
US10931450B1 (en) | 2018-04-27 | 2021-02-23 | Pure Storage, Inc. | Distributed, lock-free 2-phase commit of secret shares using multiple stateless controllers |
US10853146B1 (en) | 2018-04-27 | 2020-12-01 | Pure Storage, Inc. | Efficient data forwarding in a networked device |
US11385792B2 (en) | 2018-04-27 | 2022-07-12 | Pure Storage, Inc. | High availability controller pair transitioning |
US10579297B2 (en) | 2018-04-27 | 2020-03-03 | EMC IP Holding Company LLC | Scaling-in for geographically diverse storage |
US11436023B2 (en) | 2018-05-31 | 2022-09-06 | Pure Storage, Inc. | Mechanism for updating host file system and flash translation layer based on underlying NAND technology |
US10936196B2 (en) | 2018-06-15 | 2021-03-02 | EMC IP Holding Company LLC | Data convolution for geographically diverse storage |
US11023130B2 (en) | 2018-06-15 | 2021-06-01 | EMC IP Holding Company LLC | Deleting data in a geographically diverse storage construct |
US10572173B2 (en) * | 2018-07-10 | 2020-02-25 | EMC IP Holding Company LLC | Data management in an elastic cloud storage environment |
US11438279B2 (en) | 2018-07-23 | 2022-09-06 | Pure Storage, Inc. | Non-disruptive conversion of a clustered service from single-chassis to multi-chassis |
US11689543B2 (en) | 2018-08-10 | 2023-06-27 | Rankin Labs, Llc | System and method for detecting transmission of a covert payload of data |
WO2020041390A1 (en) | 2018-08-21 | 2020-02-27 | John Rankin | System and method for scattering network traffic across a number of disparate hosts |
US11354058B2 (en) | 2018-09-06 | 2022-06-07 | Pure Storage, Inc. | Local relocation of data stored at a storage device of a storage system |
US11520514B2 (en) | 2018-09-06 | 2022-12-06 | Pure Storage, Inc. | Optimized relocation of data based on data characteristics |
US11868309B2 (en) | 2018-09-06 | 2024-01-09 | Pure Storage, Inc. | Queue management for data relocation |
US11500570B2 (en) | 2018-09-06 | 2022-11-15 | Pure Storage, Inc. | Efficient relocation of data utilizing different programming modes |
US20200117362A1 (en) * | 2018-10-15 | 2020-04-16 | Netapp, Inc. | Erasure coding content driven distribution of data blocks |
US10454498B1 (en) | 2018-10-18 | 2019-10-22 | Pure Storage, Inc. | Fully pipelined hardware engine design for fast and efficient inline lossless data compression |
US10976947B2 (en) | 2018-10-26 | 2021-04-13 | Pure Storage, Inc. | Dynamically selecting segment heights in a heterogeneous RAID group |
US11436203B2 (en) | 2018-11-02 | 2022-09-06 | EMC IP Holding Company LLC | Scaling out geographically diverse storage |
US10901635B2 (en) * | 2018-12-04 | 2021-01-26 | EMC IP Holding Company LLC | Mapped redundant array of independent nodes for data storage with high performance using logical columns of the nodes with different widths and different positioning patterns |
US11989320B2 (en) | 2018-12-19 | 2024-05-21 | Rankin Labs, Llc | Hidden electronic file system within non-hidden electronic file system |
US11119683B2 (en) | 2018-12-20 | 2021-09-14 | EMC IP Holding Company LLC | Logical compaction of a degraded chunk in a geographically diverse data storage system |
US10931777B2 (en) | 2018-12-20 | 2021-02-23 | EMC IP Holding Company LLC | Network efficient geographically diverse data storage system employing degraded chunks |
US10892782B2 (en) | 2018-12-21 | 2021-01-12 | EMC IP Holding Company LLC | Flexible system and method for combining erasure-coded protection sets |
JP6947717B2 (ja) * | 2018-12-27 | 2021-10-13 | 株式会社日立製作所 | ストレージシステム |
US11023331B2 (en) | 2019-01-04 | 2021-06-01 | EMC IP Holding Company LLC | Fast recovery of data in a geographically distributed storage environment |
US11586595B1 (en) * | 2019-01-11 | 2023-02-21 | Amazon Technologies, Inc. | Space-efficient techniques for generating unique instances of data objects |
US11544387B2 (en) * | 2019-01-16 | 2023-01-03 | International Business Machines Corporation | Hash protection within an object storage library |
US11334456B1 (en) * | 2019-01-18 | 2022-05-17 | EMC IP Holding Company LLC | Space efficient data protection |
US10942827B2 (en) | 2019-01-22 | 2021-03-09 | EMC IP Holding Company LLC | Replication of data in a geographically distributed storage environment |
US10764135B2 (en) | 2019-01-29 | 2020-09-01 | Dell Products L.P. | Method and system for solution integration labeling |
US10846003B2 (en) | 2019-01-29 | 2020-11-24 | EMC IP Holding Company LLC | Doubly mapped redundant array of independent nodes for data storage |
US10936239B2 (en) | 2019-01-29 | 2021-03-02 | EMC IP Holding Company LLC | Cluster contraction of a mapped redundant array of independent nodes |
US10740023B1 (en) | 2019-01-29 | 2020-08-11 | Dell Products L.P. | System and method for dynamic application access-based mapping |
US10972343B2 (en) | 2019-01-29 | 2021-04-06 | Dell Products L.P. | System and method for device configuration update |
US10901641B2 (en) | 2019-01-29 | 2021-01-26 | Dell Products L.P. | Method and system for inline deduplication |
EP3918484A4 (en) | 2019-01-29 | 2023-03-29 | Cloud Storage, Inc. | METHOD FOR REPAIRING ENCODE AND MEMORY NODES FOR MINIMUM MEMORY REGENERATION CODES FOR DISTRIBUTED MEMORY SYSTEMS |
US10979312B2 (en) | 2019-01-29 | 2021-04-13 | Dell Products L.P. | System and method to assign, monitor, and validate solution infrastructure deployment prerequisites in a customer data center |
US10866766B2 (en) | 2019-01-29 | 2020-12-15 | EMC IP Holding Company LLC | Affinity sensitive data convolution for data storage systems |
US11442642B2 (en) * | 2019-01-29 | 2022-09-13 | Dell Products L.P. | Method and system for inline deduplication using erasure coding to minimize read and write operations |
US20200241781A1 (en) | 2019-01-29 | 2020-07-30 | Dell Products L.P. | Method and system for inline deduplication using erasure coding |
US10747522B1 (en) | 2019-01-29 | 2020-08-18 | EMC IP Holding Company LLC | Method and system for non-disruptive host repurposing |
US10942825B2 (en) | 2019-01-29 | 2021-03-09 | EMC IP Holding Company LLC | Mitigating real node failure in a mapped redundant array of independent nodes |
US10911307B2 (en) | 2019-01-29 | 2021-02-02 | Dell Products L.P. | System and method for out of the box solution-level configuration and diagnostic logging and reporting |
US11334254B2 (en) | 2019-03-29 | 2022-05-17 | Pure Storage, Inc. | Reliability based flash page sizing |
US10944826B2 (en) | 2019-04-03 | 2021-03-09 | EMC IP Holding Company LLC | Selective instantiation of a storage service for a mapped redundant array of independent nodes |
US11029865B2 (en) | 2019-04-03 | 2021-06-08 | EMC IP Holding Company LLC | Affinity sensitive storage of data corresponding to a mapped redundant array of independent nodes |
US11775189B2 (en) | 2019-04-03 | 2023-10-03 | Pure Storage, Inc. | Segment level heterogeneity |
US11099986B2 (en) | 2019-04-12 | 2021-08-24 | Pure Storage, Inc. | Efficient transfer of memory contents |
US10936218B2 (en) * | 2019-04-18 | 2021-03-02 | EMC IP Holding Company LLC | Facilitating an out-of-order transmission of segments of multi-segment data portions for distributed storage devices |
US11113146B2 (en) | 2019-04-30 | 2021-09-07 | EMC IP Holding Company LLC | Chunk segment recovery via hierarchical erasure coding in a geographically diverse data storage system |
US11119686B2 (en) | 2019-04-30 | 2021-09-14 | EMC IP Holding Company LLC | Preservation of data during scaling of a geographically diverse data storage system |
US11121727B2 (en) | 2019-04-30 | 2021-09-14 | EMC IP Holding Company LLC | Adaptive data storing for data storage systems employing erasure coding |
US11748004B2 (en) | 2019-05-03 | 2023-09-05 | EMC IP Holding Company LLC | Data replication using active and passive data storage modes |
US11729184B2 (en) | 2019-05-28 | 2023-08-15 | Rankin Labs, Llc | Detecting covertly stored payloads of data within a network |
US11714572B2 (en) | 2019-06-19 | 2023-08-01 | Pure Storage, Inc. | Optimized data resiliency in a modular storage system |
US11281394B2 (en) | 2019-06-24 | 2022-03-22 | Pure Storage, Inc. | Replication across partitioning schemes in a distributed storage system |
US11209996B2 (en) | 2019-07-15 | 2021-12-28 | EMC IP Holding Company LLC | Mapped cluster stretching for increasing workload in a data storage system |
US11023145B2 (en) | 2019-07-30 | 2021-06-01 | EMC IP Holding Company LLC | Hybrid mapped clusters for data storage |
US11449399B2 (en) | 2019-07-30 | 2022-09-20 | EMC IP Holding Company LLC | Mitigating real node failure of a doubly mapped redundant array of independent nodes |
US10963345B2 (en) | 2019-07-31 | 2021-03-30 | Dell Products L.P. | Method and system for a proactive health check and reconstruction of data |
US11609820B2 (en) | 2019-07-31 | 2023-03-21 | Dell Products L.P. | Method and system for redundant distribution and reconstruction of storage metadata |
US11328071B2 (en) | 2019-07-31 | 2022-05-10 | Dell Products L.P. | Method and system for identifying actor of a fraudulent action during legal hold and litigation |
US11372730B2 (en) | 2019-07-31 | 2022-06-28 | Dell Products L.P. | Method and system for offloading a continuous health-check and reconstruction of data in a non-accelerator pool |
US11775193B2 (en) | 2019-08-01 | 2023-10-03 | Dell Products L.P. | System and method for indirect data classification in a storage system operations |
CN111399755A (zh) * | 2019-09-10 | 2020-07-10 | 杭州海康威视***技术有限公司 | 数据存储管理的方法和装置 |
US11228322B2 (en) | 2019-09-13 | 2022-01-18 | EMC IP Holding Company LLC | Rebalancing in a geographically diverse storage system employing erasure coding |
US11537725B2 (en) * | 2019-09-23 | 2022-12-27 | Amazon Technologies, Inc. | Encrypted cross-zone replication for cross-zone replicated block storage devices |
US11494108B2 (en) * | 2019-09-23 | 2022-11-08 | Amazon Technologies, Inc. | Cross-zone replicated block storage devices |
US11449248B2 (en) | 2019-09-26 | 2022-09-20 | EMC IP Holding Company LLC | Mapped redundant array of independent data storage regions |
US11893126B2 (en) | 2019-10-14 | 2024-02-06 | Pure Storage, Inc. | Data deletion for a multi-tenant environment |
US11435910B2 (en) | 2019-10-31 | 2022-09-06 | EMC IP Holding Company LLC | Heterogeneous mapped redundant array of independent nodes for data storage |
US11288139B2 (en) | 2019-10-31 | 2022-03-29 | EMC IP Holding Company LLC | Two-step recovery employing erasure coding in a geographically diverse data storage system |
US11119690B2 (en) | 2019-10-31 | 2021-09-14 | EMC IP Holding Company LLC | Consolidation of protection sets in a geographically diverse data storage environment |
US11435957B2 (en) | 2019-11-27 | 2022-09-06 | EMC IP Holding Company LLC | Selective instantiation of a storage service for a doubly mapped redundant array of independent nodes |
US12001684B2 (en) | 2019-12-12 | 2024-06-04 | Pure Storage, Inc. | Optimizing dynamic power loss protection adjustment in a storage system |
US11704192B2 (en) | 2019-12-12 | 2023-07-18 | Pure Storage, Inc. | Budgeting open blocks based on power loss protection |
US11416144B2 (en) | 2019-12-12 | 2022-08-16 | Pure Storage, Inc. | Dynamic use of segment or zone power loss protection in a flash device |
US11847331B2 (en) | 2019-12-12 | 2023-12-19 | Pure Storage, Inc. | Budgeting open blocks of a storage unit based on power loss prevention |
CN111212111B (zh) * | 2019-12-17 | 2022-05-13 | 腾讯科技(深圳)有限公司 | 对象存储服务管理方法及电子设备 |
WO2021127320A1 (en) * | 2019-12-18 | 2021-06-24 | John Rankin | Distribution of data over a network with interconnected rings |
US11144220B2 (en) | 2019-12-24 | 2021-10-12 | EMC IP Holding Company LLC | Affinity sensitive storage of data corresponding to a doubly mapped redundant array of independent nodes |
US11231860B2 (en) | 2020-01-17 | 2022-01-25 | EMC IP Holding Company LLC | Doubly mapped redundant array of independent nodes for data storage with high performance |
US11210002B2 (en) | 2020-01-29 | 2021-12-28 | Samsung Electronics Co., Ltd. | Offloaded device-driven erasure coding |
US11514181B2 (en) * | 2020-02-12 | 2022-11-29 | Netapp, Inc. | Bin syncing technique for multiple data protection schemes |
US11188432B2 (en) | 2020-02-28 | 2021-11-30 | Pure Storage, Inc. | Data resiliency by partially deallocating data blocks of a storage device |
US11416357B2 (en) | 2020-03-06 | 2022-08-16 | Dell Products L.P. | Method and system for managing a spare fault domain in a multi-fault domain data cluster |
US11175842B2 (en) | 2020-03-06 | 2021-11-16 | Dell Products L.P. | Method and system for performing data deduplication in a data pipeline |
US11281535B2 (en) | 2020-03-06 | 2022-03-22 | Dell Products L.P. | Method and system for performing a checkpoint zone operation for a spare persistent storage |
US11301327B2 (en) | 2020-03-06 | 2022-04-12 | Dell Products L.P. | Method and system for managing a spare persistent storage device and a spare node in a multi-node data cluster |
US11119858B1 (en) | 2020-03-06 | 2021-09-14 | Dell Products L.P. | Method and system for performing a proactive copy operation for a spare persistent storage |
US11507308B2 (en) | 2020-03-30 | 2022-11-22 | EMC IP Holding Company LLC | Disk access event control for mapped nodes supported by a real cluster storage system |
US11507297B2 (en) | 2020-04-15 | 2022-11-22 | Pure Storage, Inc. | Efficient management of optimal read levels for flash storage systems |
US11256587B2 (en) | 2020-04-17 | 2022-02-22 | Pure Storage, Inc. | Intelligent access to a storage device |
US11474986B2 (en) | 2020-04-24 | 2022-10-18 | Pure Storage, Inc. | Utilizing machine learning to streamline telemetry processing of storage media |
US11416338B2 (en) | 2020-04-24 | 2022-08-16 | Pure Storage, Inc. | Resiliency scheme to enhance storage performance |
US20210334241A1 (en) * | 2020-04-24 | 2021-10-28 | Netapp, Inc. | Non-disrputive transitioning between replication schemes |
US11418326B2 (en) | 2020-05-21 | 2022-08-16 | Dell Products L.P. | Method and system for performing secure data transactions in a data cluster |
US11288229B2 (en) | 2020-05-29 | 2022-03-29 | EMC IP Holding Company LLC | Verifiable intra-cluster migration for a chunk storage system |
US11768763B2 (en) | 2020-07-08 | 2023-09-26 | Pure Storage, Inc. | Flash secure erase |
US11681448B2 (en) | 2020-09-08 | 2023-06-20 | Pure Storage, Inc. | Multiple device IDs in a multi-fabric module storage system |
US11513974B2 (en) | 2020-09-08 | 2022-11-29 | Pure Storage, Inc. | Using nonce to control erasure of data blocks of a multi-controller storage system |
CN114327239A (zh) * | 2020-09-27 | 2022-04-12 | 伊姆西Ip控股有限责任公司 | 存储和访问数据的方法、电子设备和计算机程序产品 |
US11693983B2 (en) | 2020-10-28 | 2023-07-04 | EMC IP Holding Company LLC | Data protection via commutative erasure coding in a geographically diverse data storage system |
US11561856B2 (en) | 2020-12-10 | 2023-01-24 | Nutanix, Inc. | Erasure coding of replicated data blocks |
WO2022128080A1 (en) * | 2020-12-16 | 2022-06-23 | Huawei Technologies Co., Ltd. | Storing data in computer storage |
US11487455B2 (en) | 2020-12-17 | 2022-11-01 | Pure Storage, Inc. | Dynamic block allocation to optimize storage system performance |
US11614880B2 (en) | 2020-12-31 | 2023-03-28 | Pure Storage, Inc. | Storage system with selectable write paths |
US11847324B2 (en) | 2020-12-31 | 2023-12-19 | Pure Storage, Inc. | Optimizing resiliency groups for data regions of a storage system |
US11847141B2 (en) | 2021-01-19 | 2023-12-19 | EMC IP Holding Company LLC | Mapped redundant array of independent nodes employing mapped reliability groups for data storage |
US11625174B2 (en) | 2021-01-20 | 2023-04-11 | EMC IP Holding Company LLC | Parity allocation for a virtual redundant array of independent disks |
US11630593B2 (en) | 2021-03-12 | 2023-04-18 | Pure Storage, Inc. | Inline flash memory qualification in a storage system |
US11507597B2 (en) | 2021-03-31 | 2022-11-22 | Pure Storage, Inc. | Data replication to meet a recovery point objective |
US11354191B1 (en) | 2021-05-28 | 2022-06-07 | EMC IP Holding Company LLC | Erasure coding in a large geographically diverse data storage system |
US11449234B1 (en) | 2021-05-28 | 2022-09-20 | EMC IP Holding Company LLC | Efficient data access operations via a mapping layer instance for a doubly mapped redundant array of independent nodes |
US12032848B2 (en) | 2021-06-21 | 2024-07-09 | Pure Storage, Inc. | Intelligent block allocation in a heterogeneous storage system |
US11832410B2 (en) | 2021-09-14 | 2023-11-28 | Pure Storage, Inc. | Mechanical energy absorbing bracket apparatus |
US11726779B2 (en) * | 2021-11-03 | 2023-08-15 | Sap Se | Code simplification system |
US11994723B2 (en) | 2021-12-30 | 2024-05-28 | Pure Storage, Inc. | Ribbon cable alignment apparatus |
US20230237020A1 (en) * | 2022-01-24 | 2023-07-27 | Cohesity, Inc. | Container-based erasure coding |
US20240111432A1 (en) * | 2022-10-04 | 2024-04-04 | Scality, S.A. | Erasure coding implementation with reduced parity calculation overhead |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH07141123A (ja) * | 1993-09-27 | 1995-06-02 | Hitachi Ltd | アレイ型ディスクシステムおよびアレイ型ディスクシステムの制御方法 |
US5761402A (en) * | 1993-03-08 | 1998-06-02 | Hitachi, Ltd. | Array type disk system updating redundant data asynchronously with data access |
WO2004046971A1 (en) * | 2002-11-14 | 2004-06-03 | Isilon Systems, Inc. | Systems and methods for restriping files in a distributed file system |
US20090119395A1 (en) * | 2007-11-01 | 2009-05-07 | Hitachi, Ltd. | Information processing system and data management method |
US20110029840A1 (en) * | 2009-07-31 | 2011-02-03 | Microsoft Corporation | Erasure Coded Storage Aggregation in Data Centers |
US20120047111A1 (en) * | 2010-08-18 | 2012-02-23 | Hayden Mark G | Method and system for parity-page distribution among nodes of a multi-node data-storage system |
US20120060072A1 (en) * | 2010-09-08 | 2012-03-08 | Microsoft Corporation | Erasure coding immutable data |
Family Cites Families (41)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5740465A (en) * | 1992-04-08 | 1998-04-14 | Hitachi, Ltd. | Array disk controller for grouping host commands into a single virtual host command |
EP0826181A4 (en) | 1995-04-11 | 2005-02-09 | Kinetech Inc | IDENTIFYING DATA IN A DATA PROCESSING SYSTEM |
US6925547B2 (en) | 2000-12-14 | 2005-08-02 | Silicon Graphics, Inc. | Remote address translation in a multiprocessor system |
US6862692B2 (en) * | 2001-01-29 | 2005-03-01 | Adaptec, Inc. | Dynamic redistribution of parity groups |
US6795903B2 (en) | 2002-01-17 | 2004-09-21 | Thomas Licensing S.A. | System and method for searching for duplicate data |
US7577816B2 (en) | 2003-08-18 | 2009-08-18 | Cray Inc. | Remote translation mechanism for a multinode system |
US6922766B2 (en) | 2002-09-04 | 2005-07-26 | Cray Inc. | Remote translation mechanism for a multi-node system |
US20040153841A1 (en) | 2003-01-16 | 2004-08-05 | Silicon Graphics, Inc. | Failure hierarchy in a cluster filesystem |
US7373345B2 (en) | 2003-02-21 | 2008-05-13 | Caringo, Inc. | Additional hash functions in content-based addressing |
US7363462B2 (en) | 2003-04-04 | 2008-04-22 | Sun Microsystems, Inc. | Performing virtual to global address translation in processing subsystem |
US7765381B2 (en) | 2003-04-04 | 2010-07-27 | Oracle America, Inc. | Multi-node system in which home memory subsystem stores global to local address translation information for replicating nodes |
US7069392B2 (en) | 2003-06-12 | 2006-06-27 | Newisys, Inc. | Methods and apparatus for extended packet communications between multiprocessor clusters |
US7310703B2 (en) * | 2003-10-23 | 2007-12-18 | Hewlett-Packard Development Company, L.P. | Methods of reading and writing data |
US7284088B2 (en) * | 2003-10-23 | 2007-10-16 | Hewlett-Packard Development Company, L.P. | Methods of reading and writing data |
US7266716B2 (en) * | 2003-10-23 | 2007-09-04 | Hewlett-Packard Development Company, L.P. | Method and recovery of data using erasure coded data from stripe blocks |
US7546342B2 (en) * | 2004-05-14 | 2009-06-09 | Microsoft Corporation | Distributed hosting of web content using partial replication |
US7552356B1 (en) * | 2004-06-30 | 2009-06-23 | Sun Microsystems, Inc. | Distributed data storage system for fixed content |
US7681105B1 (en) | 2004-08-09 | 2010-03-16 | Bakbone Software, Inc. | Method for lock-free clustered erasure coding and recovery of data across a plurality of data stores in a network |
US8843727B2 (en) | 2004-09-30 | 2014-09-23 | Intel Corporation | Performance enhancement of address translation using translation tables covering large address spaces |
US7778972B1 (en) | 2005-12-29 | 2010-08-17 | Amazon Technologies, Inc. | Dynamic object replication within a distributed storage system |
US7734603B1 (en) | 2006-01-26 | 2010-06-08 | Netapp, Inc. | Content addressable storage array element |
BRPI0604440B1 (pt) | 2006-10-10 | 2018-07-10 | EMBRAER - Empresa Brasileira de Aeronáutica S.A. | Configuração aplicada em sistema para controle da autoridade do leme de uma aeronave |
US7917469B2 (en) | 2006-11-08 | 2011-03-29 | Hitachi Data Systems Corporation | Fast primary cluster recovery |
US7590672B2 (en) | 2006-12-11 | 2009-09-15 | Bycast Inc. | Identification of fixed content objects in a distributed fixed content storage system |
US8090792B2 (en) * | 2007-03-08 | 2012-01-03 | Nec Laboratories America, Inc. | Method and system for a self managing and scalable grid storage |
KR100905218B1 (ko) | 2007-04-09 | 2009-07-01 | 삼성전자주식회사 | 애드혹 네트워크에서 콘텐츠 중복 검출 방법 |
US7979645B2 (en) | 2007-09-14 | 2011-07-12 | Ricoh Company, Limited | Multiprocessor system for memory mapping of processing nodes |
US20090089537A1 (en) | 2007-09-28 | 2009-04-02 | Sun Microsystems, Inc. | Apparatus and method for memory address translation across multiple nodes |
US7747663B2 (en) | 2008-03-05 | 2010-06-29 | Nec Laboratories America, Inc. | System and method for content addressable storage |
US8255430B2 (en) | 2008-08-26 | 2012-08-28 | Caringo, Inc. | Shared namespace for storage clusters |
US20100235409A1 (en) | 2009-03-10 | 2010-09-16 | Global Relay Communications Inc. | System and method for managing data stored in a data network |
US8069366B1 (en) | 2009-04-29 | 2011-11-29 | Netapp, Inc. | Global write-log device for managing write logs of nodes of a cluster storage system |
US8296515B1 (en) * | 2009-05-22 | 2012-10-23 | Nvidia Corporation | RAID-6 computation system and method |
US8918365B2 (en) | 2009-06-19 | 2014-12-23 | Blekko, Inc. | Dedicating disks to reading or writing |
JP5255127B2 (ja) | 2009-06-26 | 2013-08-07 | 株式会社日立製作所 | ストレージシステム、及びストレージシステムの制御方法 |
CN102549524B (zh) | 2009-07-20 | 2015-05-06 | 卡林戈公司 | 存储集群中的自适应功率保存 |
US8132043B2 (en) | 2009-12-17 | 2012-03-06 | Symantec Corporation | Multistage system recovery framework |
WO2011083508A1 (en) | 2010-01-05 | 2011-07-14 | Hitachi,Ltd. | Storage system and its file management method |
US9047351B2 (en) | 2010-04-12 | 2015-06-02 | Sandisk Enterprise Ip Llc | Cluster of processing nodes with distributed global flash memory using commodity server technology |
US8504535B1 (en) * | 2010-12-20 | 2013-08-06 | Amazon Technologies, Inc. | Erasure coding and redundant replication |
US20120266044A1 (en) * | 2011-04-18 | 2012-10-18 | The Chinese University Of Hong Kong | Network-coding-based distributed file system |
-
2012
- 2012-06-13 US US13/517,527 patent/US8799746B2/en active Active
-
2013
- 2013-06-04 WO PCT/US2013/044045 patent/WO2013188169A1/en active Application Filing
- 2013-06-04 CA CA2876638A patent/CA2876638A1/en not_active Abandoned
- 2013-06-04 EP EP13805176.8A patent/EP2862085A4/en not_active Withdrawn
- 2013-06-04 AU AU2013274626A patent/AU2013274626A1/en not_active Abandoned
- 2013-06-04 JP JP2015517293A patent/JP6280108B2/ja active Active
- 2013-06-04 CN CN201380043101.0A patent/CN104541251B/zh active Active
-
2014
- 2014-06-30 US US14/320,494 patent/US9148174B2/en active Active
-
2015
- 2015-08-24 US US14/834,017 patent/US9916198B2/en active Active
-
2017
- 2017-08-24 US US15/685,833 patent/US10437672B2/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5761402A (en) * | 1993-03-08 | 1998-06-02 | Hitachi, Ltd. | Array type disk system updating redundant data asynchronously with data access |
JPH07141123A (ja) * | 1993-09-27 | 1995-06-02 | Hitachi Ltd | アレイ型ディスクシステムおよびアレイ型ディスクシステムの制御方法 |
WO2004046971A1 (en) * | 2002-11-14 | 2004-06-03 | Isilon Systems, Inc. | Systems and methods for restriping files in a distributed file system |
JP2010044789A (ja) * | 2002-11-14 | 2010-02-25 | Isilon Systems Inc | 分散ファイルシステムにおけるファイルの再ストライピングのためのシステム及び方法 |
US20090119395A1 (en) * | 2007-11-01 | 2009-05-07 | Hitachi, Ltd. | Information processing system and data management method |
JP2009116414A (ja) * | 2007-11-01 | 2009-05-28 | Hitachi Ltd | 情報処理システム及びデータ管理方法 |
US20110029840A1 (en) * | 2009-07-31 | 2011-02-03 | Microsoft Corporation | Erasure Coded Storage Aggregation in Data Centers |
US20120047111A1 (en) * | 2010-08-18 | 2012-02-23 | Hayden Mark G | Method and system for parity-page distribution among nodes of a multi-node data-storage system |
US20120060072A1 (en) * | 2010-09-08 | 2012-03-08 | Microsoft Corporation | Erasure coding immutable data |
Cited By (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2017142605A (ja) * | 2016-02-09 | 2017-08-17 | 株式会社日立製作所 | バックアップリストアシステム及びリストア方法 |
JP2018116704A (ja) * | 2017-01-20 | 2018-07-26 | 三星電子株式会社Samsung Electronics Co.,Ltd. | ストレージシステム及びその動作方法 |
CN108334285A (zh) * | 2017-01-20 | 2018-07-27 | 三星电子株式会社 | 存储***和操作存储***的方法 |
CN108334285B (zh) * | 2017-01-20 | 2023-04-18 | 三星电子株式会社 | 存储***和操作存储***的方法 |
US11429487B2 (en) | 2017-01-20 | 2022-08-30 | Samsung Electronics Co., Ltd. | Control plane method and apparatus for providing erasure code protection across multiple storage devices |
JP7002346B2 (ja) | 2017-01-20 | 2022-01-20 | 三星電子株式会社 | ストレージシステム及びその動作方法 |
US11275762B2 (en) | 2017-03-20 | 2022-03-15 | Samsung Electronics Co., Ltd. | System and method for hybrid data reliability for object storage devices |
JP2018156656A (ja) * | 2017-03-20 | 2018-10-04 | 三星電子株式会社Samsung Electronics Co.,Ltd. | 仮想装置階層を利用した複数のメモリ装置を含む仮想装置に対する客体の格納及び読み取り方法とこれを用いたストレージ装置 |
US11288119B2 (en) | 2017-03-20 | 2022-03-29 | Samsung Electronics Co., Ltd. | Key value SSD |
JP2020518934A (ja) * | 2017-06-08 | 2020-06-25 | ヒタチ ヴァンタラ コーポレーションHitachi Vantara Corporation | 地理的に分散したオブジェクトデータの高速呼び戻し |
JP2019128959A (ja) * | 2018-01-19 | 2019-08-01 | 三星電子株式会社Samsung Electronics Co.,Ltd. | キーバリューデータ信頼性システム、そのデータ格納方法、及び、その方法を具現するコンピュータコードを含む非一時的なコンピュータ読出し可能な媒体 |
JP2019128960A (ja) * | 2018-01-19 | 2019-08-01 | 三星電子株式会社Samsung Electronics Co.,Ltd. | データストレージシステム及びキーバリューペアのオブジェクトにアクセスする方法 |
JP7140688B2 (ja) | 2018-01-19 | 2022-09-21 | 三星電子株式会社 | データストレージシステム及びキーバリューペアのオブジェクトにアクセスする方法 |
KR102460568B1 (ko) | 2018-01-19 | 2022-10-31 | 삼성전자주식회사 | 대형 키 밸류 객체들을 저장하는 시스템 및 방법 |
JP7171452B2 (ja) | 2018-01-19 | 2022-11-15 | 三星電子株式会社 | キーバリューデータ信頼性システム、そのデータ格納方法、及び、その方法を具現するコンピュータコードを含む非一時的なコンピュータ読出し可能な媒体 |
KR20190088873A (ko) * | 2018-01-19 | 2019-07-29 | 삼성전자주식회사 | 대형 키 밸류 객체들을 저장하는 시스템 및 방법 |
JP2022512064A (ja) * | 2018-10-15 | 2022-02-02 | ネットアップ,インコーポレイテッド | 様々なデータ冗長性スキームを備えた、システムにおける利用可能なストレージ空間を改善すること |
JP7312251B2 (ja) | 2018-10-15 | 2023-07-20 | ネットアップ,インコーポレイテッド | 様々なデータ冗長性スキームを備えた、システムにおける利用可能なストレージ空間を改善すること |
JP2022169072A (ja) * | 2021-04-27 | 2022-11-09 | 株式会社日立製作所 | ストレージシステム、ストレージノード仮想計算機復旧方法、及び復旧プログラム |
US11762741B2 (en) | 2021-04-27 | 2023-09-19 | Hitachi, Ltd. | Storage system, storage node virtual machine restore method, and recording medium |
JP7355778B2 (ja) | 2021-04-27 | 2023-10-03 | 株式会社日立製作所 | ストレージシステム、ストレージノード仮想計算機復旧方法、及び復旧プログラム |
Also Published As
Publication number | Publication date |
---|---|
US20150019937A1 (en) | 2015-01-15 |
EP2862085A4 (en) | 2016-03-09 |
CA2876638A1 (en) | 2013-12-19 |
JP6280108B2 (ja) | 2018-02-14 |
CN104541251A (zh) | 2015-04-22 |
US20170351575A1 (en) | 2017-12-07 |
WO2013188169A1 (en) | 2013-12-19 |
AU2013274626A1 (en) | 2015-01-22 |
EP2862085A1 (en) | 2015-04-22 |
US8799746B2 (en) | 2014-08-05 |
CN104541251B (zh) | 2017-09-05 |
US20150363269A1 (en) | 2015-12-17 |
US9148174B2 (en) | 2015-09-29 |
US20130339818A1 (en) | 2013-12-19 |
US9916198B2 (en) | 2018-03-13 |
US10437672B2 (en) | 2019-10-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6280108B2 (ja) | ストレージクラスタにおける消失符号付加および複製 | |
US10725884B2 (en) | Object storage system for an unreliable storage medium | |
US10649827B2 (en) | Two level addressing in storage clusters | |
US20170308437A1 (en) | Parity protection for data chunks in an object storage system | |
US9330137B2 (en) | Cloud data backup storage manager | |
US20150347435A1 (en) | Methods and Apparatus for Distributed Data Storage | |
US10700711B1 (en) | Multi-part upload and editing of erasure-coded objects |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20160520 |
|
A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20160520 |
|
A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20170522 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20170606 |
|
A601 | Written request for extension of time |
Free format text: JAPANESE INTERMEDIATE CODE: A601 Effective date: 20170825 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20171205 |
|
TRDD | Decision of grant or rejection written | ||
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20171219 |
|
A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20180118 |
|
R150 | Certificate of patent or registration of utility model |
Ref document number: 6280108 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |