CN107153508B - Data storage method and device - Google Patents
Data storage method and device Download PDFInfo
- Publication number
- CN107153508B CN107153508B CN201610120859.5A CN201610120859A CN107153508B CN 107153508 B CN107153508 B CN 107153508B CN 201610120859 A CN201610120859 A CN 201610120859A CN 107153508 B CN107153508 B CN 107153508B
- Authority
- CN
- China
- Prior art keywords
- metadata
- data
- stored
- storage area
- check value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
- G06F11/1012—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices using codes or arrangements adapted for a specific type of error
- G06F11/1016—Error in accessing a memory location, i.e. addressing error
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
- G06F11/1048—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices using arrangements adapted for a specific error detection or correction feature
- G06F11/1052—Bypassing or disabling error detection or correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Human Computer Interaction (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a data storage method and a data storage device, which are applied to storage equipment, wherein the storage equipment comprises: a metadata storage area, the method comprising: receiving a data storage request, determining a target storage position of data to be stored contained in the data storage request, and judging whether the target storage position is in a metadata storage area, if so, judging whether the data to be stored is metadata, if so, storing the data to be stored to the target storage position under the condition of meeting preset storage conditions, and if not, rejecting the data to be stored to the target storage position. By applying the technical scheme provided by the embodiment of the invention to store data, the reliability of the storage system is improved.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a data storage method and apparatus.
Background
For storage systems based on block storage, the importance of the metadata in the storage device is self-evident. Currently, in a storage system, in order to improve data storage efficiency, real data and metadata of the real data are often stored on the same storage device, for example: when storing data, firstly, the storage position of the data on the storage device is calculated, the storage position of the data is determined, and the data is stored to the storage position. This allows for efficient storage of data.
However, when the storage device calculates a storage location offset of data, the offset is likely to occur, for example: the storage position of data on the storage device determined by calculation is right at the storage position where the metadata is stored, the data covers the metadata stored on the storage position, and further the metadata is subjected to misoperation, and the storage device inevitably has problems when reading the metadata, and even causes the storage system to crash.
In the prior art, the protection of metadata is lacked, and when data is stored, the data is likely to cover the stored metadata, so that the storage device cannot correctly read the metadata, even a storage system is crashed, and the reliability of the storage system is reduced.
Disclosure of Invention
The embodiment of the invention aims to provide a data storage method and a data storage device so as to improve the reliability of a storage system.
In order to achieve the above object, an embodiment of the present invention discloses a data storage method applied to a storage device, where the storage device includes: a metadata storage area, the method comprising the steps of:
receiving a data storage request, wherein the data storage request comprises data to be stored;
determining a target storage position of the data to be stored, and judging whether the target storage position is in the metadata storage area;
if the data to be stored is in the metadata storage area, judging whether the data to be stored is metadata;
if so, storing the data to be stored to the target storage position under the condition of meeting a preset storage condition;
if not, refusing to store the data to be stored to the target storage position.
In a specific implementation manner of the present invention, the data storage request further includes: an equipment interface identifier;
the determining whether the data to be stored is metadata includes:
judging whether the equipment interface identification corresponds to a preset equipment interface or not, wherein the preset equipment interface is used for transmitting metadata;
if so, judging that the data to be stored is the metadata.
In a specific implementation manner of the present invention, the storing the data to be stored to the target storage location when a preset storage condition is satisfied includes:
judging whether all metadata stored in the metadata storage area are complete or not;
if so, storing the data to be stored to the target storage position.
In a specific implementation manner of the present invention, the determining whether all metadata stored in the metadata storage area is complete includes:
according to a preset verification algorithm, obtaining first verification values of all metadata stored in the metadata storage area;
judging whether the first check value is consistent with a first preset check value or not;
if yes, judging that all metadata stored in the metadata storage area are complete;
after the storing the data to be stored to the target storage location, further comprising:
according to the preset verification algorithm, second verification values of all metadata stored in the metadata storage area are obtained;
and updating the first preset check value according to the second check value.
In a specific implementation manner of the present invention, the metadata storage area includes: the storage system comprises a main metadata storage area and a standby metadata storage area, wherein the standby metadata storage area is a data backup area of the main metadata storage area, and the target storage position is in the main metadata storage area;
the determining whether all metadata stored in the metadata storage area is complete includes:
according to a preset check algorithm, obtaining third check values of all metadata stored in the main metadata storage area, and judging whether the third check values are consistent with a second preset check value or not;
if the third check value is inconsistent with the second preset check value, obtaining a fourth check value of the metadata stored in the metadata storage area according to the preset check algorithm, and judging whether the fourth check value is consistent with the second preset check value;
if yes, judging that all metadata stored in the metadata storage area are complete;
after the storing the data to be stored to the target storage location, further comprising:
acquiring fifth check values of all metadata stored in the main metadata storage area according to the preset check algorithm;
and updating the second preset check value according to the fifth check value.
In a specific implementation manner of the present invention, after determining that the fourth check value is consistent with the second preset check value, the method further includes:
and updating the metadata stored in the main metadata storage area according to all the metadata stored in the standby metadata storage area.
In a specific implementation manner of the present invention, the method further includes:
and under the condition that the fourth check value is inconsistent with the second preset check value, initializing the metadata in the main metadata storage area and the metadata in the standby metadata storage area, and judging that all the metadata stored in the metadata storage area are complete.
In order to achieve the above object, an embodiment of the present invention further discloses a data storage apparatus, which is applied to a storage device, where the storage device includes: a metadata storage area, the apparatus comprising: the device comprises a request receiving module, a storage position judging module, a data storage module and a data storage rejecting module;
the request receiving module is used for receiving a data storage request, wherein the data storage request comprises data to be stored;
the storage position judging module is used for determining a target storage position of the data to be stored, judging whether the target storage position is in the metadata storage area, and if the target storage position is in the metadata storage area, triggering the data judging module;
the data judging module is used for judging whether the data to be stored is metadata or not, if so, triggering the data storage module, and if not, triggering the data storage rejection module;
the data storage module is used for storing the data to be stored to the target storage position under the condition that a preset storage condition is met;
the data storage rejection module is used for rejecting to store the data to be stored to the target storage position.
In a specific implementation manner of the present invention, the data storage request further includes: an equipment interface identifier;
the data judgment module comprises: the device interface judgment submodule and the data judgment submodule;
the device interface judgment submodule is used for judging whether the device interface identifier corresponds to a preset device interface or not, and if so, triggering the data judgment submodule, wherein the preset device interface is used for transmitting metadata;
and the data judgment submodule is used for judging the data to be stored as the metadata.
In a specific implementation manner of the present invention, the data storage module includes: the complete data judgment submodule and the data storage submodule;
the complete data judgment submodule is used for judging whether all metadata stored in the metadata storage area are complete or not, and if so, the data storage submodule is triggered;
and the data storage submodule is used for storing the data to be stored to the target storage position.
In a specific implementation manner of the present invention, the complete data judgment sub-module includes: the device comprises a check value obtaining unit, a first check value judging unit and a first complete data judging unit;
the verification value obtaining unit is configured to obtain, according to a preset verification algorithm, first verification values of all metadata stored in the metadata storage area;
the first check value judging unit is used for judging whether the first check value is consistent with a first preset check value or not, and if so, the complete data judging unit is triggered;
the complete data judging unit is used for judging that all metadata stored in the metadata storage area are complete;
the device further comprises: the device comprises a first check value obtaining module and a first check value updating module;
the first check value obtaining module is configured to obtain, according to the preset check algorithm, second check values of all metadata stored in the metadata storage area;
and the first check value updating module is used for updating the first preset check value according to the second check value.
In a specific implementation manner of the present invention, the metadata storage area includes: the storage system comprises a main metadata storage area and a standby metadata storage area, wherein the standby metadata storage area is a data backup area of the main metadata storage area, and the target storage position is in the main metadata storage area;
the complete data judgment submodule comprises: the device comprises a second check value judging unit, a third check value judging unit and a second complete data judging unit;
the second check value judging unit is configured to obtain third check values of all metadata stored in the primary metadata storage area according to a preset check algorithm, judge whether the third check values are consistent with a second preset check value, and trigger the third check value judging unit if the third check values are inconsistent with the second preset check value;
the third check value judging unit is configured to obtain a fourth check value of the metadata stored in the metadata storage area according to the preset check algorithm, judge whether the fourth check value is consistent with the second preset check value, and trigger the second complete data judging unit if the fourth check value is consistent with the second preset check value;
the second complete data determination unit is configured to determine that all metadata stored in the metadata storage area are complete;
the device further comprises: the second check value obtaining module and the second check value updating module;
the second check value obtaining module is configured to obtain a fifth check value of all metadata stored in the main metadata storage area according to the preset check algorithm;
and the second check value updating module is used for updating the second preset check value according to the fifth check value.
In a specific implementation manner of the present invention, the apparatus further includes: a metadata update module for updating the metadata of the received data,
and the main metadata storage area is used for updating the metadata stored in the main metadata storage area according to all the metadata stored in the standby metadata storage area.
In a specific implementation manner of the present invention, the apparatus further includes: a metadata initialization module for initializing the metadata of the mobile terminal,
and the metadata management module is configured to initialize the metadata in the main metadata storage area and the metadata in the standby metadata storage area under the condition that the fourth check value is inconsistent with the second preset check value, and determine that all the metadata stored in the metadata storage area are complete.
Therefore, in the embodiment of the invention, after receiving a data storage request, a storage device determines a target storage location of data to be stored, and determines whether the target storage location is in a metadata storage area, if so, determines whether the data to be stored is metadata, if so, stores the data to be stored to the target storage location under the condition of meeting a preset storage condition, and if not, refuses to store the data to be stored, thereby avoiding the problem that real data covers the metadata, ensuring that the storage device can correctly read the metadata, and improving the reliability of a storage system.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a data storage method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart diagram of another data storage method according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart diagram of another data storage method according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart diagram of another data storage method according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a data storage device according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of another data storage device according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of another data storage device according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of another data storage device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention discloses a data storage method and a data storage device, which are applied to storage equipment, wherein the storage equipment comprises: a metadata storage area, the method comprising: receiving a data storage request, determining a target storage position of data to be stored contained in the data storage request, and judging whether the target storage position is in a metadata storage area, if so, judging whether the data to be stored is metadata, if so, storing the data to be stored to the target storage position under the condition of meeting preset storage conditions, and if not, rejecting the data to be stored to the target storage position.
The present invention will be described in detail below with reference to specific examples.
Referring to fig. 1, fig. 1 is a schematic flowchart of a data storage method provided by an embodiment of the present invention, and is applied to a storage device, where the storage device may include: a metadata storage area, the method may comprise the steps of:
s101: receiving a data storage request;
the data storage request comprises data to be stored. Here, the data to be stored may be real data or metadata of the real data.
In practical application, the data storage request may be a data storage request sent by a user and received by a virtual file system in the storage device, where the virtual file system is a file system based on cloud storage, and serves as a bottom storage basis of the cloud storage system, and may provide a storage service for an upper application.
The metadata of the real data may be: the storage location of the real data, the data size … … of the real data, and the like, which are not limited in this application.
S102: determining a target storage location of the data to be stored, and judging whether the target storage location is in the metadata storage area, if so, executing step S103;
of course, in other embodiments of the present invention, if the target storage location is not within the metadata storage area, the data to be stored may be stored directly to the target storage location.
In a specific implementation manner of the present invention, the storage device may include: the device comprises a metadata storage area and a real data storage area, wherein the metadata storage area is used for storing metadata, and the real data storage area is used for storing real data.
For a storage system based on block storage, the importance of metadata in storage devices is self-evident, because a storage system may include a plurality of storage devices, and each storage device stores a large amount of real data, the storage system can obtain real data quickly according to metadata without having to go to all real data in each storage device to obtain the real data. Therefore, in this case, after the storage device receives the data storage request, it is necessary to determine whether the target storage location of the data to be stored is in the metadata storage area. Here, the prior art of how to determine the target storage location data of the data to be stored is not described here again.
S103: judging whether the data to be stored is metadata, if so, executing step S104, otherwise, executing step S105;
s104: storing the data to be stored to the target storage position under the condition that a preset storage condition is met;
s105: and refusing to store the data to be stored to the target storage position.
Specifically, when it is determined that a target storage location of data to be stored is in the metadata storage area, in order to ensure that the metadata in the metadata storage area is correct, it is required to determine whether the data to be stored is the metadata, if the data to be stored is the metadata, the data to be stored is stored to the determined target storage location of the data to be stored under the condition that a preset storage condition is met, that is, the data to be stored is stored in the metadata storage area, and if the data to be stored is not the metadata, the data to be stored is rejected to be stored to the determined target storage location of the data to be stored, that is, the data to be stored is not allowed to be stored in the metadata storage area.
In a specific implementation manner of the present invention, the data storage request may further include: an equipment interface identifier;
in this case, the determining whether the data to be stored is metadata may include:
judging whether the equipment interface identification corresponds to a preset equipment interface or not, wherein the preset equipment interface is used for transmitting metadata;
if so, judging that the data to be stored is metadata;
if not, judging that the data to be stored is not metadata.
In practical applications, a device interface may be preset in the storage device, such as: the preset device interface is responsible for transmitting metadata and related information of the metadata, and refusing to transmit real data and related information of the real data, wherein the related information of the metadata comprises: the data size of the metadata, the target storage location of the metadata, and the like, and the relevant information of the real data includes: the data size of the real data, the target storage location of the real data, and the like. Therefore, when judging whether the data to be stored is metadata, only the equipment interface corresponding to the equipment interface identifier contained in the data storage request needs to be judged whether to be a preset equipment interface for transmitting the metadata, so that whether the data to be stored is the metadata is conveniently and quickly judged, and the real data is effectively prevented from being stored in the metadata storage area.
By applying the embodiment shown in fig. 1, after receiving a data storage request, a storage device determines a target storage location of data to be stored, and determines whether the target storage location is in a metadata storage area, if so, determines whether the data to be stored is metadata, if so, stores the data to be stored to the target storage location under the condition that a preset storage condition is met, and if not, refuses to store the data to be stored, thereby avoiding the problem that real data covers the metadata, ensuring that the storage device can correctly read the metadata, and improving the reliability of a storage system.
Referring to fig. 2, fig. 2 is a schematic flow chart of another data storage method provided by the embodiment of the present invention, which is applied to a storage device, where the storage device may include: a metadata storage area, wherein step S104 may include:
s1041: judging whether all the stored metadata in the metadata storage area are complete, if so, executing the step S1042;
s1042: and storing the data to be stored to the target storage position.
In practical applications, there is a high possibility that information of metadata in a metadata storage area is missing or wrong due to a user's misoperation, and at this time, if the incomplete metadata is not processed, or the received metadata is stored in the metadata storage area, then when the storage device reads the incomplete metadata, the storage system may be crashed. Therefore, before the metadata is stored in the target storage position of the metadata storage area, whether all the stored metadata in the metadata storage area are complete or not is judged, so that the problem that the metadata in the metadata storage area is incomplete is found and processed as soon as possible, and the problem of storage system breakdown is avoided.
By applying the embodiment shown in fig. 2, it is determined whether all metadata stored in the metadata storage area are complete, and the data to be stored is stored in the target storage location only when all metadata stored in the metadata storage area are complete. Therefore, the correctness and the integrity of the metadata in the metadata storage area are ensured, and the problem of storage system crash caused by the incomplete metadata in the metadata storage area is avoided.
Referring to fig. 3, fig. 3 is a schematic flow chart of another data storage method provided by the embodiment of the present invention, which is applied to a storage device, where the storage device may include: a metadata storage area, in the method, step S1041 may include:
s1041 a: according to a preset verification algorithm, obtaining first verification values of all metadata stored in the metadata storage area;
the preset check algorithm may be a CRC check algorithm, and may also be other check algorithms, which is not limited in this invention.
S1041 b: judging whether the first check value is consistent with a first preset check value, if so, executing step S1041 c;
the first preset check value can be obtained after the storage device stores the data to be stored in the last metadata storage area, so that the method is more intelligent; it is also possible for the user to be given metadata based on the metadata already stored in the metadata storage area, which is more user-friendly.
S1041 c: it is determined that all metadata stored in the metadata storage area is complete.
Assuming that the first preset check value is 145, if the first check value is 145, and 145 is 145, it may be determined that all metadata stored in the metadata storage area is complete, and at this time, the data to be stored may be stored to a target storage location in the metadata storage area.
In a specific implementation manner, if the first check value is inconsistent with the first preset check value, the metadata in the metadata storage area may be initialized, and then the data to be stored is stored to the target storage location in the metadata storage area. If the first preset check value is 145, and if the first check value is 120 according to the preset check algorithm, and 145 ≠ 120, it can be determined that all metadata stored in the metadata storage area is incomplete, and at this time, all metadata in the metadata storage area can be initialized. Therefore, if the part with the wrong metadata storage in the metadata storage area can be processed in time, and the reliability of the storage system is improved.
In addition, in the data storage method, after storing the data to be stored in the target storage location, the method may further include:
s106: according to the preset verification algorithm, second verification values of all metadata stored in the metadata storage area are obtained;
s107: and updating the first preset check value according to the second check value.
In practical application, after the data to be stored is stored in the metadata storage area, the metadata in the metadata storage area is changed, and in order to ensure correct verification, the first preset verification value needs to be updated. Assuming that the first preset check value before the data to be stored is 145, if the first check value is judged to be consistent with the first preset check value, the data to be stored is stored in the metadata storage area, and at this time, the second check value is obtained as 150, and the first preset check value is updated to be 150.
It should be noted that, when data is not stored in the metadata storage area, the data to be stored may be directly stored in the target storage location in the metadata storage area, a second check value is obtained after storage, the second check value is used as a first preset check value, and when data to be stored needs to be stored in the metadata storage area, checking is performed according to the above steps.
By applying the embodiment shown in fig. 3, according to a preset check algorithm, a first check value of all metadata stored in the metadata storage area is obtained, whether the first check value is consistent with a first preset check value is judged, if so, the data to be stored can be stored in a target storage location in the metadata storage area, after the data to be stored is stored, a second check value of all metadata stored in the metadata storage area is obtained, and the first preset check value is updated according to the second check value. Therefore, whether the metadata in the metadata storage area is stored wrongly can be found in time, and the reliability of the storage system is improved.
Referring to fig. 4, fig. 4 is a schematic flowchart of another data storage method provided by the embodiment of the present invention, which is applied to a storage device, where the storage device may include: a metadata storage area, wherein the metadata storage area may include: the storage system comprises a main metadata storage area and a standby metadata storage area, wherein the standby metadata storage area is a data backup area of the main metadata storage area, and a target storage position is in the main metadata storage area.
In this case, step S1041 may include:
s1041 d: obtaining third check values of all metadata stored in the main metadata storage area according to a preset check algorithm, judging whether the third check values are consistent with a second preset check value, and if the third check values are inconsistent with the second preset check value, executing step S1041 e;
the preset check algorithm may be a CRC check algorithm, and may also be other check algorithms, which is not limited in this invention. In addition, the second preset check value may be obtained according to all metadata stored in the primary metadata storage area after the storage device stores the data to be stored in the primary metadata storage area last time; or after the storage device stores the data to be stored in the primary main metadata storage area, the data to be stored is backed up to the standby metadata storage area and then is obtained according to all metadata stored in the standby metadata storage area; it may also be given to the user based on all metadata already stored in the main metadata storage area.
In a specific implementation manner, if the third check value is consistent with the second preset check value, the data to be stored may be stored in the target storage location in the main metadata storage area, and after being stored in the main metadata storage area, the data to be stored may also be backed up from the main metadata storage area to the standby metadata storage area.
Of course, if the third check value is consistent with the second preset check value, the data to be stored may also be directly stored in the standby metadata storage area while the data to be stored is stored in the target storage location in the main metadata storage area.
S1041 e: obtaining a fourth check value of the metadata stored in the metadata storage area according to the preset check algorithm, and judging whether the fourth check value is consistent with the second preset check value, if so, executing step S1041 f;
s1041 f: it is determined that all metadata stored in the metadata storage area is complete.
In a specific implementation manner of the present invention, before determining that all metadata stored in the metadata storage area is complete, the method may further include:
and updating the metadata stored in the main metadata storage area according to all the metadata stored in the standby metadata storage area.
Specifically, updating the metadata stored in the main metadata storage area according to all the metadata stored in the standby metadata storage area may be: the metadata with errors in the main metadata storage area and the correct metadata corresponding to the metadata with errors in the standby metadata storage area are obtained, and the metadata with errors is updated by the correct metadata, so that the metadata stored in the main metadata storage area can be quickly updated.
Specifically, the updating of the metadata stored in the main metadata storage area according to all the metadata stored in the standby metadata storage area may further include: all the metadata stored in the main metadata storage area is updated by all the metadata stored in the standby metadata storage area, so that the metadata stored in the main metadata storage area can be accurately updated.
In practical application, the backup metadata storage area is a data backup area of the main metadata storage area, and under normal conditions, the metadata in the main metadata storage area is the same as the metadata in the backup metadata storage area. When the metadata in the main metadata storage area is incomplete and has errors, if the metadata in the standby metadata storage area is complete, all the metadata stored in the main metadata storage area can be updated according to all the metadata stored in the standby metadata storage area, so that the accuracy and the completeness of the metadata in the metadata storage area are ensured, and the error rate of operating the metadata is reduced.
In a specific implementation manner, the data storage method may further include:
and under the condition that the fourth check value is inconsistent with the second preset check value, initializing the metadata in the main metadata storage area and the metadata in the standby metadata storage area, and judging that all the metadata stored in the metadata storage area are complete.
Specifically, when the fourth check value is inconsistent with the second preset check value, it indicates that both the metadata in the main metadata storage area and the metadata in the standby metadata storage area are incomplete, and the erroneous metadata in the main metadata storage area cannot be recovered by the metadata in the standby metadata storage area, so that the metadata in the main metadata storage area and the metadata in the standby metadata storage area need to be initialized to ensure that the subsequently stored metadata is correct, and thus the storage system is prevented from being crashed.
In addition, in the data storage method, after storing the data to be stored in the target storage location, the method may further include:
s108: acquiring fifth check values of all metadata stored in the main metadata storage area according to the preset check algorithm;
s109: and updating the second preset check value according to the fifth check value.
In practical applications, after the data to be stored is stored in the main metadata storage area, the metadata in the main metadata storage area changes, and in order to ensure correct verification, the second preset verification value needs to be updated. Assuming that the second preset check value before the data to be stored is 145, after all the metadata stored in the metadata storage area are judged to be complete, the data to be stored is stored in the main metadata storage area, and at this time, the fifth check value is obtained as 150, and the second preset check value is updated to 150.
Of course, the second preset check value may also be updated according to all metadata already stored in the metadata storage area. Specifically, after judging that all metadata stored in the metadata storage area are complete, storing data to be stored into the main metadata storage area, and after backing up the data to be stored to the state that all metadata stored in the metadata storage area are complete, storing the data to be stored into the main metadata storage area, at this time, a sixth check value of all metadata stored in the metadata storage area may be obtained according to a preset check algorithm, and a second preset check value may be updated according to the sixth check value.
It should be noted that, when data is not stored in the primary metadata storage area, the data to be stored may be directly stored in the target storage location in the primary metadata storage area, a fifth check value is obtained after storage, and the fifth check value is used as a second preset check value, and when data to be stored needs to be stored in the primary metadata storage area, checking is performed according to the above steps.
By applying the embodiment shown in fig. 4, after the metadata stored in the main metadata storage area is verified, the metadata stored in the backup main metadata storage area is verified, and if the metadata stored in the backup main metadata storage area is verified successfully, it can be determined that all the metadata stored in the metadata storage area is complete.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a data storage apparatus according to an embodiment of the present invention, which is applied to a storage device, where the storage device may include: a metadata storage area, the apparatus may include: a request receiving module 501, a storage position judging module 502, a data judging module 503, a data storage module 504 and a data storage rejecting module 505;
the request receiving module 501 is configured to receive a data storage request, where the data storage request includes data to be stored;
a storage location determining module 502, configured to determine a target storage location of data to be stored, determine whether the target storage location is in a metadata storage area, and trigger the data determining module if the target storage location is in the metadata storage area;
the data judgment module 503 is configured to judge whether the data to be stored is metadata, if so, trigger the data storage module 504, and if not, trigger the data storage rejection module 505;
a data storage module 504, configured to store data to be stored to a target storage location when a preset storage condition is met;
and a data storage rejection module 505, configured to reject to store the data to be stored in the target storage location.
In a specific implementation manner, the data storage request may further include: an equipment interface identifier;
the data determining module 503 may include: a device interface decision sub-module and a data decision sub-module (not shown in FIG. 5);
the device interface judgment submodule is used for judging whether the device interface identifier corresponds to a preset device interface or not, and if so, triggering the data judgment submodule, wherein the preset device interface is used for transmitting metadata;
and the data judgment submodule is used for judging the data to be stored as metadata.
By applying the embodiment shown in fig. 5, after receiving a data storage request, the storage device determines a target storage location of data to be stored, and determines whether the target storage location is in the metadata storage area, if so, determines whether the data to be stored is metadata, if so, stores the data to be stored to the target storage location under the condition that a preset storage condition is met, and if not, refuses to store the data to be stored, thereby avoiding the problem that real data covers the metadata, ensuring that the storage device can correctly read the metadata, and improving the reliability of the storage system.
Referring to fig. 6, fig. 6 is a schematic structural diagram of another data storage apparatus provided in the embodiment of the present invention, which is applied to a storage device, where the storage device may include: a metadata storage area, in which a data storage module 504 may include: a complete data judgment sub-module 5041 and a data storage sub-module 5042;
the complete data judgment sub-module 5041 is configured to judge whether all metadata stored in the metadata storage area are complete, and if yes, trigger the data storage sub-module 5042;
and the data storage submodule 5042 is used for storing the data to be stored in the target storage position.
By applying the embodiment shown in fig. 6, it is determined whether all metadata stored in the metadata storage area are complete, and the data to be stored is stored in the target storage location only when all metadata stored in the metadata storage area are complete. Therefore, the correctness and the integrity of the metadata in the metadata storage area are ensured, and the problem of storage system crash caused by the incomplete metadata in the metadata storage area is avoided.
Referring to fig. 7, fig. 7 is a schematic structural diagram of another data storage apparatus provided in the embodiment of the present invention, which is applied to a storage device, where the storage device may include: the metadata storage area, in the apparatus, the complete data determination sub-module 5041 may include: a check value obtaining unit 5041a, a first check value judging unit 5041b, and a first complete data judging unit 5041 c;
the verification value obtaining unit 5041a is configured to obtain, according to a preset verification algorithm, first verification values of all metadata stored in the metadata storage area;
a first check value determining unit 5041b, configured to determine whether the first check value is consistent with a first preset check value, and if so, trigger the complete data determining unit 5041 c;
a complete data determination unit 5041c for determining that all metadata already stored in the metadata storage area is complete;
in this case, the data storage device may further include: a first check value obtaining module 506 and a first check value updating module 507;
the first check value obtaining module 506 is configured to obtain, according to a preset check algorithm, second check values of all metadata stored in the metadata storage area;
the first check value updating module 507 is configured to update the first preset check value according to the second check value.
By applying the embodiment shown in fig. 7, according to a preset check algorithm, the first check value of all metadata stored in the metadata storage area is obtained, whether the first check value is consistent with the first preset check value is judged, if so, the data to be stored can be stored in the target storage location in the metadata storage area, after the data to be stored is stored, the second check value of all metadata stored in the metadata storage area is obtained, and the first preset check value is updated according to the second check value. Therefore, whether the metadata in the metadata storage area is stored wrongly can be found in time, and the reliability of the storage system is improved.
Referring to fig. 8, fig. 8 is a schematic structural diagram of another data storage apparatus according to an embodiment of the present invention, which is applied to a storage device, where the storage device may include: a metadata storage area, wherein the metadata storage area may include: the system comprises a main metadata storage area and a standby metadata storage area, wherein the standby metadata storage area is a data backup area of the main metadata storage area, and a target storage position is in the main metadata storage area;
the complete data determination sub-module 5041 includes: a second check value judgment unit 5041d, a third check value judgment unit 5041e, and a second complete data determination unit 5041 f;
the second check value judging unit 5041d is configured to obtain third check values of all metadata stored in the main metadata storage area according to a preset check algorithm, judge whether the third check values are consistent with a second preset check value, and trigger the third check value judging unit 5041e if the third check values are inconsistent with the second preset check value;
a third check value judging unit 5041e, configured to obtain a fourth check value of the metadata stored in the metadata storage area according to a preset check algorithm, judge whether the fourth check value is consistent with a second preset check value, and trigger a second complete data judging unit 5041f if the fourth check value is consistent with the second preset check value;
a second complete data determination unit 5041f, configured to determine that all metadata already stored in the metadata storage area is complete.
In this case, the data storage device may further include: a second check value obtaining module 508 and a second check value updating module 509;
the second check value obtaining module 508 is configured to obtain a fifth check value of all metadata stored in the main metadata storage area according to a preset check algorithm;
a second check value updating module 509, configured to update the second preset check value according to the fifth check value.
In a specific implementation manner, the data storage device may further include: a metadata update module (not shown in figure 8),
and the main metadata storage area is used for updating the metadata stored in the main metadata storage area according to all the metadata stored in the standby metadata storage area.
In a specific implementation manner, the data storage device may further include: a metadata initialization module (not shown in figure 8),
and the method is used for initializing the metadata in the main metadata storage area and the metadata in the standby metadata storage area under the condition that the fourth check value is inconsistent with the second preset check value, and judging that all the metadata stored in the metadata storage area are complete.
By applying the embodiment shown in fig. 8, after the metadata stored in the main metadata storage area is verified, the metadata stored in the backup main metadata storage area is verified, and if the metadata stored in the backup main metadata storage area is verified successfully, it can be determined that all the metadata stored in the metadata storage area is complete.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Those skilled in the art will appreciate that all or part of the steps in the above method embodiments may be implemented by a program to instruct relevant hardware to perform the steps, and the program may be stored in a computer-readable storage medium, which is referred to herein as a storage medium, such as: ROM/RAM, magnetic disk, optical disk, etc.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (12)
1. A data storage method is applied to a storage device, and is characterized in that the storage device comprises: a metadata storage area, the method comprising the steps of:
receiving a data storage request, wherein the data storage request comprises data to be stored and an equipment interface identifier;
determining a target storage position of the data to be stored, and judging whether the target storage position is in the metadata storage area;
if the device interface identifier is in the metadata storage area, judging whether the device interface identifier corresponds to a preset device interface, wherein the preset device interface is used for transmitting metadata;
if so, judging that the data to be stored is metadata, and storing the data to be stored to the target storage position under the condition of meeting a preset storage condition;
if not, judging that the data to be stored is not metadata, and refusing to store the data to be stored to the target storage position.
2. The method according to claim 1, wherein the storing the data to be stored to the target storage location if a preset storage condition is met comprises:
judging whether all metadata stored in the metadata storage area are complete or not;
if so, storing the data to be stored to the target storage position.
3. The method of claim 2, wherein determining whether all metadata stored in the metadata storage area is complete comprises:
according to a preset verification algorithm, obtaining first verification values of all metadata stored in the metadata storage area;
judging whether the first check value is consistent with a first preset check value or not;
if yes, judging that all metadata stored in the metadata storage area are complete;
after the storing the data to be stored to the target storage location, further comprising:
according to the preset verification algorithm, second verification values of all metadata stored in the metadata storage area are obtained;
and updating the first preset check value according to the second check value.
4. The method of claim 2, wherein the metadata storage area comprises: the storage system comprises a main metadata storage area and a standby metadata storage area, wherein the standby metadata storage area is a data backup area of the main metadata storage area, and the target storage position is in the main metadata storage area;
the determining whether all metadata stored in the metadata storage area is complete includes:
according to a preset check algorithm, obtaining third check values of all metadata stored in the main metadata storage area, and judging whether the third check values are consistent with a second preset check value or not;
if the third check value is inconsistent with the second preset check value, obtaining a fourth check value of the metadata stored in the metadata storage area according to the preset check algorithm, and judging whether the fourth check value is consistent with the second preset check value;
if yes, judging that all metadata stored in the metadata storage area are complete;
after the storing the data to be stored to the target storage location, further comprising:
acquiring fifth check values of all metadata stored in the main metadata storage area according to the preset check algorithm;
and updating the second preset check value according to the fifth check value.
5. The method according to claim 4, further comprising, after determining that the fourth check value is consistent with the second preset check value:
and updating the metadata stored in the main metadata storage area according to all the metadata stored in the standby metadata storage area.
6. The method of claim 4, further comprising:
and under the condition that the fourth check value is inconsistent with the second preset check value, initializing the metadata in the main metadata storage area and the metadata in the standby metadata storage area, and judging that all the metadata stored in the metadata storage area are complete.
7. A data storage apparatus applied to a storage device, wherein the storage device comprises: a metadata storage area, the apparatus comprising: the device comprises a request receiving module, a storage position judging module, a data storage module and a data storage rejecting module;
the request receiving module is used for receiving a data storage request, wherein the data storage request comprises data to be stored and an equipment interface identifier;
the storage position judging module is used for determining a target storage position of the data to be stored, judging whether the target storage position is in the metadata storage area, and if the target storage position is in the metadata storage area, triggering the data judging module;
the data judging module is used for judging whether the equipment interface identifier corresponds to a preset equipment interface, if so, judging that the data to be stored is metadata, and triggering the data storage module, and if not, judging that the data to be stored is not the metadata, and triggering the data storage rejection module; the preset device interface is used for transmitting metadata;
the data storage module is used for storing the data to be stored to the target storage position under the condition that a preset storage condition is met;
the data storage rejection module is used for rejecting to store the data to be stored to the target storage position.
8. The apparatus of claim 7, wherein the data storage module comprises: the complete data judgment submodule and the data storage submodule;
the complete data judgment submodule is used for judging whether all metadata stored in the metadata storage area are complete or not, and if so, the data storage submodule is triggered;
and the data storage submodule is used for storing the data to be stored to the target storage position.
9. The apparatus of claim 8, wherein the complete data determination submodule comprises: the device comprises a check value obtaining unit, a first check value judging unit and a first complete data judging unit;
the verification value obtaining unit is configured to obtain, according to a preset verification algorithm, first verification values of all metadata stored in the metadata storage area;
the first check value judging unit is used for judging whether the first check value is consistent with a first preset check value or not, and if so, the complete data judging unit is triggered;
the complete data judging unit is used for judging that all metadata stored in the metadata storage area are complete;
the device further comprises: the device comprises a first check value obtaining module and a first check value updating module;
the first check value obtaining module is configured to obtain, according to the preset check algorithm, second check values of all metadata stored in the metadata storage area;
and the first check value updating module is used for updating the first preset check value according to the second check value.
10. The apparatus of claim 8, wherein the metadata storage area comprises: the storage system comprises a main metadata storage area and a standby metadata storage area, wherein the standby metadata storage area is a data backup area of the main metadata storage area, and the target storage position is in the main metadata storage area;
the complete data judgment submodule comprises: the device comprises a second check value judging unit, a third check value judging unit and a second complete data judging unit;
the second check value judging unit is configured to obtain third check values of all metadata stored in the primary metadata storage area according to a preset check algorithm, judge whether the third check values are consistent with a second preset check value, and trigger the third check value judging unit if the third check values are inconsistent with the second preset check value;
the third check value judging unit is configured to obtain a fourth check value of the metadata stored in the metadata storage area according to the preset check algorithm, judge whether the fourth check value is consistent with the second preset check value, and trigger the second complete data judging unit if the fourth check value is consistent with the second preset check value;
the second complete data determination unit is configured to determine that all metadata stored in the metadata storage area are complete;
the device further comprises: the second check value obtaining module and the second check value updating module;
the second check value obtaining module is configured to obtain a fifth check value of all metadata stored in the main metadata storage area according to the preset check algorithm;
and the second check value updating module is used for updating the second preset check value according to the fifth check value.
11. The apparatus of claim 10, further comprising: a metadata update module for updating the metadata of the received data,
and the main metadata storage area is used for updating the metadata stored in the main metadata storage area according to all the metadata stored in the standby metadata storage area.
12. The apparatus of claim 10, further comprising: a metadata initialization module for initializing the metadata of the mobile terminal,
and the metadata management module is configured to initialize the metadata in the main metadata storage area and the metadata in the standby metadata storage area under the condition that the fourth check value is inconsistent with the second preset check value, and determine that all the metadata stored in the metadata storage area are complete.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610120859.5A CN107153508B (en) | 2016-03-03 | 2016-03-03 | Data storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610120859.5A CN107153508B (en) | 2016-03-03 | 2016-03-03 | Data storage method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107153508A CN107153508A (en) | 2017-09-12 |
| CN107153508B true CN107153508B (en) | 2020-07-24 |
Family
ID=59792437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610120859.5A Active CN107153508B (en) | 2016-03-03 | 2016-03-03 | Data storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107153508B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108052413A (en) * | 2017-11-21 | 2018-05-18 | 深信服科技股份有限公司 | Data processing method, device and storage medium based on multiple authentication |
| CN110795386B (en) * | 2018-07-31 | 2022-07-01 | 杭州海康威视***技术有限公司 | Data writing method and server |
| CN111858468B (en) * | 2020-07-22 | 2023-01-10 | 苏州浪潮智能科技有限公司 | Method, system, terminal and storage medium for verifying metadata of distributed file system |
| CN112667705A (en) * | 2020-12-23 | 2021-04-16 | 大秦铁路股份有限公司 | Train tail wind pressure acquisition method, device and medium of automatic locomotive driving system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002203070A (en) * | 2000-10-17 | 2002-07-19 | Hitachi Ltd | Metadata distribution method |
| CN102932621A (en) * | 2011-08-08 | 2013-02-13 | 杭州海康威视数字技术股份有限公司 | Method and device for storing data |
| CN103399823A (en) * | 2011-12-31 | 2013-11-20 | 华为数字技术（成都）有限公司 | Method, equipment and system for storing service data |
| CN104461404A (en) * | 2014-12-29 | 2015-03-25 | 成都致云科技有限公司 | Metadata storage method, device and system |
| US9244958B1 (en) * | 2013-06-13 | 2016-01-26 | Amazon Technologies, Inc. | Detecting and reconciling system resource metadata anomolies in a distributed storage system |
| CN105373449A (en) * | 2015-11-18 | 2016-03-02 | 华为技术有限公司 | Distributed storage metadata recovery method, apparatus and system |
-
2016
- 2016-03-03 CN CN201610120859.5A patent/CN107153508B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002203070A (en) * | 2000-10-17 | 2002-07-19 | Hitachi Ltd | Metadata distribution method |
| CN102932621A (en) * | 2011-08-08 | 2013-02-13 | 杭州海康威视数字技术股份有限公司 | Method and device for storing data |
| CN103399823A (en) * | 2011-12-31 | 2013-11-20 | 华为数字技术（成都）有限公司 | Method, equipment and system for storing service data |
| US9244958B1 (en) * | 2013-06-13 | 2016-01-26 | Amazon Technologies, Inc. | Detecting and reconciling system resource metadata anomolies in a distributed storage system |
| CN104461404A (en) * | 2014-12-29 | 2015-03-25 | 成都致云科技有限公司 | Metadata storage method, device and system |
| CN105373449A (en) * | 2015-11-18 | 2016-03-02 | 华为技术有限公司 | Distributed storage metadata recovery method, apparatus and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107153508A (en) | 2017-09-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107153508B (en) | Data storage method and device | |
| CN105260639A (en) | Face recognition system data update method and device | |
| CN111680104B (en) | Data synchronization method, device, computer equipment and readable storage medium | |
| CN111178147A (en) | Screen crushing and grading method, device, equipment and computer readable storage medium | |
| CN104850971A (en) | Correlation method between order information and logistics information, and device thereof | |
| CN111540020A (en) | Method and device for determining target behavior, storage medium and electronic device | |
| CN113204383B (en) | Method, device and system for storing BMC configuration parameters and storage medium | |
| CN112637888B (en) | Coverage hole area identification method, device, equipment and readable storage medium | |
| CN105391629A (en) | Resource backup method and device | |
| CN113806138A (en) | Backup recovery detection method and device for database, electronic equipment and storage medium | |
| CN110929704A (en) | License plate number matching method and device, storage medium and electronic device | |
| CN110674153B (en) | Data consistency detection method and device and electronic equipment | |
| CN107633173B (en) | File processing method and device | |
| CN108259214B (en) | Configuration command management method, device and machine-readable storage medium | |
| CN107548088B (en) | Mobile equipment identity identification method and service server | |
| CN113238719B (en) | Method, device and equipment for cleaning residual path of host end | |
| CN111949363A (en) | Service access management method, computer equipment, storage medium and system | |
| CN114650218B (en) | Data acquisition method, device, system and storage medium | |
| CN114238990A (en) | Distributed vulnerability scanning method and k8s distributed system | |
| CN110333968B (en) | Data management method and device applied to database and computer equipment | |
| CN109560964B (en) | Equipment compliance checking method and device | |
| CN112631853A (en) | Data monitoring method, task monitoring method, system and computer readable medium | |
| CN111274311A (en) | Data synchronization method and device for cross-machine-room database | |
| CN115460271B (en) | Network control method and device based on edge calculation and storage medium | |
| CN109542643B (en) | Method and device for recovering messages in OpenStack system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |