CN109144409B - Data processing method and device, storage medium and data system - Google Patents
Data processing method and device, storage medium and data system Download PDFInfo
- Publication number
- CN109144409B CN109144409B CN201810759961.9A CN201810759961A CN109144409B CN 109144409 B CN109144409 B CN 109144409B CN 201810759961 A CN201810759961 A CN 201810759961A CN 109144409 B CN109144409 B CN 109144409B
- Authority
- CN
- China
- Prior art keywords
- data
- offset address
- background
- disk
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
Abstract
The application discloses a data processing method and device, a storage medium and a data system, and relates to the technical field of data processing. The data processing method comprises the following steps: sending a read request to a target disk in a disk array, wherein the read request is used for requesting to read first data; receiving a read response sent by a target disk, wherein the read response comprises second data, and the second data comprises a second data body and a second background logic offset address; comparing the second background logic offset address with a first background logic offset address of pre-recorded first data; when the second background logical offset address is inconsistent with the first background logical offset address, determining that the background data of the first data references an error. The problem that data errors occur due to the fact that reading and writing offset of a background cannot be sensed at present is effectively solved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method and apparatus, a storage medium, and a data system.
Background
Current data systems typically include one or more hosts (host), and disk arrays coupled to the hosts. The disk array includes an array controller (also called a storage array) connected to each host, and one or more disks connected to the array controller.
In order to guarantee the integrity of data in a disk array, Data Integrity Field (DIF) technology is currently proposed. The DIF technique generally conforms to the T10 standard, and the T10 standard specifies that a DIF area of 8 bytes, also called Protection Information (PI) area, is additionally added after every 512-byte data storage area. The PI region may be filled with a Cyclic Redundancy Check (CRC) value and a Logical Block Address (LBA). When the array controller in the array disk receives a read command sent by the host to perform a data reading operation, the array controller can determine whether the data itself is correct through the CRC value, and can also determine whether the data is correctly referred through the LBA.
When the array controller performs read-write operations on the disk array background, background read-write offsets are easily generated, which causes reference errors of subsequent background data, whereas the current T10 standard only checks data offsets generated by read-write operations of the host, and cannot sense the background read-write offsets (i.e., cannot sense the reference errors of the background data).
Disclosure of Invention
The embodiment of the application provides a data processing method and device, a storage medium and a data system, and can solve the problem that the conventional array controller cannot sense the read-write offset of a background. The technical scheme is as follows:
in a first aspect, a data processing method is provided, where the method is applied to an array controller in a disk array, and the method includes:
sending a read request to a target disk in the disk array, wherein the read request is used for requesting to read first data;
receiving a read response sent by the target disk, wherein the read response comprises second data, the second data comprises a second data body and a second background logical offset address, and the second background logical offset address is determined by the array controller based on the second data body before the second data body is written into the target disk;
comparing the second background logical offset address with a first background logical offset address of the first data recorded in advance, the first background logical offset address being determined based on a first data ontology of the first data;
when the second background logical offset address and the first background logical offset address are inconsistent, determining that background data reference of the first data is erroneous.
In the application, when the array controller is performing a reading operation of the disk array background, the array controller may first obtain second data, where the second data includes a second data body and a second background logical offset address, compare the second background logical offset address with a first background logical offset address of pre-recorded first data, and determine that the background data of the first data is in error when the second background logical offset address is inconsistent with the first background logical offset address, thereby effectively solving a problem that the data is in error due to the fact that the read-write offset of the background cannot be sensed at present.
Optionally, before sending the read request to the target disk in the disk array, the method further includes:
determining the first background logical offset address based on a first data ontology to be stored;
generating a write request, wherein the write request is used for requesting to write the first data, the write request comprises the first data, and the first data comprises the first data body and the first background logic offset address;
and sending the write request to the target disk.
In this application, when the array controller performs a write operation in the disk array background, the array controller may determine a first background logical offset address based on a first data body to be stored, where the first data body and the first background logical offset address may constitute first data, and the array controller may send the first data to the target disk, so that the target disk may store the first data. In the application, after the array controller determines the first background logical offset address based on the first data body, the first background logical offset address may be recorded, so that when the first data is subsequently read again, the array controller may compare the recorded first background logical offset address with the background logical offset address carried in the read data based on the recorded first background logical offset address.
Optionally, the array controller stores read-write data in a sector form, where each read-write data includes a data body and protection information PI, where the PI includes a target field, and the target field is used to carry a background logical offset address determined based on the data body. In the present application, the background logical offset address determined based on the data (i.e. the data being read or the data being written) body may be recorded in the target field in the PI corresponding to the data body.
Optionally, the array controller stores read-write data in a sector form, each read-write data includes a data ontology and a PI, each PI includes an indication bit, the indication bit is used to indicate whether the data supports metadata customization, when the data supports metadata customization, the PI further includes a target field, and the target field is used to carry a background logical offset address determined based on the data ontology.
By adding the indication bit in the PI, when the indication bit in the PI indicates that the data supports metadata customization, the background logic offset address corresponding to the data ontology in the data can be recorded in the PI in the data. When the indication bit in the PI does not support metadata self-definition, the array controller does not need to search a target field, and unnecessary data overhead is avoided.
Optionally, after receiving the read response sent by the target disk, the method further includes:
and detecting an indication bit of a PI in the second data, and reading the second background logic offset address in a target field of the PI in the second data when the indication bit of the PI indicates that the second data supports metadata customization.
Optionally, the generating a write request includes:
detecting whether the array controller supports metadata customization of data;
when the array controller supports metadata customization of data, generating the first data, wherein the first data comprises the first data body and a PI, the PI in the first data comprises an indication bit and a target field, the indication bit is used for indicating that the first data supports metadata customization, and the target field is used for carrying the first background logic offset address.
Optionally, the PI includes a checksum GRD field, an application tag APP field, and a reference tag REF field, and the target field is any field in the PI.
Optionally, the background logic offset address is stored in a reserved bit in an APP field of the PI.
Optionally, the background logical offset address is stored in unused data or blank bits in the REF field of the PI.
According to the method, the PI format is adjusted on the basis of the current T10 standard, downward compatibility of the current PI format can be achieved, the array controller can sense read-write offset of a background, and the read-write operation in the following embodiment is the read-write operation of the array controller on the disk array background.
Optionally, the background logical offset address is a disk offset address or an offset address of a data block.
Optionally, the method further includes:
and after determining the background data reference error of the first data, adopting a redundancy technology of a disk array or an error correction code (EC) redundancy technology to repair the data. The error data is effectively prevented from being diffused in the disk array.
In a second aspect, a data processing apparatus is provided, the apparatus being applied to an array controller in a disk array, the apparatus comprising:
a first sending module, configured to send a read request to a target disk in the disk array, where the read request is used to request to read first data;
a receiving module, configured to receive a read response sent by the target disk, where the read response includes second data, where the second data includes a second data entity and a second background logical offset address, and the second background logical offset address is determined by the array controller based on the second data entity before the second data entity is written in the target disk;
a comparison module, configured to compare the second background logical offset address with a first background logical offset address of the first data that is pre-recorded, where the first background logical offset address is determined based on a first data ontology of the first data;
a first determining module, configured to determine a background data reference error of the first data when the second background logical offset address is inconsistent with the first background logical offset address.
Optionally, the apparatus further comprises:
a second determining module, configured to determine the first background logical offset address based on a first data ontology to be stored;
a generating module, configured to generate a write request, where the write request is used to request to write the first data, where the write request includes the first data, and the first data includes the first data body and the first background logical offset address;
and the second sending module is used for sending the write request to the target disk.
Optionally, the array controller stores read-write data in a sector form, where each read-write data includes a data body and protection information PI, where the PI includes a target field, and the target field is used to carry a background logical offset address determined based on the data body.
Optionally, the array controller stores read-write data in a sector form, each read-write data includes a data ontology and a PI, each PI includes an indication bit, the indication bit is used to indicate whether the data supports metadata customization, when the data supports metadata customization, the PI further includes a target field, and the target field is used to carry a background logical offset address determined based on the data ontology.
Optionally, the apparatus further comprises:
and the detection module is used for detecting an indication bit of the PI in the second data, and reading the second background logic offset address in a target field of the PI in the second data when the indication bit of the PI indicates that the second data supports metadata self-definition.
Optionally, the generating module is configured to:
detecting whether the array controller supports metadata customization of data;
when the array controller supports metadata customization of data, generating the first data, wherein the first data comprises the first data body and a PI, the PI in the first data comprises an indication bit and a target field, the indication bit is used for indicating that the first data supports metadata customization, and the target field is used for carrying the first background logic offset address.
Optionally, the PI includes a checksum GRD field, an application tag APP field, and a reference tag REF field, and the target field is any field in the PI.
Optionally, the background logic offset address is stored in a reserved bit in an APP field of the PI.
Optionally, the background logical offset address is stored in unused data or blank bits in the REF field of the PI.
Optionally, the background logical offset address is a disk offset address or an offset address of a data block.
Optionally, the apparatus further comprises:
and the repair module is used for repairing the data by adopting a redundancy technology of a disk array or an error correction code (EC) redundancy technology after determining the background data reference error of the first data.
In a third aspect, a data system is provided, comprising:
one or more hosts and a disk array connected to the hosts, wherein the disk array comprises an array controller and at least one disk, and the array controller comprises the data processing device according to any one of the second aspect.
In a fourth aspect, there is provided a computer readable storage medium having stored thereon instructions which, when executed by a processor in an array controller, implement the data processing method of any of the first aspects.
The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:
when the array controller performs a write operation in the disk array background, first data may be written to the target disk, where the first data includes a first data body and a first background logical offset address. When the array controller is used for reading the background of the disk array, the array controller can firstly acquire second data, the second data comprises a second data body and a second background logic offset address, then the second background logic offset address is compared with a first background logic offset address of pre-recorded first data, and when the second background logic offset address is inconsistent with the first background logic offset address, background data application errors of the first data are determined, so that the problem that data errors occur due to the fact that reading and writing offsets of the background cannot be sensed at present is effectively solved. In addition, the first data with errors can be applied to the background data for repairing, and the error data can be effectively prevented from being diffused in the disk array.
Drawings
FIG. 1 is a schematic diagram of a data system architecture;
FIG. 2 is a schematic diagram of a data storage area and a PI area in a data;
FIG. 3 is a diagram of a data format in the PI area;
fig. 4 is a flowchart of a data processing method provided in an embodiment of the present application;
fig. 5 is a schematic diagram of a target field in a PI area according to an embodiment of the present disclosure;
fig. 6 is a block diagram of a data processing apparatus according to an embodiment of the present application;
FIG. 7 is a block diagram of another data processing apparatus provided in an embodiment of the present application;
FIG. 8 is a block diagram of yet another data processing apparatus provided by an embodiment of the present application;
fig. 9 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a data system, which may include: one or more hosts 200, and a disk array 100 coupled to the hosts 200. The disk array 100 may include an array controller 101 and one or more disks 102 that establish a connection with the array controller 101.
In a data system, the array controller 101 stores read and write data in sectors, each of which may include 512 bytes of data. According to the T10 standard, read and write data can be divided into a data storage area and a PI area, for example, as shown in fig. 2, the data storage area can store 512 bytes of data, that is, one sector of data, and the PI area can store 8 bytes of PI, through which the integrity of the data in the disk array can be ensured.
The PI is mainly used to prevent and detect static data corruption including data corruption due to hardware failures and software bugs (bugs) on data channels, and data errors that are undetectable and correctable on a disk. As shown in fig. 3, fig. 3 is a schematic diagram of a data format in a PI region, where the first two bytes are a checksum (Guard, GRD) field, and a CRC value is filled in the GRD field for protecting the integrity of data itself; then 2 bytes of application tag (APP) field, where user-defined and application-related information is usually stored; and finally a 4-byte Reference Tag (REF) field, where LBA is used to protect the correctness of the data storage location.
However, when the array controller 200 performs a read/write operation in the background (the background is a relative concept and refers to an area that cannot be sensed by a host in a data system, for example, inside the disk array 100) of the disk array 100, for example, when the array controller 200 performs an operation such as a rewrite operation, an overwrite operation, a lower write operation, or a destage operation, a read/write offset in the background is likely to occur. Since the data (e.g., LBA) in the Ref field is generated by the host 200, and the array controller 101 does not have the right to rewrite the existing data in the Ref field in the T10 standard, the current T10 standard only checks the data offset generated by the read/write operation of the host 100, and cannot sense the read/write offset in the background.
In the data processing method provided by the embodiment of the application, the read-write request carries the background logic offset address determined based on the read-write data body to check whether the data has background reference errors, so that the read-write offset of the background is sensed. The read-write request can be in various formats, and only the background logic offset address corresponding to the read-write data can be effectively carried in the read-write request.
For example, in the embodiment of the present application, the array controller stores read and write data in the form of sectors, and each read and write data (i.e. read data or write data) includes a data body and a PI, where the data body may be recorded in a data storage area as shown in fig. 2, and the PI may be recorded in a PI area as shown in fig. 2.
In this embodiment, the PI may include a target field, where the target field is used to carry a background logical offset address determined based on a data (i.e., read data or write data) body. Optionally, the background logical offset address may be a disk offset address or an offset address of a data block. In the embodiment of the application, when the array controller performs the read-write operation of the background, the array controller may determine whether the background has the read-write offset problem based on the background logical offset address carried in the target field.
Please refer to the PI in T10, the PI provided in the embodiment of the present application may also include a GRD field, an APP field, and a REF field, and the format of the PI may refer to fig. 3. Wherein, the target field may be any one of a GRD field, an APP field, and a REF field.
For example, the target field is an APP field, where reserved bits are set, and the background logical offset address of the data may be stored in the reserved bits. As another example, the target field is a REF field, and the background logical offset address of the data may be stored in unused data (e.g., compressed data or metadata) or blank bits of the REF field.
Thus, the embodiment of the application adjusts the PI format on the basis of the current T10 standard, not only can the downward compatibility of the current PI format be realized, but also the array controller can sense the read-write offset of the background, and the read-write operation in the following embodiments is the read-write operation of the array controller on the disk array background.
In the embodiment of the present application, taking an example that a background logic offset address is carried in a PI, the following embodiment schematically illustrates a data processing method in the present application:
referring to fig. 4, fig. 4 is a flowchart of a data processing method provided in an embodiment of the present application, where the method is applied to an array controller in a disk array, and the method includes:
In this embodiment of the present application, when a first data entity (i.e., data to be written) needs to be written into a target disk, a target disk to which the first data entity needs to be written and a target address that needs to be written in the target disk need to be determined first according to the first data entity, where the target address includes a physical address and a first background logical offset address, and the array controller may subsequently write the first data entity into the target disk according to the target address.
After the array controller determines the first background logical offset address based on the first data ontology, the array controller may generate a write request requesting to write the first data, the write request including the first data composed of the first data ontology and the first background logical offset address. In the first data, as shown in fig. 5, the first data body may be recorded in the data storage area, and the first background logical offset address may be recorded in the PI, and in this embodiment, the first background logical offset address is recorded in a target field in the PI, for example, the target field may be an APP field.
It should be noted that the request further includes the target address, and the target address is used for the target disk to write the first data.
In this embodiment of the present application, after the array controller sends the write request to the target disk, the target disk may write the first data into a position corresponding to the target address in the target disk according to the target address carried in the write request. Optionally, after the first data is written into the target disk, a write response may be generated and sent to the array controller to inform the array controller that the data writing is successful.
The array controller may generate a CRC value corresponding to the first data entity, and write the CRC value to the PI when the write request is generated. Before the target disk writes the first data, the target disk may generate a CRC value based on the first data ontology, and perform integrity check with the CRC value stored in the PI, and if the check passes, it is determined that the first data to be written is complete data, the target disk may write the first data.
And step 404, receiving a write response sent by the target disk.
The write response may include indication information indicating that the first data has been written to the target disk. After the array controller receives the write response sent by the target disk, it may be determined that the writing of the first data is successful.
It should be noted that, the above steps 401 to 404 are described by taking a process of performing a write operation on data to be written as first data as an example, in an alternative implementation manner, the write operation on other data may also be completed through the above steps 401 to 404, and the write process may refer to the write process of the first data.
When the array controller determines a target disk to which the first data body needs to be written and a target address to which the first data body needs to be written in the target disk, the identifier and the target address of the target disk may be recorded, and when the first data needs to be read, the array controller may send a read request to the target disk based on the identifier of the target disk, where the read request is used to request to read the first data, and the read request carries the target address.
And step 406, receiving the read response sent by the target disk.
When the array controller performs data writing operation, the data to be written is integrally written into the disk, where the data to be written includes a data body and a background logical offset address corresponding to the data body, and therefore, the data read from the disk also includes the data body and the corresponding background logical offset address. The second background logical offset address is determined by the array controller based on the second data entity before it is written to the target disk, and the determination process may refer to step 401 above.
After the array controller determines a first background logical offset address based on the first data ontology, the first background logical offset address may be recorded. When the array controller receives a read response sent by the target disk, the array controller may read the second background logical offset address from the target field in the PI of the second data, and read the first background logical offset address recorded in advance. The first background logical offset address is then compared to a second background logical offset address.
And step 408, when the second background logic offset address is consistent with the first background logic offset address, determining that the background data of the first data has no reference error.
When the second background logic offset address is consistent with the first background logic offset address, the background offset of the first data is consistent with that of the second data, no read-write offset occurs, and the data position referred by the read request is correct.
It should be noted that the array controller further needs to perform integrity verification on the second data by using the CRC value, after the verification is passed, it is indicated that the second data carried in the read response is complete data, and when both the verification of the read-write offset and the verification of the integrity of the data are passed, it is indicated that the second data is the same as the first data, the data reading is successful, and the reading operation of the first data is completed.
And 409, when the second background logic offset address is inconsistent with the first background logic offset address, determining that the background data of the first data is referenced wrongly.
The array controller may determine that the first data has a background data reference error when the second background logical offset address is inconsistent with the first background logical offset address. For example, a data write error may occur to the first data; or; when the first data is read, a data read error occurs. Step 410 may be performed after the array controller determines that the background data reference of the first data is erroneous.
In the embodiment of the application, when the background data of the first data is determined to be referenced incorrectly, the redundancy technology or the EC redundancy technology of the disk array can be adopted to perform degradation repair on the first data, so that the error data is effectively prevented from being diffused in the disk array.
In another optional implementation manner, each PI includes an indication bit, where the indication bit is used to indicate whether read-write data (such as the first data or the second data) carrying the PI supports metadata (i.e., data describing data) customization, and when the data supports metadata customization, it is described that a PI in the read-write data of the array controller may define a new parameter, and at this time, the PI further includes a target field, which may be located behind the indication bit, and the target field is the same as the target field in the foregoing embodiment, which is not described herein again in this embodiment. If the data does not support metadata customization, the target field cannot be carried in the PI.
At this time, in an implementation manner, when the step 401 is executed, the step 401 may include the following steps:
step A1, detecting whether the array controller supports metadata customization.
Step B1, when the array controller supports metadata customization, generates first data.
The first data comprises a first data body and a PI, wherein the PI in the first data comprises an indication bit and a target field, the indication bit is used for indicating that the first data supports metadata self-definition, and the target field is used for carrying a first background logic offset address.
Through the above-described steps a1 and B1, a first background logical offset address may be added to the PI in the first data.
In this implementation manner, correspondingly, if the first data is read, after the step 406 is executed, the data processing method further includes the following steps:
step A3, detecting an indication bit of a PI in the second data, and reading a second background logic offset address in a target field of the PI in the second data when the indication bit of the PI indicates that the second data supports metadata customization.
Through the above step a3, a second background logical offset address may be added to the PI in the second data.
In the implementation mode, the indication bit is added in the PI to indicate whether read-write data support metadata customization, and when the read-write data do not support metadata customization, the array controller does not need to search a target field, so that unnecessary data overhead is avoided.
To sum up, in the data processing method provided in the embodiment of the present application, when the array controller performs a write operation in the background of the disk array, first data may be written in the target disk, where the first data includes a first data body and a first background logical offset address. When the array controller is used for reading the background of the disk array, the array controller can firstly acquire second data, the second data comprises a second data body and a second background logic offset address, then the second background logic offset address is compared with a first background logic offset address of pre-recorded first data, and when the second background logic offset address is inconsistent with the first background logic offset address, background data application errors of the first data are determined, so that the problem that data errors occur due to the fact that reading and writing offsets of the background cannot be sensed at present is effectively solved. In addition, the first data with errors can be applied to the background data for repairing, and the error data can be effectively prevented from being diffused in the disk array.
It should be noted that, the order of the steps of the data processing method provided in the embodiment of the present application may be appropriately adjusted, and the steps may also be increased or decreased according to the circumstances, and any method that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application, and therefore, the details are not described again.
Referring to fig. 6, fig. 6 is a block diagram of a data processing apparatus provided in an embodiment of the present application, where the data processing apparatus 600 includes:
the first sending module 601 is configured to send a read request to a target disk in the disk array, where the read request is used to request to read the first data.
A receiving module 602, configured to receive a read response sent by the target disk, where the read response includes second data, and the second data includes a second data entity and a second background logical offset address, and the second background logical offset address is determined by the array controller based on the second data entity before the second data entity is written into the target disk.
The comparing module 603 is configured to compare the second background logical offset address with a first background logical offset address of the pre-recorded first data, where the first background logical offset address is determined based on a first data body of the first data.
The first determining module 604 is configured to determine that the background data reference of the first data is incorrect when the second background logical offset address is inconsistent with the first background logical offset address.
Optionally, referring to fig. 7, fig. 7 is a block diagram of another data processing apparatus provided in the embodiment of the present application, where the data processing apparatus 600 further includes:
a second determining module 605, configured to determine a first background logical offset address based on the first data ontology to be stored.
A generating module 606, configured to generate a write request, where the write request is used to request to write first data, and the write request includes the first data, where the first data includes a first data body and a first background logical offset address.
A second sending module 607, configured to send the write request to the target disk.
Optionally, the array controller stores read-write data in a sector form, each read-write data includes a data body and a PI, the PI includes a target field, and the target field is used to carry a background logic offset address determined based on the data body.
Optionally, the array controller stores read-write data in a sector form, each read-write data includes a data body and a PI, each PI includes an indication bit, the indication bit is used to indicate whether the data supports metadata customization, when the data supports metadata customization, the PI further includes a target field, and the target field is used to carry a background logical offset address determined based on the data body.
Optionally, the data processing apparatus further includes:
and the detection module is used for detecting an indication bit of the PI in the second data, and reading a second background logic offset address in a target field of the PI in the second data when the indication bit of the PI indicates that the second data supports metadata self-definition.
Optionally, the generating module 606 is configured to: detecting whether the array controller supports metadata customization of the data; when the array controller supports metadata customization of data, first data are generated, the first data comprise a first data body and a PI, the PI in the first data comprises an indicating bit and a target field, the indicating bit is used for indicating that the first data support metadata customization, and the target field is used for carrying a first background logic offset address.
Optionally, the PI includes a GRD field, an APP field, and a REF field, and the target field is any one of the PI.
Optionally, the background logical offset address is stored in a reserved bit in the APP field of the PI.
Optionally, the background logical offset address is stored in unused data or blank bits in the REF field of the PI.
Optionally, the background logical offset address is a disk offset address or an offset address of the data block.
Optionally, please refer to fig. 8, fig. 8 is a block diagram of another data processing apparatus provided in the embodiment of the present application, where the data processing apparatus 600 further includes:
and a repair module 608, configured to repair the data by using a redundancy technology of the disk array or an EC redundancy technology after determining that the background data of the first data refers to an error.
To sum up, in the data processing apparatus provided in this embodiment of the present application, when the array controller performs a write operation in the background of the disk array, first data may be written in the target disk, where the first data includes a first data body and a first background logical offset address. When the array controller is used for reading the background of the disk array, the array controller can firstly acquire second data, the second data comprises a second data body and a second background logic offset address, then the second background logic offset address is compared with a first background logic offset address of pre-recorded first data, and when the second background logic offset address is inconsistent with the first background logic offset address, background data application errors of the first data are determined, so that the problem that data errors occur due to the fact that reading and writing offsets of the background cannot be sensed at present is effectively solved. In addition, the first data with errors can be applied to the background data for repairing, and the error data can be effectively prevented from being diffused in the disk array.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Fig. 9 is a schematic structural diagram of another data processing apparatus provided in an embodiment of the present application, and referring to fig. 9, the data processing apparatus 700 may include: processor 710, communication interface 720, and memory 730, communication interface 720 and memory 730 are each coupled to processor 710, illustratively, as shown in fig. 9, communication interface 720 and memory 730 are coupled to processor 710 via bus 740.
The processor 710 may be a Central Processing Unit (CPU), and the processor 710 includes one or more processing cores. The processor 710 executes various functional applications and data processing by executing software programs.
The number of the communication interfaces 720 may be plural, and the communication interfaces 720 are used for the data processing apparatus 700 to communicate with external devices, such as a host computer or a magnetic disk.
Wherein the memory 730 stores computer programs executable on the processor 710, the memory 730 may include but is not limited to: random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM), flash memory, optical memory. The memory 730 is responsible for information storage, for example, the memory 730 is used for storing software programs.
Optionally, the data processing apparatus 700 may further include: an input/output (I/O) interface (not shown in FIG. 9). The I/O interface is coupled to processor 710, communication interface 720, and memory 730. The I/O interface may be, for example, a Universal Serial Bus (USB).
The processor 710 is configured to execute a computer program stored in the memory 730, by which the processor 730 implements the method of steps 401 to 410 as described above in the embodiment shown in fig. 4 by executing the computer program.
An exemplary embodiment of the present application also provides a data system, including: the system comprises one or more hosts and a disk array connected with the hosts, wherein the disk array comprises an array controller and at least one disk. The structure of the data system may refer to the data system shown in fig. 1. The array controller may comprise a data processing device such as that shown in fig. 6, 7, 8 or 9.
The exemplary embodiments of this application also provide a computer readable storage medium having stored thereon instructions that, when executed by a processor in an array controller, implement the method as described above in the embodiment of fig. 4 from step 401 to step 410.
The exemplary embodiments of the present application also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the data processing method provided by the exemplary embodiments of the present application.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (18)
1. A data processing method applied to an array controller in a disk array, the method comprising:
determining a first background logic offset address based on a first data ontology to be stored;
generating a write request, wherein the write request is used for requesting to write first data, the write request comprises the first data, and the first data comprises the first data body and the first background logic offset address;
sending the write request to a target disk in the disk array;
sending a read request to a target disk in the disk array, wherein the read request is used for requesting to read the first data;
receiving a read response sent by the target disk, wherein the read response comprises second data, the second data comprises a second data body and a second background logical offset address, and the second background logical offset address is determined by the array controller based on the second data body before the second data body is written into the target disk;
comparing the second background logic offset address with a first background logic offset address of the first data recorded in advance;
when the second background logical offset address is inconsistent with the first background logical offset address, determining a background data reference error of the first data;
the array controller stores read-write data in a sector form, each read-write data comprises a data body and protection information PI, each PI comprises an indication bit, the indication bit is used for indicating whether the data support metadata customization, when the data support metadata customization, the PI further comprises a target field, and the target field is used for carrying a background logic offset address determined based on the data body.
2. The method of claim 1,
after the receiving the read response sent by the target disk, the method further includes:
and detecting an indication bit of a PI in the second data, and reading the second background logic offset address in a target field of the PI in the second data when the indication bit of the PI indicates that the second data supports metadata customization.
3. The method of claim 1,
the generating a write request includes:
detecting whether the array controller supports metadata customization of data;
when the array controller supports metadata customization of data, generating the first data, wherein the first data comprises the first data body and a PI, the PI in the first data comprises an indication bit and a target field, the indication bit is used for indicating that the first data supports metadata customization, and the target field is used for carrying the first background logic offset address.
4. The method according to any one of claims 1 to 3,
the PI comprises a check sum GRD field, an application label APP field and a reference label REF field, and the target field is any field in the PI.
5. The method of claim 4,
the background logical offset address is stored in reserved bits in the APP field of the PI.
6. The method of claim 4,
the background logical offset address is stored in unused data or blank bits in the REF field of the PI.
7. The method of any of claims 1 to 3, wherein the background logical offset address is a disk offset address or an offset address of a data block.
8. The method of claim 1, further comprising:
and after determining the background data reference error of the first data, adopting a redundancy technology of a disk array or an error correction code (EC) redundancy technology to repair the data.
9. A data processing apparatus, wherein the apparatus is applied to an array controller in a disk array, the apparatus comprising:
the second determining module is used for determining a first background logic offset address based on the first data body to be stored;
a generating module, configured to generate a write request, where the write request is used to request to write first data, where the write request includes the first data, and the first data includes the first data body and the first background logical offset address;
a second sending module, configured to send the write request to a target disk in the disk array;
a first sending module, configured to send a read request to a target disk in the disk array, where the read request is used to request to read first data;
a receiving module, configured to receive a read response sent by the target disk, where the read response includes second data, where the second data includes a second data entity and a second background logical offset address, and the second background logical offset address is determined by the array controller based on the second data entity before the second data entity is written in the target disk;
the comparison module is used for comparing the second background logic offset address with a first background logic offset address of the first data recorded in advance;
a first determining module, configured to determine a background data reference error of the first data when the second background logical offset address is inconsistent with the first background logical offset address;
the array controller stores read-write data in a sector form, each read-write data comprises a data body and protection information PI, each PI comprises an indication bit, the indication bit is used for indicating whether the data support metadata customization, when the data support metadata customization, the PI further comprises a target field, and the target field is used for carrying a background logic offset address determined based on the data body.
10. The apparatus of claim 9, further comprising:
and the detection module is used for detecting an indication bit of the PI in the second data, and reading the second background logic offset address in a target field of the PI in the second data when the indication bit of the PI indicates that the second data supports metadata self-definition.
11. The apparatus of claim 9,
the generation module is configured to:
detecting whether the array controller supports metadata customization of data;
when the array controller supports metadata customization of data, generating the first data, wherein the first data comprises the first data body and a PI, the PI in the first data comprises an indication bit and a target field, the indication bit is used for indicating that the first data supports metadata customization, and the target field is used for carrying the first background logic offset address.
12. The apparatus according to any one of claims 9 to 11,
the PI comprises a check sum GRD field, an application label APP field and a reference label REF field, and the target field is any field in the PI.
13. The apparatus of claim 12,
the background logical offset address is stored in reserved bits in the APP field of the PI.
14. The apparatus of claim 12,
the background logical offset address is stored in unused data or blank bits in the REF field of the PI.
15. The apparatus of any of claims 9 to 11, wherein the background logical offset address is a disk offset address or an offset address of a data block.
16. The apparatus of claim 9, further comprising:
and the repair module is used for repairing the data by adopting a redundancy technology of a disk array or an error correction code (EC) redundancy technology after determining the background data reference error of the first data.
17. A data system, comprising:
one or more hosts, and a disk array connected to the hosts, the disk array comprising an array controller and at least one disk, the array controller comprising the data processing apparatus of any of claims 9 to 16.
18. A computer-readable storage medium having stored thereon instructions which, when executed by a processor in an array controller, carry out a data processing method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810759961.9A CN109144409B (en) | 2018-07-11 | 2018-07-11 | Data processing method and device, storage medium and data system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810759961.9A CN109144409B (en) | 2018-07-11 | 2018-07-11 | Data processing method and device, storage medium and data system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109144409A CN109144409A (en) | 2019-01-04 |
| CN109144409B true CN109144409B (en) | 2020-02-21 |
Family
ID=64800206
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810759961.9A Active CN109144409B (en) | 2018-07-11 | 2018-07-11 | Data processing method and device, storage medium and data system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109144409B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117953936A (en) * | 2022-10-27 | 2024-04-30 | 华为技术有限公司 | Storage device, data searching method and system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021147A (en) * | 2011-09-30 | 2016-10-12 | 英特尔公司 | Storage device for presenting direct access under logical drive model |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6775090B2 (en) * | 2001-03-09 | 2004-08-10 | Seagate Technology Llc | Adapting logical track addresses in relation to head offsets |
| US9104315B2 (en) * | 2005-02-04 | 2015-08-11 | Sandisk Technologies Inc. | Systems and methods for a mass data storage system having a file-based interface to a host and a non-file-based interface to secondary storage |
| CN103413569B (en) * | 2013-07-22 | 2016-03-09 | 华为技术有限公司 | One reads and one writes static RAM |
| CN104346298B (en) * | 2013-08-06 | 2018-12-25 | 北京数码视讯软件技术发展有限公司 | Data processing method, device and smart card based on smart card |
| CN104182701B (en) * | 2014-08-15 | 2017-02-22 | 华为技术有限公司 | Array control unit, array and data processing method |
-
2018
- 2018-07-11 CN CN201810759961.9A patent/CN109144409B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021147A (en) * | 2011-09-30 | 2016-10-12 | 英特尔公司 | Storage device for presenting direct access under logical drive model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109144409A (en) | 2019-01-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8812901B2 (en) | Methods and apparatus for marking writes on a write-protected failed device to avoid reading stale data in a RAID storage system | |
| US7546436B2 (en) | Storage device with SCSI formatting | |
| EP3633517B1 (en) | Storage device and method of operating storage device | |
| US11157357B2 (en) | Operation methods of memory system and host, and computing system | |
| TWI396202B (en) | Error correcting controller and flash memory chip system and error correcting method thereof | |
| US20060120166A1 (en) | Data processing apparatus and method for flash memory | |
| CN109582216B (en) | Data storage device and data processing method of memory device | |
| US8074113B2 (en) | System and method for data protection against power failure during sector remapping | |
| CN103631721A (en) | Method and system for isolating bad blocks in internal storage | |
| US11521701B2 (en) | Memory controller with read error handling | |
| US20180157428A1 (en) | Data protection of flash storage devices during power loss | |
| US8219883B2 (en) | Data accessing method, controller and storage system using the same | |
| WO2017143843A1 (en) | Metadata recovery method and device | |
| US8868517B2 (en) | Scatter gather list for data integrity | |
| CN102890645A (en) | Memory storage device, memory controller and data writing method | |
| CN109144409B (en) | Data processing method and device, storage medium and data system | |
| CN113220221B (en) | Memory controller and data processing method | |
| CN111428280B (en) | SoC (System on chip) security chip key information integrity storage and error self-repairing method | |
| KR101512927B1 (en) | Method and apparatus for managing page of non-volatile memory | |
| JP2004342112A (en) | Device and method for responding to data retention loss in nonvolatile memory unit using error-checking and correction techniques | |
| CN107168650B (en) | Method and device for processing data in memory of BIOS | |
| CN112445432B (en) | Method and device for maintaining redundant VPD (virtual private device) in double-control system | |
| CN111124742B (en) | Flash data verification method, flash controller, storage medium and device | |
| CN112052112A (en) | Bit flipping error detection method and device based on NOR Flash storage and storage medium | |
| US20080155193A1 (en) | Staging method for disk array apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |