CN108073709B - Data recording operation method, device, equipment and storage medium - Google Patents
Data recording operation method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN108073709B CN108073709B CN201711386517.9A CN201711386517A CN108073709B CN 108073709 B CN108073709 B CN 108073709B CN 201711386517 A CN201711386517 A CN 201711386517A CN 108073709 B CN108073709 B CN 108073709B
- Authority
- CN
- China
- Prior art keywords
- data
- field
- record
- original
- fields
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses an operation method, an operation device, equipment and a storage medium for data recording. Wherein, the method comprises the following steps: when the record is organized and analyzed, the extension field corresponding to the column added by the operation of modifying the table structure is distinguished from the original field corresponding to the original column when the table is built. The specific content comprises the following steps: acquiring record organization description of a table where data records are located, recording the number of original columns and added columns in the record organization description acquisition table, and determining physical identification of an extension field according to the number of the original columns and the number of other added columns before the added columns; determining the data offset of the extension field according to the sum of the data lengths of the original field and the sum of the data lengths of other extension fields before the extension field; the field data is sequentially organized into data records based on the physical identification of the extension field and the data offset. According to the technical scheme of the embodiment of the invention, when the columns are added to the table, only the data dictionary is operated, data does not need to be reconstructed, and the efficiency of adding the columns to the table is improved.
Description
Technical Field
The embodiment of the invention relates to the field of database processing, in particular to an operation method, device, equipment and storage medium for data recording.
Background
The most common organizing method for row records of a data list in a database is to organize each field according to the sequence of a table definition column, which is called a sequential organizing method, and also called a sequence organizing method, wherein the organizing method is to respectively collect fields of a fixed length type and fields of a variable length type together according to the sequence of the fixed length type and the variable length type in the fields, and adjust the organizing sequence of the fields in the records. For example, Table TAB is defined as (C1VARCHAR (10), C2INT, C3VARBINARY (10), C4 INT); the order of fields is C1, C2, C3 and C4 according to the sequential organization method, columns C2 and C4in the table TAB are fixed in length, columns C1 and C3 are lengthened, and the order of fields is C2, C4, C1 and C3 according to the sequence organization method.
Compared with the sequential organization method, in all records organized by using the sequence-adjusting organization method, regardless of the value of the variable-length data, the position of the fixed-length type field in the record is fixed, and the data offset in the record can be obtained according to the number of the fixed-length fields during data access, so that the leader field does not need to be traversed from the beginning, and the data access speed in the record can be effectively improved.
However, the sorted organization method is not sufficient when columns are added to the table. The data fields newly added and listed in the record corresponding to the data fields may be between other fields, and subsequent fields cannot be omitted according to the record length, so that for the currently existing data, data must be reconstructed when columns are added, which may cause a large amount of IO to be generated in the operation of adding columns to the data table, and the efficiency is very low.
Disclosure of Invention
Embodiments of the present invention provide an operation method, an apparatus, a device, and a storage medium for data records, so that on the premise of ensuring data access speed in most cases, when adding a column to a table where a data record is located, only a data dictionary needs to be operated, data does not need to be reconstructed, and efficiency of adding columns to the table is improved.
In a first aspect, an embodiment of the present invention provides an operation method for a data list, where the method includes:
acquiring a record organization description corresponding to a data record, wherein the record organization description acquires the number of original columns and added columns in a table where the data record is located, and determines a physical identifier of an extended field in the data record according to the number of the original columns and the number of other added columns before the added columns, and the extended field is a field corresponding to the added column in the table where the data record is located;
acquiring data lengths of an original field and an extended field in the data record, and determining data offset of the extended field according to the sum of the data lengths of the original field and the sum of the data lengths of other extended fields before the extended field, wherein the original field is a field corresponding to an original column in a table where the data record is located;
and sequentially organizing the data of each extension field into the data record according to the physical identification and the data offset of each extension field.
In a second aspect, an embodiment of the present invention provides an apparatus for operating a data list, where the apparatus includes:
the record organization description acquisition module is used for acquiring record organization description corresponding to the data record, acquiring the number of original columns and added columns in a table where the data record is located, and determining the physical identification of an extended field in the data record according to the number of the original columns and the number of other added columns before the added columns, wherein the extended field is a field corresponding to the added column in the table where the data record is located;
the data offset determining module is used for acquiring the data lengths of an original field and an extended field in the data record, and determining the data offset of the extended field according to the sum of the data lengths of the original field and the sum of the data lengths of other extended fields before the extended field, wherein the original field is a field corresponding to an original column in a table where the data record is located;
and the data organization module is used for sequentially organizing the data of each extension field into the data records according to the physical identification and the data offset of each extension field.
In a third aspect, an embodiment of the present invention provides an apparatus, where the apparatus includes:
one or more processors;
storage means for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors implement the method for operating the data record according to any embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements an operation method of data recording according to any embodiment of the present invention.
According to the data record operation method, the device, the equipment and the storage medium, the physical identification of the corresponding extension field in the record organization description is set, all the extension fields are organized sequentially after the original field according to the data offset of the extension field in the data record, the original field still organizes data in an order adjusting mode, the data access speed in most cases is guaranteed, and after columns are added, the existing data record can analyze the value of the extension field into NU LL, so that when the columns are added in the table, only the data dictionary is operated, data do not need to be reconstructed, and the efficiency of adding the columns in the table is improved.
Drawings
Fig. 1 is a flowchart of an operation method of data recording according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for generating a record organization description in the method according to the second embodiment of the present invention;
fig. 3 is a flowchart of an operation method of data recording according to a third embodiment of the present invention;
FIG. 4 is a flowchart of a method for organizing data records according to a third embodiment of the present invention;
fig. 5 is a flowchart of a method for analyzing data records in the method according to the fourth embodiment of the present invention;
fig. 6 is a flowchart of a method for analyzing data records in the method according to the fourth embodiment of the present invention;
fig. 7 is a schematic structural diagram of an operation device for data recording according to a fifth embodiment of the present invention;
fig. 8 is a schematic structural diagram of an apparatus according to a sixth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of an operation method of a data record according to an embodiment of the present invention, and this embodiment is applicable to any situation in which a data record in a row storage manner is organized and analyzed in a database technology. The data recording operation method provided in this embodiment may be executed by a data recording operation device, which may be implemented in a software and/or hardware manner, and integrated into a device for executing the method, where the device for executing the method in this embodiment may be any intelligent terminal capable of operating data recording, such as a tablet computer, a desktop computer, and a notebook computer. Specifically, referring to fig. 1, the method specifically includes the following steps:
and S110, acquiring record organization description corresponding to the data record.
Specifically, the data records are row records in a table structure. In order to add columns to the table where data records are located without reconstructing the data, it is necessary to obtain the number of original columns in the table definition and add a new column after the original columns. When columns are added to a table, objects in a data dictionary need to be modified, specifically, the data dictionary defines and describes data items, data structures, data streams, data stores, processing logics, external entities and the like of data, and aims to describe the logic identification, column name, storage type, length and the like of each column in the table where data records are located in detail. Further, after columns are added to the table, each added column is described in the data dictionary, the number of original columns is recorded in the data dictionary as the logic identifier of the first added column, the logic identifiers are sequentially added to correspondingly record the rest added columns, and information such as column names, storage types and lengths corresponding to the added columns is recorded. Preferably, in order to distinguish the original column from the added column in the table, a distinguishing identifier may be correspondingly set when each column is described in the data dictionary, for example, the original column is identified by the character "0", and the added column is identified by the character "1".
Further, when data in data records in a table needs to be organized or analyzed, a record organization description of the table needs to be generated according to information in a data dictionary. Specifically, the record organization description is generated according to the description of each column of information in the table in the data dictionary, and is a memory structure for assisting in organizing and analyzing data records in the database. Further, the record organization description in this embodiment obtains the numbers of the original columns and the added columns in the table according to the description of each column information in the table in the data dictionary, and determines the physical identifier of the extension field in the data record according to the number of the original columns and the number of other added columns before the added columns. Specifically, the physical identifier indicates an actual sequence of fields stored in the data record, preferably, the physical identifier may be a physical ID of a field in the data record, and correspondingly, an order of a column corresponding to the field when the table where the data record is located is defined is a logical identifier of the field, and preferably, the logical identifier of the field may be a logical ID of a column corresponding to the field when the table is defined in the data record. For example, the logical ID of the corresponding field in the data record listed by C1 in the preceding table TAB is 0, and the physical ID is 2. Specifically, the record organization description describes each field information in the data record according to the column information in the table where the data record is located, where the record organization description may include the number of fields in the data record, an array of each field information, and a logical ID array of fields in the record physical format. Further, the record organization description obtains the number of original columns and added columns according to the distinguishing identification of each column in the table where the data records are located in the data dictionary, generates description information of the original fields corresponding to the original columns in the data records through a sequence adjusting organization method, and determines the physical identification of the extended fields corresponding to the added columns in the data records, wherein the extended fields are the fields corresponding to the added columns in the table where the data records are located in the data records. Specifically, each field in each field information array is arranged according to a corresponding logical ID, and the information array of each field may include the contents of the physical ID, the data type, the encryption compression information, and the like of the field; the logical ID array is recorded with the correspondence between the physical ID and the logical ID of each field registered. Furthermore, the record organization description is generated according to the data dictionary when the data in the data record is organized or analyzed, and when the data in the data record is not required to be organized or analyzed, the corresponding record organization description can be correspondingly deleted, so that resources are saved.
Further, when field data in the data records are organized or analyzed, firstly, according to the description of each column information in the table where the data records are located in the data dictionary, the original columns and the added columns in the table are distinguished according to the distinguishing identifiers, the physical identifiers of the original columns are determined through a sequence adjusting organization method, according to the number of the original columns and the number of other added columns before the added columns, the physical identifiers of the added columns are determined, the corresponding relation between the physical ID and the logic ID of each field is recorded, and the record organization description is generated. When field data in a data record is organized or analyzed, a record organization description generated corresponding to the data record is obtained.
S120, acquiring the data length of each original field and each extension field in the data record, and determining the data offset of each extension field according to the sum of the data lengths of the original fields and the sum of the data lengths of other extension fields before the extension fields.
Specifically, when data in a data record in a table is organized, a record organization description corresponding to the data record is obtained first, and when the corresponding record organization description is obtained, physical identifiers of fields in the data record are determined. When data records are organized, on the premise of ensuring data access speed in most cases, data of extension fields need to be organized into original fields in order to avoid data reconstruction when columns are added, and in order to determine the positions of the extension fields in the data records, the data length of each original field and each extension field in the data records needs to be acquired so as to determine the data offset of the extension fields in the data records.
Further, the original field may include a fixed length field and a variable length field, and correspondingly, the original column may include a fixed length column and a variable length column, specifically, the fixed length field is a corresponding field in the data record of a column with a fixed data length determined according to a data type when defining a table where the data record is located, and the variable length field is a corresponding field in the data record of a column with a data length that can be customized when defining a table where the data record is located, for example, the table TAB is defined as (C1VARCHAR (10), C2INT, C3VARBINARY (10), C4INT), where data in the C2 column and the C4 column are shaping data and occupy 4 bytes and are a fixed length column in the table TAB, data in the corresponding field in the data record is a fixed length field, data in the C1 column is character data, the corresponding field in the data record may store a character string with a variable length, and the number of bytes occupied may be changed by customization, the data in column C3 is binary data, the corresponding field in the data record can store binary data with variable length, the occupied byte number can be changed by self-definition, columns C1 and C3 are variable length columns in table TAB, and the field in the data record is a variable length field. Furthermore, the record organization description can also obtain the data length of the fixed length column in the original column when the table where the data record is located is defined, and because the original fields in the data record are organized according to the sequence of fixing length first and then lengthening length through the sequence adjusting organization method, the data offset of the first variable length column in the table can be determined according to the sum of the data lengths of the fixed length column, so that the data offset of the first variable length field in the data record is determined, wherein the first variable length field is the corresponding field of the first variable length column in the table where the data record is located in the data record. Further, the record organization description may further include the number of original fields, the number of fixed-length fields, and the data offset of the first variable-length field in the data record, and the information array of each field may further include the data offset of the field. Wherein, because the record organization description is generated according to the data dictionary and does not relate to field data, the data offset in the field information array is only valid for a fixed-length field in the original field, and is the position of the field in the data record.
Further, when field data in a data record is organized, first, a physical identifier of a field in a record organization description corresponding to the data record is obtained, data of an original field in the data record is organized through a sequence organization method, in order to determine a position of an extension field in the data record, a data length of each original field and each extension field in the data record is obtained, and a data offset of each extension field in the data record is determined according to a sum of the data lengths of the original field and a sum of data lengths of other extension fields before the extension field.
S130, organizing the data of each extension field into data records according to the physical identification and the data offset of each extension field.
Specifically, data of an original field in the data record is organized through an ordering organization method, for an extension field, after physical identification and data offset of each extension field are determined, a physical ID of last non-NU LL data in the extension field is obtained, and field values corresponding to the extension field are sequentially added to a corresponding data offset position of the extension field in the data record from the physical ID of the first extension field until the physical ID of the last non-NU LL data.
Further, the organizing the data of each extension field into the data record sequentially comprises: the data of each extension field is sequentially organized into extension field areas in the data record according to the record format.
Specifically, the present embodiment defines a recording format for data organization in data records, and as shown in table 1, in a database, data records are generally divided into three parts, namely a record header, a data part and a record trailer.
TABLE 1
Wherein, the length of the record, the control information of each field and other information are registered in the record head; the data part is added with an extension field area on the basis of a fixed-length field area and a variable-length field area of a sequence-adjusting organization method; the tail of the record registers some check or additional information.
In the embodiment of the invention, the recording length, the control information and the data part in the recording head are particularly concerned, and other contents and the recording tail in the recording head are not concerned.
Specifically, the length of a record in a record header is the total length of the whole record, control information comprises information such as whether each field is NU LL and whether the field is compressed, so that the control information of the field is not large, a few bits (bits) can represent the completion, and the control information of all the fields is concentrated together and can be concentrated in a few bytes, so that the used space is reduced.
Further, the fixed field area has a value space having a length equal to the length of the data type of the fixed field in the data portion regardless of whether the value is NU LL, and the space has a length equal to the length of the data type of the fixed field in the data portion, and if the variable length field value of the variable length field area is NU LL, there is no value space in the data portion, and if the variable length field value of the variable length field area is NU LL in the extended field area, these fields are not necessarily written in the record, and if the intermediate extended field is NU LL, there is no control information in the extended field area, and if the extended field is NU LL.
Further, the physical ID of the last non-NU LL data in the extension field is obtained according to the record organization description, and the field values corresponding to the extension field are sequentially organized according to the defined record format from the physical ID of the first extension field to the corresponding data offset positions in the extension field area of the data record until the physical ID of the last non-NU LL data.
According to the technical scheme, all the extension fields are sequentially organized behind the original field by setting the physical identification of the corresponding extension field in the record organization description and according to the data offset of the extension fields in the data record, the original field still organizes data in an order adjusting mode, the speed of data access in most cases is guaranteed, and after columns are added, the value of the extension field of the existing data record can be analyzed into NU LL, so that when the columns are added to the table, only the data dictionary is operated, the data do not need to be reconstructed, and the efficiency of adding the columns to the table is improved.
Example two
Fig. 2 is a flowchart of a method for generating a record organization description in the method according to the second embodiment of the present invention. The second embodiment of the present invention is to further exemplify the flow of record organization description generated according to the data dictionary on the basis of the above-mentioned embodiments.
As shown in fig. 2, the record organization description is automatically generated according to the data dictionary of the database, and the generation step is roughly divided into two steps.
And step one, traversing all the field objects of all the columns in the table where the data records are located after the extension fields are added, and filling the information of each field. And for the original field, counting the sum of the number of fixed-length fields in the field before the field and the data length. If the field is a fixed-length field, setting the physical ID of the field as the number of the fixed-length field before the field, and setting the data offset as the sum of the data lengths of the fixed-length field before the field; if the field is a variable length field, the physical ID is temporarily set to-1, and the data offset is not calculated. For the extension field, registering corresponding column information of the field in a table where the data record is located, wherein the physical ID of the extension field is equal to the corresponding logical ID, namely, the extension field is sequentially increased according to the number of the extension fields before the field on the basis of the number of the original fields; and step two, after all columns in the table where the data records are located are processed, traversing the original fields in the field information array, and setting the physical IDs of the variable length fields.
Specifically, the generated record organization description may include the number of fields in the data record, the number of original fields, the number of fixed-length fields, the data offset of the first variable-length field, the information arrays of each field, and the logical ID array of the field in the record physical format. Wherein, the data offset of the first variable length field is the sum of the data lengths of the fixed length fields in the original field; the record logic ID array represents the logic IDs corresponding to the fields in sequence after the corresponding physical IDs are arranged.
For example, a table TAB is defined as (C1VARCHAR (10), C2INT, C3VARBINARY (10), C4INT), and when data records in the table TAB are organized after adding columns C5INT, C6VARBINARY, the contents of the record organization description generated are as shown in table 2.
TABLE 2
According to the technical scheme, all the extension fields are sequentially organized behind the original field by setting the physical identification of the corresponding extension field in the record organization description and according to the data offset of the extension fields in the data record, the original field still organizes data in an order adjusting mode, the speed of data access in most cases is guaranteed, and after columns are added, the value of the extension field of the existing data record can be analyzed into NU LL, so that when the columns are added to the table, only the data dictionary is operated, the data do not need to be reconstructed, and the efficiency of adding the columns to the table is improved.
EXAMPLE III
Fig. 3 is a flowchart of an operation method of data recording according to a third embodiment of the present invention. In this embodiment, the organization method of each field data in the data record is optimized based on the above embodiments. Referring to fig. 3, the method of the present embodiment specifically includes:
s210, organizing the original field according to the order adjusting mode, and determining the data offset of the first extension field.
Specifically, when data of each field in the data record is organized, the data offset of the first variable length field is not changed according to the mode of organizing the record in a sequence. Further, the sum of the data lengths of the variable length fields is determined according to the data length of each variable length field in the original field, the data of the variable length fields are organized sequentially from the data offset of the first variable length field, and the data offset of the first extension field is determined according to the sum of the data offset of the first variable length field and the data length of the variable length field in the record organization description.
S220, determining the data offset of the extension field according to the sum of the data offset of the first extension field and the data length of other extension fields before the extension field.
Specifically, after determining the data offset of the first extension field, organizing the data of all extension fields, and first determining the sum of the data lengths of other extension fields before each extension field according to the data length of the extension field. Further, the data offset of the extension field is determined according to the sum of the data offset of the first extension field and the data length of other extension fields before the extension field, and the data offsets of all the extension fields in the data record are sequentially determined according to the same method.
And S230, organizing the data of each extension field into an extension field area in the data record according to the physical identification and the data offset of each extension field and the record format.
According to the technical scheme of the embodiment, all the extension fields are sequentially organized behind the original field according to the data offset of the extension fields in the data record, the original field still adopts a sequence adjusting mode to organize data, the data access speed in most cases is guaranteed, after columns are added, the values of the extension fields of the existing data record can be analyzed into NU LL, when the columns are added to the table, only the data dictionary is operated, data do not need to be reconstructed, and the efficiency of adding the columns to the table is improved.
Further, an organization method of each field data in the data record after adding the extension field is further exemplarily described. And organizing the data of each field in the data record added with the extension field, namely reorganizing the data record added with the column of the table where the data record is located.
Specifically, in the present embodiment, the organizing method of each field data in the data record is to organize all field data in the data record according to the physical order of each field in the data record, that is, the physical ID of each field, according to the method specified in the record format, and sequentially write the field data in the data record according to the corresponding physical ID. As shown in fig. 4, the steps of data record organization are generally as follows:
the method comprises the steps of firstly organizing original field data according to physical IDs, filling data of a fixed length field area and a variable length field area of a data part in a data record, and setting corresponding control information at a record head, secondly, obtaining the physical ID of the last non-NU LL field in an extension field, setting the physical ID as the physical ID of the last data needing to be written in the record, and thirdly, calculating data offset of the extension field successively according to field information in record organization description from the physical ID of the first extension field to the physical ID of the last data needing to be written in the record, organizing the data among the extension fields, and writing corresponding field values into the data record sequentially.
Illustratively, the table TAB is defined as (C1VARCHAR (10), C2INT, C3VARBINARY (10), C4INT), wherein the organization format of the data records ('C1 _ value', 2, 'C3 _ value', 4) of the table TAB when no columns are added is as shown in table 3.
TABLE 3
The organizational format of the data records of table TAB ('C1 _ value', 2, 'C3 _ value', 4, NU LL LL) is still as shown in table 3 after the table TAB adds columns C5INT, C6VARBINARY, the organizational format of the data records of table TAB (NU LL LL, 'C3 _ value', 4, NU LL, 0x1234) is as shown in table 4.
TABLE 4
Example four
Fig. 5 is a flowchart of a method for parsing a data record in an operation method of the data record according to a fourth embodiment of the present invention. On the basis of the above embodiment, the embodiment of the present invention further includes: and analyzing the fields in the data records to be analyzed according to the physical identifiers of the fields in the data records to be analyzed in the record organization description, and acquiring the field values of the fields.
Specifically, when a user needs to analyze a data record, the fields in the data record to be analyzed are analyzed according to the physical identifiers, namely the physical IDs, of the fields in the record organization description corresponding to the data record to be analyzed. Where the field includes each field or designated field in the data record to be parsed. Specifically, according to the use condition of the analyzed data record, dividing the fields in the analyzed data record into each field in the data record to be analyzed, acquiring the field value of each field, analyzing the corresponding designated field in the data record to be analyzed according to the set column identifier, and acquiring the field value of the designated field. Further, when each field in the data record to be analyzed is sequentially analyzed, the field value of the field is obtained from the start field, and the data pointer to be analyzed is gradually shifted backward by the data length of the field, so as to obtain the field value of the next field until all fields in the data record to be analyzed are processed.
The method comprises the steps of sequentially analyzing each field in a data record to be analyzed, and obtaining the field value of each field, wherein a data pointer to be analyzed is set to be the initial position of a data part in the data record to be analyzed, a current physical ID is set to be 0, a current logic ID is obtained according to the current physical ID, corresponding field information is obtained in record organization description according to the logic ID, the field value of the corresponding field is analyzed according to the field information, the data pointer to be analyzed is moved backwards according to the data length of the current field, the current physical ID value is increased, if all the fields are processed, the current physical ID value is ended, if the data pointer to be analyzed is not processed and is not moved to the tail of the record, the field value of the next field is sequentially and circularly obtained, if the data pointer to be analyzed is moved to the tail of the record, the fields corresponding to the subsequent physical ID are set to be NU LL and then the fields are returned, and the field values of each field in the data record to be analyzed are obtained.
Further, analyzing a corresponding designated field of the column in the data record to be analyzed according to the set column identifier, when the field value of the designated field is obtained, determining the physical identifier of the field through the column identifier, determining the data offset of the field according to the sum of the data lengths of other fields before the field, and obtaining the field value of the field at the data offset position in the data record.
Further, on the basis of the above technical solution, the field value of the acquired field is further explained according to the field type in the data record. Fields in a data record can be divided into a fixed-length field and a variable-length field in an original field, and an extended field. Referring specifically to fig. 5, obtaining the field value of the field may include:
s310, determining the field type of the field according to the physical identification of the field and the number of the original fields.
Specifically, the physical identification of the fields is preferably a physical ID, and the physical IDs of the fields are sequentially set by organizing the extended fields into the original fields in the data record. So when the physical ID of the field is less than the number of the original field, the field is the original field; when the physical ID of a field is greater than or equal to the number of original fields, the field is an extension field.
S320, judging whether the field is an original field. If yes, go to S330; if not, go to S340.
S330, acquiring the field value of the field according to the data offset of the field and the control information in the record head.
Specifically, when a field is an original field, the control information of the original field in the record header determines whether the field is empty according to the corresponding control information, if the field is empty, the field value is returned to NU LL, and if the field is not empty, the field value corresponding to the field is acquired at the data offset determined by the field.
S340, acquiring the field value of the field according to the data offset of the field and the control information of the field in the extended field area.
Specifically, when the field is an extended field, the control information of the extended field is in the extended field area, the corresponding control information in the extended field area is acquired at the data offset determined by the field, whether the field is empty is determined according to the control information, if the field is empty, the control information is returned, the field value is set to NU LL, and if the field is not empty, the control information of the extended field is acquired at the determined data offset, and further, if the data offset of the field is equal to the record length, the field value of the field is set to NU LL.
According to the technical scheme of the embodiment, the field at the offset position of the corresponding data is analyzed by setting the record format, and the field value of the corresponding field is obtained, so that the data of the original field in the data record after the column is added in the table can be accurately analyzed, and the data does not need to be rebuilt when the column is added.
Further, a field value method of acquiring a specified field among operation methods of the data record is exemplarily described. As shown in fig. 6, the steps of obtaining the field value of the designated field are roughly as follows:
the method comprises the steps of firstly, obtaining information of a field from a field information array of corresponding record organization description according to column identification of the field in a table where data records are located, secondly, judging whether the field is an original field according to corresponding physical ID in the field information and the number of the original fields in the record organization description, if so, executing the third step, if not, executing the seventh step, if so, obtaining control information in a record header, judging whether the field is NU LL, if not, returning, if not, executing the fourth step, comparing the physical ID with the number of the fixed length fields, judging whether the field is the fixed length field, if so, executing the fifth step, if not, executing the sixth step, obtaining a field value at the data offset position according to corresponding data offset in the field information, returning, if not, traversing all variable length fields from the physical ID of a first variable length field is set, starting from data offset of the first variable length field, obtaining a data offset position of the data from the data offset position of the variable length field, and returning the data offset position of the data, otherwise, and executing the seventh step, and returning the data offset position of the data after the ninth step, obtaining the data offset position of the data, and returning, if not, the data offset position of the data of the ninth variable length field is set length field, the ninth step, the data, and the ninth step, and the seventh step, and the step, and.
EXAMPLE five
Fig. 7 is a schematic structural diagram of an operation apparatus for data recording according to a fifth embodiment of the present invention, as shown in fig. 7, the apparatus includes:
the organization description obtaining module 501 is configured to obtain record organization descriptions of the table, record descriptions of original columns in the organization description obtaining table, obtain the numbers of the original columns and the added columns at the same time, and determine a physical identifier of an extension field in the data record according to the number of the original columns and the number of other added columns before the added columns, where the extension field is a field corresponding to the added column in the table where the data record is located.
The data offset determining module 502 is configured to obtain data lengths of an original field and an extended field in a data record, and determine a data offset of the extended field according to a sum of the data lengths of the original field and a sum of data lengths of other extended fields before the extended field, where the original field is a field corresponding to an original column in a table where the data record is located.
A data organization module 503, configured to sequentially organize the data of each extension field into data records according to the physical identifier and the data offset of each extension field.
According to the technical scheme of the embodiment, all the extension fields are sequentially organized behind the original field by setting the physical identification of the corresponding extension field in the record organization description and according to the data offset of the extension field in the data record, the data of the original field in the data record is not affected, the data access speed in most cases is guaranteed, and when the column is added to the table, only the data dictionary is operated without reconstructing the data, so that the column adding efficiency of the table is improved.
Further, the original field may include a fixed-length field and a variable-length field.
Further, the data offset determining module 502 may be specifically configured to: determining the data offset of a first extension field according to the sum of the data offset of the first variable length field in the record organization description and the data length of the variable length field; and determining the data offset of the extension field according to the sum of the data offset of the first extension field and the data length of other extension fields before the extension field.
Further, the data organization module 503 may be specifically configured to: the data of each extension field is sequentially organized into extension field areas in the data record according to the record format.
Further, the above apparatus may further include: a field parsing module 504, configured to parse, after determining a data offset of an extension field, a field in a data record to be parsed according to a physical identifier of the field in the data record to be parsed in the record organization description and the data offset, and obtain a field value of the field; the fields include each field or designated field in the data record to be parsed.
Further, the field parsing module 504 may be specifically configured to: determining the field type of the field according to the physical identification of the field and the number of the original fields; when the field is an original field, acquiring the field value of the field according to the data offset of the field and the control information in the record head; and when the field is the extended field, acquiring the field value of the field according to the data offset of the field and the control information of the field in the extended field area.
The data recording operation device provided by the embodiment is suitable for the data recording operation method provided by any embodiment, and has corresponding functions and beneficial effects.
EXAMPLE six
Fig. 8 is a schematic structural diagram of an apparatus according to a sixth embodiment of the present invention. As shown in fig. 8, the apparatus comprises a processor 60, a storage means 61 and a communication means 62; the number of processors 60 in the device may be one or more, and one processor 60 is taken as an example in fig. 8; the processor 60, the storage means 61 and the communication means 62 of the device may be connected by a bus or other means, as exemplified by the bus connection in fig. 8.
The storage device 61, which is a computer-readable storage medium, may be used to store software programs, computer-executable programs, and modules, such as modules corresponding to the operation method of data recording in the embodiment of the present invention (for example, an organization description acquisition module 501, a data offset determination module 502, and a data organization module 503 in the operation device for data recording). The processor 60 executes various functional applications of the apparatus and data processing, i.e., implements the above-described operation method of data recording, by executing software programs, instructions, and modules stored in the storage device 61.
The storage device 61 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the storage device 61 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the storage 61 may further include memory located remotely from the processor 60, which may be connected to the device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The communication device 62 may be used to implement a network connection or a mobile data connection.
The device provided by the embodiment can be used for executing the data recording operation method provided by any embodiment, and has corresponding functions and beneficial effects.
EXAMPLE seven
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, can implement the operation method of data recording in any of the above embodiments. The method specifically comprises the following steps:
acquiring record organization description of a table where data records are located, wherein the record organization description acquires the number of original columns and added columns in the table, and determines physical identifiers of extended fields in the data records according to the number of the original columns and the number of other added columns before the added columns, and the extended fields are fields corresponding to the added columns in the table where the data records are located;
acquiring the data length of each original field and each extended field in the data record, and determining the data offset of each extended field according to the sum of the data lengths of the original fields and the sum of the data lengths of other extended fields before the extended fields, wherein the original fields are fields corresponding to original columns in a table where the data records are located;
the data of each extension field is sequentially organized into data records according to the physical identification of each extension field and the data offset.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the operations of the method described above, and may also perform related operations in the data recording operation method provided by any embodiment of the present invention.
Based on the understanding that the technical solutions of the present invention can be embodied in the form of software products, such as floppy disks, Read-Only memories (ROMs), Random Access Memories (RAMs), flash memories (F L ASHs), hard disks or optical disks of a computer, etc., and include instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. A method of operating a data record, comprising:
acquiring record organization description of a table where data records are located, wherein the record organization description acquires the number of original columns and added columns in the table, and determines physical identification of an extended field in the data records according to the number of the original columns and the number of other added columns before the added columns, and the extended field is a field corresponding to the added column in the table where the data records are located;
acquiring the data length of each original field and each extended field in the data record, and determining the data offset of each extended field according to the sum of the data lengths of the original fields and the sum of the data lengths of other extended fields before the extended fields, wherein the original fields are fields corresponding to original columns in a table where the data record is located;
organizing the data of each extension field into the data records in sequence according to the physical identification and the data offset of each extension field;
wherein the original field comprises a fixed-length field and a variable-length field;
correspondingly, the original column comprises a fixed length column and a variable length column, the record organization description obtains the data length of the fixed length column, and the data offset of a first variable length field in the data record is determined according to the sum of the data lengths of the fixed length column, wherein the first variable length field is a field corresponding to the first variable length column in a table where the data record is located;
the organizing the data of each extension field into the data record sequentially comprises:
and sequentially organizing the data of each extension field into an extension field area in the data record according to a record format, wherein the record format comprises a record header and a data part, and the extension field area comprises control information, data length and field value of the extension field.
2. The method of claim 1, wherein determining the data offset of the extension field according to the sum of the data lengths of the original field and the sum of the data lengths of other extension fields before the extension field comprises:
determining the data offset of a first extension field according to the sum of the data offset of the first variable length field in the record organization description and the data length of the variable length field;
and determining the data offset of the extension field according to the sum of the data offset of the first extension field and the data length of other extension fields before the extension field.
3. The method of claim 1, wherein after determining the data offset of the extension field, further comprising:
analyzing the fields in the data records to be analyzed according to the physical identifiers and the data offsets of the fields in the data records to be analyzed in the record organization description, and obtaining the field values of the fields in the data records to be analyzed, wherein the fields in the data records to be analyzed comprise each field or designated field in the data records to be analyzed.
4. The method of claim 3, wherein obtaining field values of fields in the data record to be parsed comprises:
determining the field type of the field in the data record to be analyzed according to the physical identification of the field in the data record to be analyzed and the number of the original fields;
when the field in the data record to be analyzed is an original field, acquiring the field value of the field in the data record to be analyzed according to the data offset of the field in the data record to be analyzed and the control information in the record head;
and when the field in the data record to be analyzed is an extended field, acquiring the field value of the field in the data record to be analyzed according to the data offset of the field in the data record to be analyzed and the control information of the field in the data record to be analyzed in the extended field area.
5. An operating device for data recording, comprising:
the record organization description acquisition module is used for acquiring record organization description corresponding to the data record, acquiring the number of original columns and added columns in a table where the data record is located, and determining the physical identification of an extended field in the data record according to the number of the original columns and the number of other added columns before the added columns, wherein the extended field is a field corresponding to the added column in the table where the data record is located;
the data offset determining module is used for acquiring the data lengths of an original field and an extended field in the data record, and determining the data offset of the extended field according to the sum of the data lengths of the original field and the sum of the data lengths of other extended fields before the extended field, wherein the original field is a field corresponding to an original column in a table where the data record is located;
the data organization module is used for organizing the data sequence of each extension field into the data record according to the physical identification and the data offset of each extension field;
wherein the original field comprises a fixed-length field and a variable-length field;
correspondingly, the original column comprises a fixed length column and a variable length column, the record organization description obtains the data length of the fixed length column, and the data offset of a first variable length field in the data record is determined according to the sum of the data lengths of the fixed length column, wherein the first variable length field is a field corresponding to the first variable length column in a table where the data record is located;
the data organization module is specifically configured to:
and sequentially organizing the data of each extension field into an extension field area in the data record according to a record format, wherein the record format comprises a record header and a data part, and the extension field area comprises control information, data length and field value of the extension field.
6. The apparatus of claim 5, further comprising:
and the data analysis module is used for analyzing the fields in the data records to be analyzed according to the physical identifiers and the data offsets of the fields in the data records to be analyzed in the record organization description after determining the data offsets of the extension fields, and acquiring the field values of the fields in the data records to be analyzed, wherein the fields in the data records to be analyzed comprise each field or designated field in the data records to be analyzed.
7. An electronic device, characterized in that the electronic device comprises:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a method of operation of a data record as claimed in any one of claims 1-4.
8. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method of operation of a data record according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711386517.9A CN108073709B (en) | 2017-12-20 | 2017-12-20 | Data recording operation method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711386517.9A CN108073709B (en) | 2017-12-20 | 2017-12-20 | Data recording operation method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108073709A CN108073709A (en) | 2018-05-25 |
| CN108073709B true CN108073709B (en) | 2020-07-24 |
Family
ID=62159007
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711386517.9A Active CN108073709B (en) | 2017-12-20 | 2017-12-20 | Data recording operation method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108073709B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107679245A (en) * | 2017-10-30 | 2018-02-09 | 山东浪潮通软信息科技有限公司 | A kind of organizational structure method of adjustment and device |
| CN112835886A (en) * | 2019-11-25 | 2021-05-25 | 浙江大搜车软件技术有限公司 | Data table field adding method and device |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH09146816A (en) * | 1995-11-17 | 1997-06-06 | Nec Corp | Table expansion system |
| CN100511237C (en) * | 2007-09-11 | 2009-07-08 | 金蝶软件（中国）有限公司 | Data base extension method and device |
| US9189503B2 (en) * | 2012-12-06 | 2015-11-17 | Microsoft Technology Licensing, Llc | Database scale-out |
| CN103617284A (en) * | 2013-12-11 | 2014-03-05 | 北京金和软件股份有限公司 | Data field extension method |
| CN104346466B (en) * | 2014-11-12 | 2018-03-23 | 中国建设银行股份有限公司 | The method and apparatus for the data that Added New Attribute in database |
| CN105808540A (en) * | 2014-12-29 | 2016-07-27 | 金蝶软件（中国）有限公司 | Configuration data storage method and system |
| CN106469224A (en) * | 2016-09-26 | 2017-03-01 | 武汉工程大学 | A kind of expansible data model for SaaS platform |
-
2017
- 2017-12-20 CN CN201711386517.9A patent/CN108073709B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108073709A (en) | 2018-05-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110147204B (en) | Metadata disk-dropping method, device and system and computer-readable storage medium | |
| CN106503008B (en) | File storage method and device and file query method and device | |
| CN108073709B (en) | Data recording operation method, device, equipment and storage medium | |
| CN110222046B (en) | List data processing method, device, server and storage medium | |
| CN110969000B (en) | Data merging processing method and device | |
| CN112559482B (en) | Binary data classification processing method and system based on distribution | |
| CN116126997B (en) | Document deduplication storage method, system, device and storage medium | |
| CN111752941B (en) | Data storage and access method and device, server and storage medium | |
| CN109271383B (en) | Dynamic label processing method and device, computer equipment and storage medium | |
| CN111859863A (en) | Document structure conversion method and device, storage medium and electronic equipment | |
| CN107315806B (en) | Embedded storage method and device based on file system | |
| CN113360095B (en) | Hard disk data management method, device, equipment and medium | |
| CN114328486A (en) | Data quality checking method and device based on model | |
| CN109783024B (en) | Data storage processing method and device | |
| CN110377584A (en) | A kind of access method and device of the data structure edition compatibility based on metadata | |
| CN109918373A (en) | Date storage method, device, server and storage medium | |
| CN114168079B (en) | Method, device and related component for identifying tiff type data deleted from disk | |
| CN116126429B (en) | Method for persistence and recovery of reference of non-data type object | |
| CN116055589B (en) | Data management method and device and computer equipment | |
| CN114185890B (en) | Database retrieval method and device, storage medium and electronic equipment | |
| CN115238131A (en) | Method and device for searching signaling timeout session based on combination of mapping table and linked list | |
| CN112181539B (en) | File processing method, device, equipment and medium | |
| CN112100170B (en) | Method and device for decoding interactive data of database | |
| CN114860684A (en) | Stream data access method and device in stream data storage system | |
| CN116431066A (en) | Data storage method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |