CN113407107B

CN113407107B - Data storage method, device and equipment

Info

Publication number: CN113407107B
Application number: CN202010182414.6A
Authority: CN
Inventors: 张滔
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2020-03-16
Filing date: 2020-03-16
Publication date: 2023-03-24
Anticipated expiration: 2040-03-16
Also published as: CN113407107A

Abstract

The application provides a data storage method, a data storage device and data storage equipment, wherein the method comprises the following steps: acquiring data information, wherein the data information comprises data to be stored and a data identifier of the data to be stored; the data to be stored is an image to be stored or a characteristic model to be stored; determining a target pre-allocation file of the data to be stored from a plurality of created pre-allocation files, wherein the target pre-allocation file comprises a plurality of storage areas, and each storage area comprises a header information field and a data field; selecting an unoccupied target storage region from a plurality of storage regions of the target pre-allocation file; and storing the data to be stored into a data field of the target storage area, and storing the data identifier of the data to be stored into a header information field of the target storage area. According to the technical scheme, the storage performance and the retrieval performance can be improved, and the data storage speed is increased.

Description

Data storage method, device and equipment

Technical Field

The present application relates to the field of storage technologies, and in particular, to a data storage method, apparatus, and device.

Background

Machine learning is a way to realize artificial intelligence, is a multi-field cross subject, and relates to a plurality of subjects such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. Machine learning is used to study how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to improve their performance. Machine learning focuses more on algorithm design, so that a computer can automatically learn rules from data and predict unknown data by using the rules.

In the related technology of machine learning, feature comparison is used as an effective means and widely applied to the fields of face recognition, human body recognition, vehicle recognition and the like. The characteristic comparison means that the characteristics to be detected are compared with a large number of characteristic models one by one, and then target characteristic models meeting requirements are screened from the large number of characteristic models.

Because the number of the feature models is very large, how to store the feature models is not an effective storage mode at present, the storage performance is low, the storage effect is poor, and the user experience is poor.

Disclosure of Invention

In view of the above, the present application provides a data storage method, including:

acquiring data information, wherein the data information comprises data to be stored and a data identifier of the data to be stored; the data to be stored is an image to be stored or a characteristic model to be stored;

determining a target pre-allocation file of the data to be stored from a plurality of created pre-allocation files, wherein the target pre-allocation file comprises a plurality of storage areas, and each storage area comprises a header information field and a data field;

selecting an unoccupied target storage region from a plurality of storage regions of the target pre-allocation file;

and storing the data to be stored into a data field of the target storage area, and storing the data identifier of the data to be stored into a header information field of the target storage area.

The present application provides a data storage device, the device comprising:

the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring data information, and the data information comprises data to be stored and a data identifier of the data to be stored; the data to be stored is an image to be stored or a characteristic model to be stored;

the determining module is used for determining a target pre-allocation file of the data to be stored from a plurality of created pre-allocation files, wherein the target pre-allocation file comprises a plurality of storage areas, and each storage area comprises a header information field and a data field;

a selection module for selecting an unoccupied target storage region from the plurality of storage regions of the target pre-allocation file;

and the storage module is used for storing the data to be stored into the data field of the target storage area and storing the data identifier of the data to be stored into the header information field of the target storage area.

The present application provides a data storage device comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor;

the processor is configured to execute machine executable instructions to perform the steps of:

selecting an unoccupied target storage area from a plurality of storage areas of the target pre-allocation file;

According to the technical scheme, in the embodiment of the application, the data to be stored can be stored in the data field of the target storage area of the target pre-allocation file, and the data identification of the data to be stored is stored in the header information field of the target storage area of the target pre-allocation file, that is, the data to be stored (such as an image to be stored or a feature model to be stored) can be stored through the pre-allocation file, so that the storage of a large amount of data is supported, the storage performance is high, the storage effect is good, the user experience is good, the storage performance and the retrieval performance can be improved, the storage speed of the data to be stored is improved, and the reading performance of the data to be stored is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments of the present application or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings of the embodiments of the present application.

FIG. 1 is a flow chart of a data storage method in one embodiment of the present application;

FIG. 2 is a schematic illustration of a pre-allocation file in one embodiment of the present application;

FIG. 3 is a schematic diagram illustrating a storage area of a pre-allocated file in one embodiment of the present application;

FIG. 4 is a schematic diagram of a pre-allocation file management process in one embodiment of the present application;

FIG. 5 is a schematic diagram of a pre-allocation file allocation process in one embodiment of the present application;

FIG. 6 is a schematic diagram of a pre-allocation file reclamation process in one embodiment of the present application;

FIG. 7 is a schematic illustration of a storage process for database files, images and feature models in the present application;

FIG. 8 is a block diagram of a data storage device in one embodiment of the present application;

FIG. 9 is a block diagram of a data storage device in one embodiment of the present application.

Detailed Description

The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein is meant to encompass any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Depending on the context, moreover, the word "if" is used may be interpreted as "at … …" or "at … …" or "in response to a determination".

The feature comparison is widely applied to the fields of face recognition, human body recognition, vehicle recognition and the like. The characteristic comparison means that the characteristics to be detected are compared with a large number of characteristic models one by one, and then target characteristic models meeting requirements are screened from the large number of characteristic models. In order to implement feature comparison, a large number of images need to be stored in a database, and a large number of feature models need to be stored in the database. Illustratively, the feature model is a string of binary data describing an object in an image through an algorithm, and when the number of feature points in the feature model is larger, the feature model is longer, and the comparison effect of the feature model is better. When the number of feature points in the feature model is smaller, the feature model is shorter, and the comparison effect of the feature model is worse. In summary, based on a large number of images stored in the database, a large number of feature models can be generated, and the generation manner of the feature models is not limited.

Due to the large number of images that need to be stored, how to store these images, in one possible implementation, each image is stored as a file. For example, each image is stored in a folder, i.e., each image as one file. Obviously, a large number of images are written in the folder in the above manner, and as the number of images increases, the file system partition table is frequently updated, so that the read-write performance is poor.

Because of the large number of feature models that need to be stored, how to store these feature models, in one possible implementation, each feature model is stored as a file. For example, each feature model is stored in a folder, i.e., each feature model is stored as a file. Obviously, the above method may result in writing a large number of feature models under the folder, frequently updating the file system partition table, and having poor read-write performance.

In summary, how to store a large number of images and a large number of feature models does not have an effective storage mode at present, and has the disadvantages of low storage performance, poor read-write performance, poor storage effect and poor user experience.

In view of the above findings, in the embodiment of the present application, a plurality of pre-allocation files may be created, where each pre-allocation file may include a plurality of storage areas, and each storage area is used to store data to be stored (for example, an image to be stored or a feature model to be stored).

The technical solutions of the embodiments of the present application are described below with reference to specific embodiments.

Referring to fig. 1, a flow chart of a data storage method is schematically shown, where the method may include:

step 101, data information is obtained, wherein the data information comprises data to be stored and data identification of the data to be stored. For example, the data to be stored may be an image to be stored or a feature model to be stored. Of course, the data to be stored may also be other types of data, which is not limited to this, as long as the data storage method of the present application can be adopted, and the image to be stored and the feature model to be stored are described as an example in the following.

For example, when an image needs to be stored (for the sake of convenience of distinction, the image that needs to be stored is referred to as an image to be stored), data information of the image to be stored may be acquired, the data information including the image to be stored and a data identifier of the image to be stored, the data identifier being a unique identifier of the image. When a certain feature model needs to be stored (for convenience of distinction, the feature model needing to be stored is referred to as a feature model to be stored), data information of the feature model to be stored may be obtained, where the data information may include data identifiers of the feature model to be stored and the feature model to be stored, and the data identifier may be a unique identifier of the feature model to be stored.

For example, the data identifier of each image to be stored is unique for the data identifiers of all the images to be stored, that is, the data identifiers of different images to be stored may be different.

For example, the data identifier of each feature model to be stored is unique for all the data identifiers of the feature models to be stored, that is, the data identifiers of different feature models to be stored may be different.

For example, assuming that the to-be-stored feature model a is generated based on the to-be-stored image a, the data identifier of the to-be-stored feature model a is the same as the data identifier of the to-be-stored image a, and based on this, the to-be-stored feature model a corresponding to the data identifier may be queried by using the data identifier of the to-be-stored image a, or the to-be-stored image a corresponding to the data identifier may be queried by using the data identifier of the to-be-stored feature model a.

Step 102, determining a target pre-allocation file of data to be stored from a plurality of created pre-allocation files, wherein the target pre-allocation file comprises a plurality of storage areas, and each storage area comprises a header information field and a data field.

Illustratively, prior to step 102, a plurality of pre-allocated files need to be created, and for each pre-allocated file in the plurality of pre-allocated files, the pre-allocated file may include an index region and a plurality of storage regions, each of which may include a header information field and a data field. Referring to fig. 2, a schematic diagram of a pre-allocation file is shown, and of course, the pre-allocation file may further include other areas, which is not limited in this respect.

For the process of creating a plurality of pre-allocated files, the plurality of pre-allocated files may be created in the storage medium, for example, the plurality of pre-allocated files may be created under a folder of the storage medium. The storage medium may include a Random Access Memory (RAM), a volatile Memory, a nonvolatile Memory, a magnetic disk, a flash Memory, a storage drive (e.g., a hard disk drive), a solid state disk, any type of storage disk (e.g., an optical disk, a dvd, etc.), and the like, and the type of the storage medium is not limited.

For each pre-allocated file in the plurality of pre-allocated files, the pre-allocated file may be a binary file, although the binary file is only an example and does not limit the type of the pre-allocated file.

For the process of creating a plurality of pre-allocated files, the plurality of pre-allocated files may be created in advance under the file system, that is, the plurality of pre-allocated files are created under the file system. A file system is a method and data structure used by an operating system to specify files on a storage medium or partition, i.e. a method to organize files on a storage medium, and a software mechanism in the operating system that is responsible for managing and storing data is called a file management system, which is called a file system for short.

For the process of creating a plurality of pre-allocated files, the size of the storage space occupied by each pre-allocated file can be determined first, and the sizes of the storage spaces occupied by different pre-allocated files are the same or different. Then, a pre-allocation file matching the storage space size is created, i.e. the size of the pre-allocation file is the storage space size.

For example, assuming that 4 pre-allocated files need to be created, 2 pre-allocated files occupy 64MB of storage space, and the other 2 pre-allocated files occupy 128MB of storage space, 64MB of pre-allocated file 1 can be created, 64MB of pre-allocated file 2 can be created, 128MB of pre-allocated file 3 can be created, and 128MB of pre-allocated file 4 can be created. The size of the pre-allocated file 1 is 64MB, meaning: although pre-allocated file 1 currently does not store data, 64MB of storage space is reserved for pre-allocated file 1, that is, pre-allocated file 1 can store 64MB of data, the addresses of the storage space occupied by pre-allocated file 1 may be address A1-address B1, and the length between address A1-address B1 is 64MB. For other pre-allocated files, the situation is similar to that of the pre-allocated file 1, and the description is not repeated here.

Referring to fig. 2, for each storage area in the plurality of storage areas of the pre-allocation file, the storage area is used to store data to be stored (such as an image to be stored or a feature model to be stored), and a specific storage process refers to a subsequent embodiment and is not described herein again. And aiming at the index area of the pre-allocated file, the index area is used for recording the management information of the pre-allocated file and then managing the pre-allocated file according to the management information.

The index area may be located at the head of the pre-allocated file, or at the tail of the pre-allocated file, or at other positions of the pre-allocated file, and is exemplified as being located at the tail of the pre-allocated file in fig. 2. For example, when the size of the index area is 1MB, the index area occupies a storage space of 1MB at the tail of the pre-allocated file.

For an index region of a pre-allocated file, the index region may include a file state of the pre-allocated file, which may be a used state, or a full state, or an idle state.

After the pre-allocation file is created, the file state of the pre-allocation file can be set to be an idle state, the idle state indicates that the pre-allocation file does not store any data to be stored currently, the file is an idle pre-allocation file, and the data to be stored can be stored in the pre-allocation file.

After storing data to be stored in a certain pre-allocation file, if the pre-allocation file can also continue to store the data to be stored, the file state of the pre-allocation file can be set to be a use state, the use state indicates that the pre-allocation file currently stores the data to be stored, but the pre-allocation file is not full and is an incomplete pre-allocation file, and the data to be stored can continue to be stored in the pre-allocation file.

After storing data to be stored in a certain pre-allocation file, if the pre-allocation file cannot continuously store the data to be stored, the file state of the pre-allocation file may be set to be a full state, where the full state indicates that the pre-allocation file currently stores the data to be stored, and the pre-allocation file is full, and is a full pre-allocation file, and cannot continuously store the data to be stored in the pre-allocation file.

For an index region of a pre-allocated file, the index region may include a usage type of the pre-allocated file, such as the usage type is an image type or a feature model type. For example, for each pre-allocation file, data to be stored of one usage type may be stored, and if the pre-allocation file is only used for storing an image to be stored, the usage type of the pre-allocation file is set to be the image type, which indicates that all images stored in the pre-allocation file are to be stored. If the pre-allocation file is only used for storing the feature model to be stored, setting the use type of the pre-allocation file as the feature model type, and indicating that all the feature models stored in the pre-allocation file are to be stored.

For an index region of a pre-allocated file, the index region may include first address information of occupied storage regions in the pre-allocated file. Alternatively, for an index region of the pre-allocated file, the index region may include second address information for an unoccupied storage region in the pre-allocated file.

For example, referring to fig. 3, the addresses of the storage space occupied by the pre-allocation file 1 are addresses A1 to B1, the length between the addresses A1 to B1 is 64MB, the length between the addresses C1 to B1 is 1MB, and the addresses have been occupied by the index area, and assuming that the addresses A1 to D1 have data to be stored, then:

the address A1 to the address D1 are occupied storage regions, the first address information may include the address A1 and the address D1, which indicates that an occupied storage region is located between the address A1 and the address D1, and the first address information may also include the address D1, which indicates that the address D1 is an occupied storage region before, which is not limited to this, as long as it can be known which addresses are occupied storage regions based on the first address information. Based on the first address information, it may be determined that the storage region after the address D1 is not occupied yet, i.e., the storage region between the address D1 and the address C1 is not occupied yet, and may be used for storing data to be stored.

Alternatively, the address D1-the address C1 are unoccupied storage areas, the second address information may include the address D1 and the address C1, which indicates that an unoccupied storage area exists between the address D1 and the address C1, and the second address information may also include the address D1, which indicates that an unoccupied storage area exists after the address D1, which is not limited to this, as long as it can be known which addresses are unoccupied storage areas based on the second address information. Based on the second address information, it may be determined that the storage region following the address D1 is not already occupied, i.e., the storage region between the address D1 and the address C1 is not already occupied, and may be used to store data to be stored.

Based on the created plurality of pre-allocation files, for step 102, a target pre-allocation file for storing data is determined from the created plurality of pre-allocation files, which may include, but is not limited to, the following ways:

in the method 1, if a pre-allocation file with a file state being a use state exists in a plurality of pre-allocation files, the pre-allocation file is determined as a target pre-allocation file of data to be stored. If the pre-allocation files with the file states in the use states do not exist in the pre-allocation files, the pre-allocation files with the file states in the idle states are obtained from the pre-allocation files, and the pre-allocation files are determined as target pre-allocation files of the data to be stored.

For example, for each pre-allocated file in a plurality of pre-allocated files, the file state of the pre-allocated file is read from the index area of the pre-allocated file, and the file state is a use state, a full state or an idle state. And if the file state of the pre-allocation file 1 is the use state, determining the pre-allocation file 1 as a target pre-allocation file of the data to be stored. If the file states of pre-allocation file 1 and pre-allocation file 2 are full state and the file states of pre-allocation file 3 and pre-allocation file 4 are idle state, pre-allocation file 3 or pre-allocation file 4 can be determined as the target pre-allocation file of the data to be stored.

For example, if a pre-allocation file with a file state being a use state exists in the pre-allocation files, and the use type of the pre-allocation file is matched with the data to be stored, the pre-allocation file is determined as a target pre-allocation file of the data to be stored. And if the use type of the pre-allocated file does not match with the data to be stored, not determining the pre-allocated file as a target pre-allocated file of the data to be stored.

For example, assuming that the file states of the pre-allocation file 1 and the pre-allocation file 2 are the use states, the use type of the pre-allocation file 1 is the image type, and the use type of the pre-allocation file 2 is the feature model type, then: when the data to be stored is the image to be stored, the use type of the pre-allocation file 1 is matched with the image to be stored, and the use type of the pre-allocation file 2 is not matched with the image to be stored, so that the pre-allocation file 1 is determined as a target pre-allocation file of the image to be stored. When the data to be stored is the feature model to be stored, the use type of the pre-allocation file 1 is not matched with the feature model to be stored, and the use type of the pre-allocation file 2 is matched with the feature model to be stored, so that the pre-allocation file 2 is determined as a target pre-allocation file of the feature model to be stored.

For another example, assuming that the file state of the pre-allocation file 1 is the use state, and the use type of the pre-allocation file 1 is the image type, then: when the data to be stored is the feature model to be stored, the use type of the pre-allocation file 1 is not matched with the feature model to be stored, the pre-allocation file with the file state being an idle state is obtained from the pre-allocation files, and if the pre-allocation file 2 is obtained, the pre-allocation file 2 is determined as a target pre-allocation file of the feature model to be stored. When the file state of the pre-allocation file 2 is an idle state, the use type of the pre-allocation file 2 is null, and the pre-allocation file 2 can store the feature model to be stored and can also store the image to be stored. After the pre-allocation file 2 is determined as a target pre-allocation file of the feature model to be stored, the file state of the pre-allocation file 2 may be set as the use state, and the use type of the pre-allocation file 2 is set as the feature model type, which indicates that the pre-allocation file 2 only stores the feature model to be stored.

In the mode 2, the data information may further include a set identifier of a set to which the data to be stored belongs, and based on this, the pre-allocated file corresponding to the set identifier may be obtained from the plurality of pre-allocated files, and the pre-allocated file corresponding to the set identifier is determined as the target pre-allocated file of the data to be stored. For example, if there exists a pre-allocated file corresponding to the set identifier in the pre-allocated files, and the file state of the pre-allocated file corresponding to the set identifier is a full state, or there does not exist a pre-allocated file corresponding to the set identifier in the pre-allocated files, then: the pre-allocation file with the file state being the idle state can be obtained from the plurality of pre-allocation files, and the obtained pre-allocation file is determined as the pre-allocation file corresponding to the set identifier.

For example, for a large amount of data to be stored, the data to be stored may be divided into the same or different sets based on characteristic information of the data to be stored, which may include, but is not limited to, geographic location, gender, age, and the like, and this characteristic information is not limited and may be configured arbitrarily. For example, the geographic location of the user A1-the user a100 belongs to a city a, and the data to be stored of the user A1-the user a100 are divided into a set a; the geographical positions of the users B1 and B20 belong to a city B, and the data to be stored of the users B1 and B20 are divided into a set B, and so on. For another example, the gender of the users C1-C500 is male, and the data to be stored of the users C1-C500 are divided into a set C; the gender of the users D1-D500 is female, and the data to be stored of the users D1-D500 are divided into a set D, and so on. Of course, the above is only an example, and the set to which the data to be stored belongs is not limited.

The same pre-allocation file can be stored for a plurality of data to be stored in the same set, and the same pre-allocation file can not be stored for a plurality of data to be stored in different sets. For example, although a plurality of data to be stored of set a is stored in pre-allocation file 1, even if the file state of pre-allocation file 1 is in the use state, the data to be stored of set B is not stored in pre-allocation file 1.

For example, in step 101, the data information may include data to be stored, a data identifier of the data to be stored, and a set identifier of a set to which the data to be stored belongs, such as the set identifier aaa.

In step 102, it is first determined whether a pre-allocated file corresponding to the set identifier aaa exists in the plurality of pre-allocated files. If the pre-allocation file corresponding to the set identifier aaa does not exist, the pre-allocation file with the file state being in the idle state, such as the pre-allocation file 3, may be obtained from the plurality of pre-allocation files, and the pre-allocation file 3 is determined as the pre-allocation file corresponding to the set identifier aaa, and the pre-allocation file 3 is determined as the target pre-allocation file of the data to be stored, and the file state of the pre-allocation file 3 is set to the use state. Or,

if there is a pre-allocated file corresponding to the set identifier aaa, such as pre-allocated file 1, the file state of pre-allocated file 1 is read from the index area of pre-allocated file 1, and the file state is a use state or a full state. If the file state of the pre-allocation file 1 is the use state, the pre-allocation file 1 can be determined as a target pre-allocation file of the data to be stored. If the file state of the pre-allocation file 1 is the full state, acquiring a pre-allocation file with a file state being an idle state from the plurality of pre-allocation files, such as the pre-allocation file 3, determining the pre-allocation file 3 as a pre-allocation file corresponding to the set identifier aaa, determining the pre-allocation file 3 as a target pre-allocation file of data to be stored, and setting the file state of the pre-allocation file 3 as a use state.

For example, if there is a pre-allocation file 1 corresponding to the set identifier aaa and the file state of the pre-allocation file 1 is the use state, it may be further determined whether the use type of the pre-allocation file 1 matches the data to be stored. If so, determining the pre-allocation file 1 as a target pre-allocation file of the data to be stored. If not, then pre-allocation file 1 is not determined to be the target pre-allocation file for the data to be stored.

For example, assuming that the usage type of the pre-allocation file 1 is an image type and the data to be stored is an image to be stored, the usage type of the pre-allocation file 1 is matched with the image to be stored, and the pre-allocation file 1 is determined as a target pre-allocation file of the image to be stored. Assuming that the use type of the pre-allocation file 1 is an image type and the data to be stored is a feature model to be stored, the use type of the pre-allocation file 1 is not matched with the feature model to be stored, and the pre-allocation file 1 is not determined as a target pre-allocation file of the feature model to be stored; based on this, a pre-allocation file with a file state in an idle state is obtained from a plurality of pre-allocation files, for example, the pre-allocation file 3 is determined as a pre-allocation file corresponding to the set identifier aaa, the pre-allocation file 3 is determined as a target pre-allocation file of the feature model to be stored, and the file state of the pre-allocation file 3 is set to be a use state.

When the file state of the pre-allocation file 3 is an idle state, the use type of the pre-allocation file 3 is null, and at this time, the pre-allocation file 3 may store the feature model to be stored, and may also store the image to be stored. After the pre-allocation file 3 is determined as the target pre-allocation file of the feature model to be stored, the file state of the pre-allocation file 3 may be set as the use state, and the use type of the pre-allocation file 3 is set as the feature model type, thereby indicating that the pre-allocation file 3 only stores the feature model to be stored.

Step 103, selecting an unoccupied target storage area from a plurality of storage areas of the target pre-allocation file. For example, since the target pre-allocation file includes a plurality of storage regions, an unoccupied target storage region may be selected from the plurality of storage regions of the target pre-allocation file.

Illustratively, the target pre-allocation file further includes an index region, and if the index region includes first address information of an occupied storage region in the target pre-allocation file, an unoccupied target storage region may be selected from the plurality of storage regions of the target pre-allocation file according to the first address information, or if the index region includes second address information of an unoccupied storage region in the target pre-allocation file, an unoccupied target storage region may be selected from the plurality of storage regions of the target pre-allocation file according to the second address information.

For example, assuming that the first address information indicates the last position that is already occupied, based on this, the storage area following the first address information may be taken as the target storage area according to the first address information.

For example, referring to the above-mentioned embodiment, assuming that the target pre-allocated file is pre-allocated file 1, and the first address information includes address A1 and address D1, it is determined that a storage region between address A1 and address D1 is occupied, and it is determined that a storage region between address D1 and address C1 is not occupied, and therefore, a storage region beginning at address D1 may be determined as the target storage region. For another example, assuming that the target pre-allocated file is pre-allocated file 1 and the second address information includes address D1 and address C1, it is determined that the storage region between addresses D1-C1 is not occupied yet, and therefore, the storage region starting at address D1 may be determined as the target storage region.

In summary, based on the first address information or the second address information, an unoccupied target storage region may be selected from the target pre-allocation file, and the target storage region may include a header information field and a data field.

And step 104, storing the data to be stored into the data field of the target storage area, and storing the data identifier of the data to be stored into the header information field of the target storage area, so as to complete the data storage process.

Illustratively, after selecting an unoccupied target storage region from the target pre-allocation file, since the target storage region includes a header information field (i.e., a region for storing a data identifier of the data to be stored) and a data field (i.e., a region for storing the data to be stored), the data to be stored may be stored in the data field of the target storage region, and the data identifier of the data to be stored may be stored in the header information field of the target storage region.

As can be seen from the above technical solutions, in the embodiment of the present application, data to be stored may be stored in a data field of a target storage area of a target pre-allocation file, and a data identifier of the data to be stored may be stored in a header information field of the target storage area of the target pre-allocation file, that is, the data to be stored (such as an image to be stored or a feature model to be stored) may be stored through the pre-allocation file, so as to support storage of a large amount of data, and have the advantages of high storage performance, good storage effect, good user experience, and capability of improving storage performance and retrieval performance, increasing storage speed of the data to be stored, and improving reading performance of the data to be stored.

In a possible embodiment, after step 104, position information of the data to be stored in the target pre-allocation file, such as an offset starting position and an offset length, may also be determined, where the offset starting position represents a data identifier of the data to be stored and a starting position of the data to be stored in the target pre-allocation file, and the offset length represents a data identifier of the data to be stored and a length of the data to be stored in the target pre-allocation file.

For example, assuming that the address interval of the target storage area is address D1-address E1, after storing the data to be stored in the data field of the target storage area and storing the data identifier of the data to be stored in the header information field of the target storage area, determining the offset start position as address D1 and determining the offset length as the length between address D1 and address E1, where the offset start position and the offset length are position information of the data to be stored in the target pre-allocation file. Based on the offset starting position and the offset length, the data identification of the data to be stored and the data to be stored can be inquired from the target pre-allocation file.

Then, the data identifier of the data to be stored, the identity information corresponding to the data to be stored, the file identifier of the target pre-allocation file and the location information may be stored in the database file.

For example, a database file for an image to be stored (denoted as database file a) and a database file for a feature model to be stored (denoted as database file b) may be created. If the data to be stored is the image to be stored, the data identifier of the image to be stored, the identity information corresponding to the image to be stored, the file identifier of the target pre-allocation file and the position information can be stored in the database file a. If the data to be stored is the feature model to be stored, the data identifier of the feature model to be stored, the identity information corresponding to the feature model to be stored, the file identifier of the target pre-allocation file and the position information can be stored in the database file b.

In one possible embodiment, for each pre-allocated file already including data to be stored, the data stored in the target pre-allocated file is referred to as stored data for the sake of convenience of distinction as the target pre-allocated file of the data to be stored. Based on this, when the stored data is deleted from the target pre-allocation file, whether the target pre-allocation file meets the recovery condition can also be determined according to the number of the remaining stored data in the target pre-allocation file. If so, the remaining stored data is migrated to the migration pre-allocation file.

For example, the index area of the target pre-allocation file may further include the number of remaining stored data in the target pre-allocation file, and the total number of data in the target pre-allocation file. Based on this, the number of remaining stored data and the total number of data may be obtained from the index area of the target pre-allocation file. If the ratio of the number of the remaining stored data to the total number of the data is greater than a preset threshold, determining that the target pre-allocation file does not accord with the recovery condition; if the ratio of the number of the remaining stored data to the total number of the data is not greater than the preset threshold, it may be determined that the target pre-allocation file meets the reclamation condition.

The above technical solution of the embodiment of the present application is described below with reference to specific application scenarios.

In the application scenario, the application of feature comparison to the field of face recognition is taken as an example for explanation, and aiming at other fields such as human body recognition and vehicle recognition, the implementation process is similar to that in the field of face recognition.

In the application scenario, a pre-allocation file management process, a pre-allocation file allocation process, a pre-allocation file recovery process, and a storage process of database files, images, and feature models are described below.

Referring to FIG. 4, a diagram of a pre-allocation file management process is shown. In the pre-allocation file management process, a face library storage directory can be created in advance, wherein the face library storage directory comprises a face library folder, and the face library folder is used for storing the images to be stored and the feature models to be stored. Illustratively, the face library folder may include a plurality of pre-allocated files, and the size of the storage space occupied by different pre-allocated files is the same or different, for example, the size of the storage space occupied by each pre-allocated file is 64MB (64 MB is an example and can be arbitrarily configured), that is, the size of each pre-allocated file can be 64MB.

Illustratively, the face library folder (i.e., the facelib folder) is used to store the image to be stored and the feature model to be stored, files in the face library folder are pre-allocated files, and the image to be stored and the feature model to be stored may be stored in different pre-allocated files respectively. That is, after a certain pre-allocated file stores an image to be stored, the feature model to be stored is not stored in the pre-allocated file any more; and if a certain pre-allocation file stores the characteristic model to be stored, no image to be stored is stored in the pre-allocation file.

For example, for each pre-allocated file, the pre-allocated file may include an index area and a plurality of storage areas, and each storage area may include a header information field (i.e., head) for storing a data identifier of data to be stored and a data field (i.e., body) for storing data to be stored (e.g., an image to be stored or a feature model to be stored). The index area is used for recording management information of the pre-allocated file.

Illustratively, the index region may include, but is not limited to: 1. the file state of the file is pre-allocated, and the file state can be a use state, a full state or an idle state. 2. The usage type of the pre-assigned file, which may be an image type or a feature model type. 3. First address information of occupied storage areas in the pre-allocation file, or second address information of unoccupied storage areas in the pre-allocation file. 4. The number of remaining stored data in the pre-allocation file (i.e. the total number of valid data, assuming that 10 data are stored in the pre-allocation file, i.e. 10 storage areas are occupied, the number of remaining stored data is 10), and the total number of data in the pre-allocation file (i.e. how many data are stored in the pre-allocation file at most, assuming that 100 data are stored in the pre-allocation file at most, the total number of data is 100). Of course, the above are only a few examples of the content in the index area, and the content in the index area is not limited.

For example, the number of pre-allocated files may be determined according to the total required capacity of the face library, and the number of pre-allocated files may be created, for example, the size of each pre-allocated file is 64MB (configurable), and the total required capacity of the face library is 600MB, then the number of pre-allocated files is 10, and 10 pre-allocated files may be created.

Illustratively, each pre-allocated file has its own usage type, which means that the pre-allocated file can only be used for storing the image to be stored if the usage type is an image type. If the usage type is a feature model type, it means that the pre-allocation file can be used only for storing the feature model to be stored.

For example, each set (a face library may be a set) may use a pre-allocation file separately, that is, different sets of data to be stored need to be stored in different pre-allocation files.

Referring to fig. 4, the face library storage directory further includes a database file (such as a face library database file, which may also be referred to as a storvcalibdb file), and the number of the database files may be one or more.

Illustratively, the database file is used to record a data identifier of the data to be stored, identity information corresponding to the data to be stored, a file identifier of the target pre-allocation file, and location information (such as an offset start location and an offset length) of the data to be stored in the target pre-allocation file. For example, after storing the data a into the pre-allocation file 1, the data identifier of the data a, the identity information corresponding to the data a, the file identifier of the pre-allocation file 1, and the location information of the data a in the pre-allocation file 1 may be recorded in the database file.

The data identifier of the data to be stored, the identity information corresponding to the data to be stored, the file identifier of the target pre-allocation file and the position information of the data to be stored in the target pre-allocation file all belong to structured data, so that the information can be stored in the database file, and the retrieval and the updating are facilitated.

For the identity information corresponding to the data to be stored, if the data to be stored is the data to be stored for the user a, the identity information refers to the identity information of the user a, and the identity information is not limited, as long as the user a can be correctly identified based on the identity information, such as a mobile phone number, an identity card number, a name, an address, and the like.

For example, a database file for the image to be stored and a database file for the feature model to be stored may be created. The database file of the image to be stored is used for recording the content related to the image to be stored, and the database file of the characteristic model to be stored is used for recording the content related to the characteristic model to be stored.

Referring to fig. 5, a schematic diagram of a pre-allocation file allocation process is shown, which may include:

step 501, data information is obtained, where the data information includes data to be stored (such as an image to be stored or a feature model to be stored), a data identifier of the data to be stored, and a set identifier of a set to which the data to be stored belongs.

Step 502, determine whether there is a pre-allocated file corresponding to the set identifier.

If so, step 503 may be performed, and if not, step 506 may be performed.

Step 503, determine whether there is a pre-allocated file whose usage type matches the data to be stored in the pre-allocated file corresponding to the set identifier. If so, step 504 is performed, and if not, step 506 is performed.

For example, if the usage type of the pre-allocated file corresponding to the set identifier is an image type and the data to be stored is an image to be stored, the usage type of the pre-allocated file is matched with the image to be stored; or, if the use type of the pre-allocated file corresponding to the set identifier is the image type and the data to be stored is the feature model to be stored, the use type of the pre-allocated file is not matched with the feature model to be stored.

Step 504, reading the file state of the pre-allocated file from the index area of the pre-allocated file with the usage type matched with the data to be stored, wherein the file state can be a usage state or a full state. If the file status is the use status, go to step 505; if the file status is full, go to step 506.

Step 505, the pre-allocation file whose file state is the use state (i.e. the pre-allocation file whose use type matches the data to be stored) is determined as the target pre-allocation file of the data to be stored, and then step 508 is executed.

Step 506, the pre-allocation file with the file state being the idle state is obtained from the plurality of pre-allocation files.

For example, the pre-allocation file may be determined as the pre-allocation file corresponding to the set identifier, the file status of the pre-allocation file may be set as the usage status, and the usage type of the pre-allocation file may be set as the usage type matching the data to be stored. For example, if the data to be stored is an image to be stored, the usage type of the pre-allocated file may be set as an image type; if the data to be stored is the feature model to be stored, the usage type of the pre-allocation file may be set as the feature model type.

In step 507, the obtained pre-allocated file (i.e. the pre-allocated file with the file state being an idle state) is determined as the target pre-allocated file of the data to be stored, and then step 508 is executed.

Step 508, select an unoccupied target storage region from the plurality of storage regions of the target pre-allocation file. Illustratively, the target storage area may include a header information field and a data field.

Step 509, store the data to be stored in the data field of the target storage area, and store the data identifier of the data to be stored in the header information field of the target storage area, so as to complete the data storage process.

In a possible implementation manner, the management information of the pre-allocated file may also be cached in the memory, and the file allocation may be performed through the management information of the pre-allocated file. The management information of the pre-allocation file can be updated in the memory in real time, and the management information of the pre-allocation file can be written into the index area periodically.

Illustratively, the thread can be started to sort and recycle the pre-allocated files, so that space waste and space deficiency are avoided. Referring to fig. 6, a schematic diagram of a pre-allocation file recycling process is shown, which may include:

step 601, when the stored data is deleted from the pre-allocation file, the number of the remaining stored data and the total number of data can be read from the index area of the pre-allocation file.

For example, when stored data is deleted from a pre-allocated file, a pre-allocated file reclamation process is triggered. Of course, deleting the stored data from the pre-allocation file is only an example, and other trigger conditions may also be adopted, such as periodically triggering the pre-allocation file recovery process, which is not limited.

Step 602, determining whether the pre-allocation file meets the recycling condition according to the quantity of the remaining stored data and the total quantity of the data. If yes, go to step 603, if no, do not execute recycle operation to the pre-allocation file, and continue to go to step 601 when deleting the stored data from the pre-allocation file next time.

For example, if the ratio of the number of remaining stored data to the total number of data is greater than a preset threshold (which may be configured empirically, such as 50%), the pre-allocation file does not meet the reclamation condition; and if the ratio of the number of the remaining stored data to the total number of the data is not greater than a preset threshold value, the pre-allocation file accords with the recovery condition.

Step 603, recording the pre-allocated file as a finishing starting state. And when the pre-allocated file is in the finishing starting state, no read-write operation is executed through the pre-allocated file any more, and the pre-allocated file is recovered.

Step 604, determine a migration pre-allocated file of remaining stored data from the plurality of pre-allocated files.

For example, the determination process of the migration pre-allocated file may be referred to in steps 502-507, for example, based on the set identifier of the set to which the remaining stored data belongs, the type of the remaining stored data (such as the image type or the feature model type), steps 502-507 may be performed to finally obtain the target pre-allocated file, and in step 604, the target pre-allocated file of the remaining stored data is recorded as the migration pre-allocated file.

Step 605, migrate the remaining stored data to the migration pre-allocation file.

For example, an unoccupied target storage area may be selected from the plurality of storage areas of the migration pre-allocation file, and the remaining stored data may be stored in the data field of the target storage area, and the data identifier of the remaining stored data may be stored in the header information field of the target storage area.

Step 606, determining the location information of the remaining stored data in the migration pre-allocation file.

For example, the location information of the remaining stored data in the migration pre-allocation file may be location information a, which may include the offset start location and the offset length.

Step 607, using the data identifier of the remaining stored data to query the table entry corresponding to the data identifier from the database file, and updating the location information in the table entry using the location information a.

For example, each entry of the database file is used to record a data identifier, identity information, a file identifier of the pre-allocated file, and a mapping relationship between location information of data in the target pre-allocated file, based on which, an entry corresponding to the data identifier may be queried from the database file by using the data identifiers of the remaining stored data, and then, location information a is used to replace the location information in the entry.

Step 608, the pre-allocated file is recycled as a pre-allocated file with a file state being an idle state.

Illustratively, the database file, the image and the feature model may be stored separately, and as shown in fig. 7, the process is a schematic diagram of a storage process of the database file, the image and the feature model, and the process may include:

step 701, acquiring an image to be stored, and acquiring a pre-allocation file for storing the image to be stored, that is, the usage type of the pre-allocation file is matched with the image to be stored.

Step 702, storing the image to be stored in the pre-allocation file.

Step 703, determining the position information of the image to be stored in the pre-allocated file.

Step 704, recording the data identifier of the image to be stored, the identity information corresponding to the image to be stored, the file identifier of the pre-allocated file, and the position information of the image to be stored in the pre-allocated file into a database file.

For example, the above-described contents may be recorded into a database file for an image to be stored.

Step 705, obtaining the feature model to be stored, and obtaining the pre-allocation file for storing the feature model to be stored, that is, the usage type of the pre-allocation file is matched with the feature model to be stored.

Step 706, store the feature model to be stored to the pre-allocation file.

Step 707, determining the position information of the feature model to be stored in the pre-allocation file.

Step 708, recording the data identifier of the feature model to be stored, the identity information corresponding to the feature model to be stored, the file identifier of the pre-allocated file, and the location information into a database file.

For example, the above may be recorded into a database file for the feature model to be stored.

In the above embodiment, the pre-allocation file is stored in a header field and a data field, and this storage structure can implement: the image to be stored and the feature model to be stored can be multiplexed with the storage structure; when the feature matching is performed, the data stored in the storage structure can be directly loaded from the disk into the memory. Illustratively, the above method can avoid frequent writing of a disk, real-time updating of file index information in a memory, regular writing in a storage medium, and statistical correction once by using a database when starting up.

In a possible implementation manner, when performing the feature comparison, the feature model in the pre-allocation file may be loaded into the memory, and then the feature to be detected is compared with the feature model in the memory, which does not limit the feature comparison process, and a conventional feature comparison manner may be adopted.

Based on the same application concept as the method, the embodiment of the present application further proposes a data storage device, as shown in fig. 8, which is a structural diagram of the data storage device, and the device includes:

an obtaining module 81, configured to obtain data information, where the data information includes data to be stored and a data identifier of the data to be stored; the data to be stored is an image to be stored or a characteristic model to be stored; a determining module 82, configured to determine a target pre-allocation file of the data to be stored from a plurality of created pre-allocation files, where the target pre-allocation file includes a plurality of storage areas, and each storage area includes a header information field and a data field; a selecting module 83, configured to select an unoccupied target storage region from the plurality of storage regions of the target pre-allocation file; a storage module 84, configured to store the data to be stored in the data field of the target storage area, and store the data identifier of the data to be stored in the header information field of the target storage area.

Illustratively, the data information further includes a set identifier of a set to which the data to be stored belongs, and the determining module 82 is specifically configured to, when determining a target pre-allocation file of the data to be stored from the created plurality of pre-allocation files: and acquiring the pre-allocated files corresponding to the set identification from the plurality of pre-allocated files, and determining the pre-allocated files corresponding to the set identification as target pre-allocated files of the data to be stored.

The determining module 82 is specifically configured to, when obtaining the pre-allocated file corresponding to the set identifier from the plurality of pre-allocated files: if the pre-allocated files corresponding to the set identifier already exist in the pre-allocated files, and the file state of the pre-allocated files corresponding to the set identifier is a full state, or the pre-allocated files corresponding to the set identifier do not exist in the pre-allocated files, then the pre-allocated files corresponding to the set identifier are added to the pre-allocated files, and the pre-allocated files corresponding to the set identifier are added to the pre-allocated files, so that the files are not added to the pre-allocated files

And acquiring the pre-allocated files with the file states in the idle states from the plurality of pre-allocated files, and determining the acquired pre-allocated files as the pre-allocated files corresponding to the set identifications.

Illustratively, for each pre-allocated file of the plurality of pre-allocated files, the pre-allocated file comprises an index area, the index area comprising a file state of the pre-allocated file; the file state is a use state, a full state or an idle state.

Illustratively, the target pre-allocation file further includes an index area, and the selection module 83 is specifically configured to, when selecting an unoccupied target storage area from the plurality of storage areas of the target pre-allocation file:

and if the index area comprises first address information of an occupied storage area in the target pre-allocation file, wherein the first address information represents the occupied last position, selecting an unoccupied target storage area from the plurality of storage areas of the target pre-allocation file according to the first address information.

The determination module 82 is further configured to: after the data to be stored is stored in the data field of the target storage area, determining the position information of the data to be stored in the target pre-allocation file; the storage module 84 is further configured to: and storing the data identifier of the data to be stored, the identity information corresponding to the data to be stored, the file identifier of the target pre-allocation file and the position information into a database file.

The determination module 82 is further configured to: after the data to be stored is stored in the data field of the target storage area, when the stored data is deleted from the target pre-allocation file, determining whether the target pre-allocation file meets the recovery condition or not according to the quantity of the residual stored data in the target pre-allocation file; the storage module 84 is further configured to: if the determination module 82 determines that the target pre-allocation file meets the reclamation condition, the remaining stored data is migrated to a migration pre-allocation file.

The target pre-allocation file further comprises an index area, wherein the index area comprises the number of the remaining stored data in the target pre-allocation file and the total number of the data in the target pre-allocation file;

the determining module 82 is specifically configured to, when determining whether the target pre-allocation file meets the recovery condition according to the number of the remaining stored data in the target pre-allocation file: acquiring the number of the remaining stored data and the total number of the data from the index area; if the ratio of the number of the remaining stored data to the total number of the data is greater than a preset threshold value, determining that the target pre-distributed file does not accord with a recovery condition; and if the ratio of the number of the residual stored data to the total number of the data is not greater than a preset threshold value, determining that the target pre-allocation file meets the recovery condition.

Based on the same application concept as the method, a data storage device is also provided in the embodiment of the present application, and from a hardware level, a schematic diagram of a hardware architecture of the data storage device may be as shown in fig. 9. The data storage device may include: a processor 91 and a machine-readable storage medium 92, the machine-readable storage medium 92 storing machine-executable instructions executable by the processor 91; the processor 91 is configured to execute machine-executable instructions to implement the methods disclosed in the above examples of the present application. For example, the processor 91 is configured to execute machine-executable instructions to perform the following steps:

Based on the same application concept as the method, embodiments of the present application further provide a machine-readable storage medium, where several computer instructions are stored on the machine-readable storage medium, and when the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

For example, the computer instructions, when executed by a processor, enable the following steps:

The machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: a RAM (random Access Memory), a volatile Memory, a non-volatile Memory, a flash Memory, a storage drive (e.g., a hard drive), a solid state drive, any type of storage disk (e.g., an optical disk, a dvd, etc.), or similar storage medium, or a combination thereof.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Furthermore, these computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art to which the present application pertains. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method of storing data, the method comprising:

determining a target pre-allocation file of the data to be stored from a plurality of created pre-allocation files, wherein the target pre-allocation file comprises an index area and a plurality of storage areas, each storage area comprises a header information field and a data field, the index area is positioned at the head or the tail of the target pre-allocation file, and the data field in each storage area is positioned behind the header information field in the corresponding storage area; the data storage system comprises a header information field in each storage area, an index field and a data storage field, wherein the header information field in each storage area is used for storing a data identifier of data to be stored, the data field in each storage area is used for storing the data to be stored, and the index field is used for storing management information of a pre-allocated file;

storing the data to be stored into a data field of the target storage area, and storing a data identifier of the data to be stored into a header information field of the target storage area;

determining the position information of the data to be stored in the target pre-allocation file; and storing the data identifier of the data to be stored, the identity information corresponding to the data to be stored, the file identifier of the target pre-allocation file and the position information into a database file.

2. The method of claim 1,

the data information further includes a set identifier of a set to which the data to be stored belongs, and the determining a target pre-allocation file of the data to be stored from the created plurality of pre-allocation files includes:

and acquiring the pre-allocation files corresponding to the set identification from the pre-allocation files, and determining the pre-allocation files corresponding to the set identification as target pre-allocation files of the data to be stored.

3. The method of claim 2,

the obtaining of the pre-allocated file corresponding to the set identifier from the pre-allocated files includes:

if the pre-allocated files corresponding to the set identifier already exist in the pre-allocated files, and the file state of the pre-allocated files corresponding to the set identifier is a full state, or the pre-allocated files corresponding to the set identifier do not exist in the pre-allocated files, then the pre-allocated files corresponding to the set identifier are added to the pre-allocated files, and the pre-allocated files corresponding to the set identifier are added to the pre-allocated files, so that the files are not added to the pre-allocated files

4. The method of claim 3, wherein for each pre-allocated file in the plurality of pre-allocated files, the index area in the pre-allocated file comprises a file status of the pre-allocated file; the file state is a use state, a full state or an idle state.

5. The method of claim 1, wherein selecting an unoccupied target storage region from the plurality of storage regions of the target pre-allocated file comprises:

6. The method of claim 1, wherein after storing the data to be stored in the data field of the target storage area, the method further comprises:

when the stored data is deleted from the target pre-allocation file, determining whether the target pre-allocation file meets the recovery condition or not according to the quantity of the residual stored data in the target pre-allocation file;

and if so, migrating the residual stored data to a migration pre-allocation file.

7. The method of claim 6, wherein the index area in the target pre-allocation file comprises a number of remaining stored data in the target pre-allocation file, a total number of data in the target pre-allocation file; determining whether the target pre-allocation file meets a recovery condition according to the number of the remaining stored data in the target pre-allocation file comprises:

acquiring the number of the remaining stored data and the total number of data from the index area;

if the ratio of the number of the remaining stored data to the total number of the data is greater than a preset threshold value, determining that the target pre-distributed file does not accord with a recovery condition;

and if the ratio of the number of the residual stored data to the total number of the data is not greater than a preset threshold value, determining that the target pre-allocation file meets the recovery condition.

8. A data storage device, characterized in that the device comprises:

the determining module is used for determining a target pre-allocation file of the data to be stored from a plurality of created pre-allocation files, wherein the target pre-allocation file comprises an index area and a plurality of storage areas, each storage area comprises a header information field and a data field, the index area is positioned at the head or the tail of the target pre-allocation file, and the data field in the storage areas is positioned behind the header information field in the storage area; the data storage system comprises a header information field in each storage area, an index field and a data storage field, wherein the header information field in each storage area is used for storing a data identifier of data to be stored, the data field in each storage area is used for storing the data to be stored, and the index field is used for storing management information of a pre-allocated file;

the storage module is used for storing the data to be stored into a data field of the target storage area and storing a data identifier of the data to be stored into a header information field of the target storage area;

the storage module is further used for determining the position information of the data to be stored in the target pre-allocation file; and storing the data identifier of the data to be stored, the identity information corresponding to the data to be stored, the file identifier of the target pre-allocation file and the position information into a database file.

9. A data storage device, comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor;

determining a target pre-allocation file of the data to be stored from a plurality of created pre-allocation files, wherein the target pre-allocation file comprises an index area and a plurality of storage areas, each storage area comprises a header information field and a data field, the index area is positioned at the head or the tail of the target pre-allocation file, and the data field in each storage area is positioned behind the header information field in the corresponding storage area; the data storage system comprises a header information field in each storage area, an index field and a data identification field, wherein the header information field in each storage area is used for storing a data identifier of data to be stored;