CN114996224A

CN114996224A - File information statistics method and device and electronic equipment

Info

Publication number: CN114996224A
Application number: CN202210763248.8A
Authority: CN
Inventors: 吴立; 殷俊; 黄鹏; 周祥明; 程长
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2022-07-01
Filing date: 2022-07-01
Publication date: 2022-09-02
Anticipated expiration: 2042-07-01
Also published as: CN114996224B

Abstract

The application relates to a method and a device for file information statistics and electronic equipment, which are used for solving the problem that the time consumed for file information statistics is long when massive file information is faced. The method comprises the steps of determining an operation mode of a first folder according to target operation records generated by the first folder based on n operations, and then counting current attribute records of the first folder and current attribute records of parent node folders of the first folder by adopting the operation mode of the first folder. Based on the method, the real-time statistics of the file information can be realized, the attribute information of the file can be rapidly counted, and the technical effect of improving the efficiency of the file information statistics is achieved.

Description

File information statistical method and device and electronic equipment

Technical Field

The present application relates to the field of information statistics, and in particular, to a method and an apparatus for document information statistics, and an electronic device.

Background

In a scenario facing massive files, when a request for counting information of all files in a specified folder is received, files in all levels in the specified folder need to be traversed layer by layer, and corresponding file information is counted. However, in a scenario of a large amount of files, the specified folder may contain files of several layers, that is, traversing the several layers one by one will take a lot of time, which will make the user wait time longer, and further cause the user experience to be poor.

Disclosure of Invention

The application aims to provide a method and a device for file information statistics and electronic equipment, so that when a large number of files are faced, attribute information of the files can be rapidly counted in real time, and time for counting file information under a folder is shortened.

In a first aspect, the present application provides a method for document information statistics, where the method includes:

acquiring a target operation record generated by a first folder based on n operations, and determining an operation mode of the first folder according to the target operation record; wherein n is an integer greater than or equal to 1;

when the operation mode of the first folder is an updating operation mode, acquiring a first attribute set of the first folder; wherein the first set of attributes includes the latest attribute information of child node files of the first folder;

if the historical attribute record of the first folder exists, determining a difference attribute set between a first historical attribute set and a first attribute set in the historical attribute record; the historical attribute record comprises a first historical attribute set and a second historical attribute set, the first historical attribute set comprises historical attribute information of child node files of the first folder, and the second historical attribute set comprises historical attribute information of all files under the first folder;

adding the difference attribute set to a second historical attribute set of the first folder to generate a second attribute set aiming at the first folder, and obtaining a current attribute record of the first folder based on the second attribute set; wherein the second attribute set comprises current attribute information of all folders under the first folder;

and adding the difference attribute sets to the second historical attribute sets of the parent node folders to generate respective second attribute sets aiming at the parent node folders of the first folder, and obtaining the current attribute records of the parent node folders based on the respective second attribute sets.

By the method, the target operation records generated based on the n operations can be understood as combining a plurality of operation records of the same folder into one operation record, so that the number of times of updating the attribute information of the folder according to the operation records can be reduced, and the effect of updating the attribute information of the file in real time is achieved. Furthermore, the attribute information of the next-level file of the same folder can be counted once the same folder performs operation, namely the first attribute set of the same folder, after the operation mode is determined based on the target operation record, the attribute information of the current folder can be updated by adopting the first attribute set, so that the respective attribute information of the parent node folder is updated sequentially upwards, the real-time counting of the file information is realized, and the counting mode does not need to traverse the information of all files under the folder, so that the time for counting the file information is reduced, and the efficiency for counting the file information is improved.

In one possible design, the method further includes:

if the historical attribute record of the first folder does not exist, taking the first attribute set of the first folder as a second attribute set of the first folder;

obtaining a current attribute record of the first folder according to the current father node folder identifier of the first folder, the first attribute set and the second attribute set;

and aiming at each father node folder of the first folder, adding the first attribute set in the second historical attribute set of each father node folder to obtain the current attribute record of each father node folder.

The attribute information of the current folder is updated so as to update the respective attribute information of the parent node folders of the current folder in turn, so that the real-time statistics of the file information can be realized, and the statistics mode does not need to traverse the information of all files under the folder, thereby reducing the time for file information statistics and improving the efficiency of file information statistics.

In a possible design, after the obtaining a target operation record generated by the first folder based on n operations and determining an operation mode of the first folder according to the target operation record, the method further includes:

when the operation mode of the first folder is a mobile operation mode, acquiring a folder identifier of a current father node of the first folder, a first attribute set and a historical attribute record;

replacing a history father node folder identifier in the history attribute record with the current father node folder identifier aiming at the first folder to obtain a current attribute record of the first folder;

for a first father node folder identified by the current father node folder, adding the first attribute set to a second historical attribute set of respective historical attribute records of the first father node folder and each father node folder of the first father node folder to generate respective current attribute records;

and deleting the first attribute set in a second history attribute set of the history attribute record of each father node folder of the second father node folder and the second father node folder aiming at the second father node folder identified by the history father node folder, and generating the current attribute record of each father node folder.

when the operation mode of the first folder is a deleting operation mode, acquiring a historical attribute record of the first folder;

for the first folder, emptying a first attribute set and a second attribute set in the historical attribute record, and modifying a historical father node folder identifier in the historical attribute record into a deletion identifier to obtain a current attribute record of the first folder;

and deleting the second history attribute set of the first folder from the second history set in the history attribute record of each father node folder aiming at each father node folder of the first folder to obtain the current attribute record of each father node folder.

In a possible design, before the obtaining the target operation record generated by the first folder based on n operations, the method further includes:

acquiring a first operation record and a second operation record generated by a first folder in a preset time period;

judging whether the generation time difference between the first operation record and the second operation record is smaller than a preset threshold value or not;

if not, replacing the first operation record with the second operation record;

if so, merging the first operation record and the second operation record to generate a third operation record;

and in response to the generation time of the third operation record being greater than the preset threshold value or the third operation record being the latest generated operation record in the preset time period, taking the third operation record as a target operation record.

By combining different operation records of the same folder into one operation record within a preset time period, the number of times that the folder updates the attribute records according to the operation records can be reduced, the effect of updating the folder attribute records in real time is achieved, and an updating mode needs to be determined according to the combined operation records when the folder attribute records are subsequently updated.

In one possible design, the merging the first operation record and the second operation record includes:

determining a first operation value in the first operation record and a second operation value in the second operation record; the operation value is used for representing the operation type of the operation recorded in the operation record;

in response to that the phase of the binary number of the operation value corresponding to the first operation type is1, the phase of the binary number of the operation value corresponding to the other operation types is 0;

responding to that the first operation type comprises a plurality of sub-operation types, accumulating the operation values corresponding to the plurality of sub-operation types into the operation value corresponding to the first operation type, wherein the phase of the binary number of the operation value corresponding to any sub-operation type is1, and the phase of the binary number of the operation value corresponding to other sub-operation types is 0;

performing bit-wise or calculation of binary numbers on the first operation value and the second operation value to obtain a third operation value;

and replacing the first operation value in the first operation record by using the third operation value, and obtaining a third operation record after replacement.

The operations are classified, and then the binary codes are adopted to assign values to different operation types, so that the subsequent combination of different operation records of the same file is facilitated, and the assignment mode can effectively reduce the time spent on combination calculation.

In a possible design, after obtaining the current attribute record of each parent node folder, the method further includes:

storing the current attribute records of the first folder and the current attribute records of each father node folder of the first folder into a first storage space and a second storage space of a cache storage space; the storage duration of each attribute record in the first storage space is shorter than that in the second storage space;

and transferring the current attribute record of the first folder and the current attribute record of each father node folder of the first folder from the second storage space to the bottom storage space.

By the method, a cache storage space is added on the basis of the bottom storage space so as to cache the attribute records of the folder, and the storage pressure of the bottom storage space is reduced.

In one possible design, the unloading the current attribute record of the first folder and the current attribute records of the parent node folders of the first folder from the second storage space to the bottom storage space includes:

in response to the second storage space being in a suspended state, transferring the current attribute records of the first folder and the current attribute records of each parent node folder of the first folder from the second storage space to a third storage space of the cache storage space; wherein the second storage space is not used to save newly generated attribute records during the suspended state;

and transferring the current attribute record of the first folder in the second storage space and the current attribute record of each father node folder of the first folder from the third storage space to a bottom storage space.

Based on the method, when the second storage space is used for transferring the cached attribute record to the third storage space, the second storage space is in a suspended state, namely only the transfer action is executed, and the newly generated attribute record is not received, so that the accuracy of the attribute record stored in the third storage space can be ensured, and because the second storage space and the third storage space both belong to the cached storage space, the transfer process has no problems of format conversion and the like, so that the time for transferring the attribute record from the second storage space to the third storage space is less than the time for directly storing the attribute record to the bottom layer module, and the purpose of saving the transfer time is further achieved.

In one possible design, the saving the current attribute record of the first folder and the current attribute records of the respective parent node folders of the first folder to a first storage space of a cache storage space includes:

storing the current attribute record of the first folder and the current attribute record of each father node folder of the first folder into a first storage space of a cache storage space;

and if the storage time of any attribute record in the first storage space is longer than a first preset storage time, deleting any attribute record from the first storage space.

By the method, on one hand, the attribute records of the folder are cached in the first storage space, so that the storage pressure of the bottom storage space is reduced; on the other hand, the first storage space regularly clears the attribute records, so that the purpose that only the latest attribute record is stored in the first storage space is achieved.

In one possible design, the unloading, from the second storage space to the third storage space of the cache storage space, the current attribute record of the first folder and the current attribute record of each parent node folder of the first folder in response to the second storage space being in the suspended state includes:

responding to the fact that the second storage space is in a suspended state, if the storage time of any attribute record stored in the second storage space is longer than a second preset storage time, all attribute records stored in the second storage space are transferred to a third storage space;

and if the total capacity of the attribute records stored in the second storage space is greater than a preset capacity threshold, transferring all the attribute records stored in the second storage space to the third storage space.

Based on the method, the attribute is transferred from the second storage space to the third storage space, and on one hand, the second storage space and the third storage space both belong to the cache storage space, so that compared with the method of directly transferring from the second storage space to the bottom storage space, the time spent on transferring can be saved; on the other hand, if the second storage space is transferred to the third storage space and meets the atomicity requirement, that is, a certain condition is met, all the attribute information of the second storage space is transferred to the third storage space, and the second storage space only executes the transfer operation in the transfer process, so that the accuracy and stability of the transfer attribute record are ensured.

In one possible design, the historical attribute record of the first folder is obtained by:

acquiring a historical attribute record of the first folder from a first storage space of the cache storage space;

if the historical attribute record is not acquired from the first storage space, acquiring the historical attribute record from a second storage space of the cache storage space;

and if the historical attribute record is not acquired from the second storage space, acquiring the historical attribute record from the bottom storage space.

According to the method, the first storage space of the cache storage space stores the latest attribute record, the second storage space of the cache storage space stores the attribute record for the latest period of time, and the bottom storage space stores all the attribute records.

In a second aspect, the present application provides an apparatus for document information statistics, the apparatus comprising:

the operation mode determining module is used for acquiring a target operation record generated by the first folder based on n operations and determining the operation mode of the first folder according to the target operation record; wherein n is an integer greater than or equal to 1;

the attribute information acquisition module is used for acquiring a first attribute set of the first folder when the operation mode of the first folder is an updating operation mode; wherein the first set of attributes includes the latest attribute information of child node files of the first folder;

a differential attribute determining module, configured to determine a differential attribute set between a first historical attribute set in the historical attribute records and the first attribute set if the historical attribute records of the first folder exist; the historical attribute record comprises a first historical attribute set and a second historical attribute set, the first historical attribute set comprises historical attribute information of child node files of the first folder, and the second historical attribute set comprises historical attribute information of all files under the first folder;

a first adding module, configured to add the difference attribute set to a second historical attribute set of the first folder to generate a second attribute set, and obtain a current attribute record of the first folder based on the second attribute set; the second attribute set comprises current attribute information of all folders under the first folder;

and the second adding module is used for adding the differential attribute sets to the second historical attribute sets of the father node folders to generate respective second attribute sets aiming at the father node folders of the first folder, and obtaining the current attribute records of the father node folders based on the respective second attribute sets.

In one possible design, the apparatus is further configured to: if the historical attribute record of the first folder does not exist, taking the first attribute set of the first folder as a second attribute set of the first folder; obtaining a current attribute record of the first folder according to the current father node folder identifier of the first folder, the first attribute set and the second attribute set; and aiming at each father node folder of the first folder, adding the first attribute set into the second historical attribute set of each father node folder to obtain the current attribute record of each father node folder.

In one possible design, the operation mode determining module is further configured to: when the operation mode of the first folder is a mobile operation mode, acquiring a folder identifier of a current father node of the first folder, a first attribute set and a historical attribute record; replacing a history father node folder identifier in the history attribute record with the current father node folder identifier aiming at the first folder to obtain a current attribute record of the first folder; for a first father node folder identified by the current father node folder, adding the first attribute set to a second historical attribute set of respective historical attribute records of the first father node folder and each father node folder of the first father node folder to generate respective current attribute records; and deleting the first attribute set in a second history attribute set of the history attribute record of each father node folder of the second father node folder and the second father node folder aiming at the second father node folder identified by the history father node folder, and generating the current attribute record of each father node folder.

In one possible design, the operation mode determining module is further configured to: when the operation mode of the first folder is a deleting operation mode, acquiring a historical attribute record of the first folder; for the first folder, emptying a first attribute set and a second attribute set in the historical attribute record, and modifying a historical father node folder identifier in the historical attribute record into a deletion identifier to obtain a current attribute record of the first folder; and deleting the second history attribute set of the first folder from the second history set in the history attribute record of each father node folder aiming at each father node folder of the first folder to obtain the current attribute record of each father node folder.

In one possible design, the operation mode determining module is further configured to: acquiring a first operation record and a second operation record generated by a first folder in a preset time period; judging whether the generation time difference between the first operation record and the second operation record is smaller than a preset threshold value or not; if not, replacing the first operation record with the second operation record; if so, merging the first operation record and the second operation record to generate a third operation record; and in response to the generation time of the third operation record being greater than the preset threshold value or the third operation record being the latest generated operation record in the preset time period, taking the third operation record as a target operation record.

In one possible design, the determine operation mode module is further configured to: determining a first operation value in the first operation record and a second operation value in the second operation record; the operation value is used for representing the operation type of the operation recorded in the operation record; in response to that the phase of the binary number of the operation value corresponding to the first operation type is1, the phase of the binary number of the operation value corresponding to the other operation types is 0; responding to that the first operation type comprises a plurality of sub-operation types, accumulating the operation values corresponding to the plurality of sub-operation types into the operation value corresponding to the first operation type, wherein the phase of the binary number of the operation value corresponding to any sub-operation type is1, and the phase of the binary number of the operation value corresponding to other sub-operation types is 0; performing bit-wise or calculation of binary numbers on the first operation value and the second operation value to obtain a third operation value; and replacing the first operation value in the first operation record by using the third operation value, and obtaining a third operation record after replacement.

In one possible design, the apparatus is further configured to: after the obtaining of the current attribute record of each parent node folder, the method further includes: storing the current attribute records of the first folder and the current attribute records of each father node folder of the first folder into a first storage space and a second storage space of a cache storage space; the storage duration of each attribute record in the first storage space is shorter than that in the second storage space; and transferring the current attribute record of the first folder and the current attribute record of each father node folder of the first folder from the second storage space to the bottom storage space.

In one possible design, the apparatus is further configured to: unloading the current attribute record of the first folder and the current attribute record of each father node folder of the first folder from the second storage space to an underlying storage space, including: in response to the second storage space being in a suspended state, transferring the current attribute records of the first folder and the current attribute records of each parent node folder of the first folder from the second storage space to a third storage space of the cache storage space; wherein the second storage space is not used to save newly generated attribute records during the suspended state; and transferring the current attribute record of the first folder in the second storage space and the current attribute record of each father node folder of the first folder from the third storage space to a bottom storage space.

In one possible design, the apparatus is further configured to: saving the current attribute record of the first folder and the current attribute record of each father node folder of the first folder to a first storage space of a cache storage space, including: storing the current attribute record of the first folder and the current attribute record of each father node folder of the first folder into a first storage space of a cache storage space; and if the storage time of any attribute record in the first storage space is longer than a first preset storage time, deleting any attribute record from the first storage space.

In one possible design, the apparatus is further configured to: in response to the second storage space being in a suspended state, the transferring the current attribute record of the first folder and the current attribute records of the parent node folders of the first folder from the second storage space to a third storage space of the cache storage space includes: responding to the fact that the second storage space is in a suspended state, if the storage time of any attribute record stored in the second storage space is longer than a second preset storage time, all attribute records stored in the second storage space are transferred to a third storage space; and if the total capacity of the attribute records stored in the second storage space is greater than a preset capacity threshold, transferring all the attribute records stored in the second storage space to the third storage space.

In one possible design, the apparatus is further configured to: the historical attribute record of the first folder is obtained by the following method: acquiring a historical attribute record of the first folder from a first storage space of the cache storage space; if the historical attribute record is not acquired from the first storage space, acquiring the historical attribute record from a second storage space of the cache storage space; and if the historical attribute record is not acquired from the second storage space, acquiring the historical attribute record from the bottom-layer storage space.

In a third aspect, the present application provides an electronic device comprising:

a memory for storing a computer program;

and the processor is used for realizing the method steps of the file information statistics when executing the computer program stored in the memory.

In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps of statistics of file information.

For each of the second to fourth aspects and possible technical effects of each aspect, please refer to the above description of the first aspect or the possible technical effects of each of the possible solutions in the first aspect, and no repeated description is given here.

Drawings

FIG. 1 is a schematic diagram of a hierarchical structure of a folder provided herein;

FIG. 2 is a schematic diagram of a document information statistics method provided in the present application;

FIG. 3 is a flowchart of a document information statistics method provided in the present application;

FIG. 4 is a block diagram of a large-scale real-time document information statistics system provided in the present application;

FIG. 5 is a schematic diagram of a document information statistics apparatus provided in the present application;

fig. 6 is a schematic structural diagram of an electronic device provided in the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more clear, the present application will be further described in detail with reference to the accompanying drawings. The particular methods of operation in the method embodiments may also be applied to apparatus embodiments or system embodiments.

In the description of the present application "plurality" is understood to mean "at least two". "and/or" describes the association relationship of the associated object, indicating that there may be three relationships, for example, a and/or B, which may indicate: a exists alone, A and B exist simultaneously, and B exists alone. A is connected with B and can represent: a and B are directly connected and A and B are connected through C. In the description of the present application, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or order.

In order to facilitate the technical solutions provided by the embodiments of the present application to be better understood by those skilled in the art, the following brief descriptions of related terms are provided as follows:

in this embodiment of the present application, attribute records are used to represent statistical information of folders, where the folder attribute records may include a current folder identifier, a previous folder identifier, an attribute set of all files in a next layer of the current folder, and an attribute set of files in all layers of the current folder, and the attribute records of a folder may be represented as { d, p, s, a }, where d is the current folder, p is the previous folder of the current folder, s is the attribute set of all files in a next layer of the current folder, and a is the attribute set of files in all layers of the current folder, and specifically shown in table 1 below:

TABLE 1

The attribute set s of all files in the next layer of the current folder and the attribute set a of all files in the next layer of the current folder may include: the number of dn folders, the number of in pictures, the number of an audio, the number of vn videos, the number of on other type files, the number of un unknown type files, the total size of is pictures, the total size of as audio, the total size of vs videos, the total size of os other type files, and the total size of us unknown type files are specifically shown in the following table 2:

TABLE 2

In order to facilitate those skilled in the art to better understand the technical solutions provided in the embodiments of the present application, the following briefly describes the related application scenarios:

referring to fig. 1, an application scenario related in this embodiment of the application includes, in fig. 1, a plurality of folders are included, and a corresponding association relationship exists between each file, where each folder has a corresponding identifier, for example, d in fig. 1 represents a current folder, p represents a previous-level folder of the current folder, and name represents a name of the current folder, directory information of the folders is composed based on the identifiers d, p, and name, and all folders may be linked in a cascading manner, p of the current folder is d of the upper-level directory, and p of the root folder is 0. Each directory information is stored in a database collection user _ space (user space).

Based on the application scenario, the embodiment of the application provides a method and a device for file information statistics and electronic equipment, and the problem that when massive file information is faced, the time consumed for file information statistics is long is solved.

It is worth to be noted that the technical solution provided in the embodiment of the present application may be applicable to any large-scale file information scenario, and particularly, for a large-scale file information scenario in a platform, the platform may be an AI (Artificial Intelligence) open platform. Specifically, a research and development staff can use the AI open platform to perform operations such as model construction, model training, model conversion, model deployment, and the like, and during the process of performing the operations, the research and development staff generate a large amount of model file information and related sample file information. Based on the above scenario, the technical scheme provided by the embodiment of the application can be used for counting massive file information on an AI open platform, so that real-time counting of the file information is realized, the counting speed of the massive file information is increased, and the experience of a user is improved.

Briefly, referring to fig. 2, an embodiment of the present application provides a technical solution that includes determining an operation manner of a first folder according to target operation records of the first folder generated based on n operations, and then counting current attribute records of the first folder and current attribute records of parent node folders of the first folder by using the operation manner of the first folder. Based on the method, the real-time statistics of the file information can be realized, the attribute information of the file can be rapidly counted, and the technical effect of improving the efficiency of the file information statistics is achieved.

The method provided by the embodiment of the application is further described in detail with reference to the attached drawings.

Referring to fig. 3, an embodiment of the present application provides a method for statistics of file information, which includes the following specific processes:

step 301: acquiring a target operation record generated by the first folder based on n operations, and determining an operation mode of the first folder according to the target operation record;

in an embodiment of the present application, the operation record may include: the folder identifier d, the operation value op, and the generation time t are specifically expressed as { d, op, t }, where the operation value op may be used to characterize the operation type.

The operation types may specifically include three operation categories: update operations, move operations, and delete operations. Further, the update operation may include two classes of operations: update files and update folders. In more detail, the update file may also contain two operation subclasses: adding and deleting files; the update folder comprises two operation subclasses: add folders and delete folders as shown in table 3 below.

TABLE 3

In the embodiment of the present application, different operation types correspond to different operation values, and a setting rule of the operation type corresponding to the operation value is briefly described below.

The operation value of the operation type is set according to binary coding, firstly, the left shift operation is adopted to calculate < <' to obtain the operation value of the operation subclass, the binary form of the operation value corresponding to the operation subclass is calculated according to the phase or the | to obtain the corresponding operation middle class, and then the binary form of the operation value corresponding to the operation middle class is calculated according to the phase or the | to obtain the corresponding operation large class. For the operation classes, if the phase of the binary number of the operation value corresponding to one operation class is1, the phase of the binary number of the operation value corresponding to the other operation class is 0.

In the embodiment of the present application, the operation values of the respective operation types may be set as in table 4 below.

TABLE 4

Based on the correspondence between the operation type and the operation value described above, if the first folder executes an operation, the first folder generates a corresponding operation record according to the operation value corresponding to the operation type of the operation, and optionally, the folder associated with the operation also generates a corresponding operation record.

Specifically, executing each type of operation generates a corresponding operation record in one or more of the following cases, for example.

Adding a file: and adding the file to the folder m, and generating 1 operation record (d: 2, op:1, t: 1648742400) if m is { d:2, p:1, name:' a1 }.

Adding a folder: a folder is added to a folder m, if m is { d:2, p:1, name: "a 1}, 1 operation record { d:2, op:4, t:1648742400} is generated.

Moving the file: move file n to folder m, if n is { d:5, p:6, name: "b 1" }, m is { d:2, p:1, name: "a 1" }, generate 2 records { d:6, op:2, t:1648742400}, { d:2, op:1, t:1648742400 }. Wherein { d:6, op:2, t:1648742400} represents that the upper folder of n performs the operation of deleting the file n, and { d:2, op:1, t:1648742400} represents that the folder m performs the operation of adding the file n.

Moving the folder: folder n is moved to folder m, if n is { d:5, p:6, name: "a 2" }, m is { d:2, p:1, name: "a 1" }, 3 records { d:6, op:8, t:1648742400}, { d:2, op:4, t:1648742400}, { d:5, op:16, t:1648742400}, wherein the records { d:6, op:8, t:1648742400} represent deletion of folder n in the upper layer of folder n, and { d:5, op:16, t:1648742400} represents moving folder n.

And (4) deleting the file: deleting the files in the folder m, and if m is { d:2, p:1, name: "a 1}, generating 1 operation record { d:2, op:2, t:1648742400 }.

Deleting the folder: deleting the folder m, if m is { d:2, p:1, name: "a 1}, generating two records { d:1, op:8, t:1648742400}, { d:2, op:32, t:1648742400}, wherein the record { d:1, op:8, t:1648742400} indicates that the folder m on the previous layer of the folder m executes the operation of deleting the folder m, and the record { d:2, op:32, t:1648742400} indicates that the folder m is deleted.

In this embodiment of the present application, if the first folder generates the second operation record in the preset time period after the first folder generates the first operation record in the preset time period, it is necessary to determine whether a generation time difference between the first operation record and the second operation record is smaller than a preset threshold: if not, replacing the first operation record with the second operation record; and if so, combining the first operation record and the second operation record to generate a third operation record.

Specifically, if the generation time difference between the first operation record and the second operation record is smaller than a preset threshold, a first operation value in the first operation record and a second operation value in the second operation record are determined, then phase or "|" calculation is performed on a binary number of the first operation value and a binary number of the second operation value to obtain a binary number of a third operation value, and then the first operation value in the first operation record is replaced by the third operation value to obtain the third operation record.

For example, folder d1 is at [ T ] ₁ ，T ₂ ]And generating a first operation record { d: d1, op:1, t:1648742400} and a second operation record { d: d1, op:4, t:1648724205} in the time interval, wherein the generation time difference between the first operation record and the second operation record is less than m seconds, and merging the first operation record and the second operation record. Specifically, the binary number of the first operation value "1" is "000001", the binary number of the second operation value "4" is "001000", and in bitwise phase or calculation, if one of the binary numbers of the corresponding two bits is1, the final result bit is 1. Based on this, the binary result obtained by the bit-wise or calculation of the first operation value "000001" and the second operation value "001000" is "001001", that is, the third operation value obtained by bit-wise or combination of the first operation value "1" and the second operation value "4" is 5: 1|4=5, replacing the first operation value "5" in the first operation record with a third operation value "5", generating a third operation record as { d: d1, op:5, t:1648742405}, where t in the third operation record is the generation time of the third operation record.

Further, if the generation time of the third operation record is greater than a preset threshold, or the third operation record is the last generated operation record in a preset time period, the third operation record is taken as the target operation record.

It should be noted that the preset time period may be understood as a forced cache time of each newly generated operation record, the preset threshold may be understood as a merge waiting time of each newly generated operation record, and the preset time period and the preset threshold may be set according to an actual application requirement, and generally, the preset time period is greater than the preset threshold.

After the target operation record is obtained, an operation mode corresponding to the operation type represented by the target operation value in the target operation record is further determined.

Specifically, the operation values corresponding to the target operation values and the operation types of the three operation categories are respectively calculated according to the bit phase and "&", and the operation mode corresponding to the target operation value is determined according to the calculation result.

For example, referring to table 4, if the target operation value is 5, performing a bitwise and operation on the target operation value 5 and the operation value 15 of the update operation, the operation value 16 of the move operation, and the operation value 32 of the delete operation, respectively, in the bitwise and operation, only two bits of the corresponding bits are 1, and the resulting bit is1, otherwise, the resulting bit is 0, based on which the bitwise and operation on the target operation value 5 and the operation value 15 of the update operation result in a first value of 5&15=5, the bitwise and operation on the target operation value 5 and the operation value 16 of the move operation result in a second value of 5&16=0, and the bitwise and operation on the target operation value 5 and the operation value 32 of the delete operation result in a third value of 5&32= 0. Then respectively judging whether the first value 5, the second value 0 and the third value 0 are greater than a preset mode threshold value: if the preset mode threshold is 0, only the first value 5 is greater than the preset mode threshold 0, then the operation type corresponding to the first value 5 is an update operation, and therefore the operation mode corresponding to the operation type represented by the first value 5 is an update operation mode.

When the operation mode of the first folder is determined to be the update operation mode, the following step 302 is performed.

Step 302: when the operation mode of the first folder is the updating operation mode, acquiring a first attribute set of the first folder;

in the embodiment of the application, a first attribute set of a first folder is obtained from a database set user _ space, where the first attribute set includes the latest attribute information of child node files of the first folder, that is, s of the first folder.

The latest attribute information of each folder is stored in the database set user _ space, wherein the attribute information comprises: the attribute information of the current folder d, the previous folder p of the current folder and the attribute set s of all files of the next layer of the current folder is expressed as { d, p, s }.

When the first folder executes any one of the operation types in the above embodiments, the attribute information of the first folder stored in the database collection user _ space is updated in real time correspondingly. For example, folder d2 is moved from folder p1 to folder p2, and the attribute information of folder d2 in the database is updated from { d: d2, p: p1, s: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1} } to { d: d2, p: p2, s: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1} }.

Step 303: if the historical attribute record of the first folder exists, determining a difference attribute set between a first historical attribute set and a first attribute set in the historical attribute record;

the historical attribute record for the first folder is denoted as { d, p, s, a }, and the historical attribute record includes a first set of historical attributes and a second set of historical attributes. The first historical attribute set comprises historical attribute information of child node files of the first folder, namely s in the historical attribute record of the first folder; the second historical attribute set comprises the historical attribute information of all files in the first folder, namely a in the historical attribute record of the first folder.

If the historical attribute record of the first folder exists, determining a difference attribute set between a first historical attribute set and a first attribute set in the historical attribute record.

For example, if the historical attribute record of the first folder is s2= { d: d1, p: p2, s: { dn: dn2, in: in2, an: an2, vn: vn2}, a: { dn: dn2, in: in2, an: an2, vn: vn2, on: on2, un: un2, is: is2, as: as2} }, the first set of attributes of the first folder is s1.s = { dn: dn2, in: in2, an: an2, vn: vn2, the first set of attributes of the first folder is s = { dn: dn2, in: in2, an: an2, vn: vn2}, the second set of historical attributes of the first folder is 2.s = { dn: dn2, in: an2, the second set of historical attributes of the first folder is 72, the first set of the historical attributes of the first folder is s2.s = { dn: 2, the historical set of the historical attributes of the historical attribute records of the historical attribute record of the historical set of the historical attribute record of the historical attribute of the historical set of the historical attribute of the historical index of the.

By the method, the difference attribute set of the first folder can be obtained, and the difference attribute set can be used for updating the attribute records of the first folder and the parent node folder thereof.

Step 304: adding a difference attribute set to a second historical attribute set of the first folder to generate a second attribute set aiming at the first folder, and obtaining a current attribute record of the first folder based on the second attribute set; specifically, if the historical attribute record of the first folder d is s = { d: d, p: p, s: { dn: dn, in: in, an: an, vn: vn }, a: { dn: dn, in: in, an: an, vn: vn, on: on, un: un, is: is, as: as } }, the first set of attributes of the first folder d is s1.s = { dn: dn, in: in, an: an, vn: vn }, i.e. the difference attribute set of the first folder d is Δ = s1.s-s2.s, then the difference attribute set of the first folder d is S2.s = { dn: dn, in: in, an: an, vn: vn } is added with the difference attribute set of the first folder d = s1.s-s2.s, the difference attribute set of the first folder d is = s1. s-s1.s, the difference attribute set of the first folder d is = sin: an, and the difference attribute set of the second folder d is = s1. s-s1. in, in: (sa, s1. s:. s, as:, resulting in the current property record s3 for the first folder d 1.

For a computing device, the process of the first folder d1 obtaining the current property record may be seen in the following steps:

a) modifying s of s1, s1.s = s1.s-s2. s;

b) s of modification s2, s2.s = s2.s + s1. s;

c) a, s2.a = s2.a + s1.s of modification s2.

The current property record s3 for the first folder d1 results.

Step 305: and aiming at each father node folder of the first folder, adding a difference attribute set in the second historical attribute set of each father node folder to generate a respective second attribute set, and obtaining the current attribute record of each father node folder based on the respective second attribute set. After determining the difference attribute set between the first history attribute set and the first attribute set in the history attribute record of the first folder, the current attribute record of each parent node folder can be obtained by adopting the difference attribute set.

Following the example of step 304 above, for a computing device, the process of obtaining the current attribute records of the respective parent node folders of the first folder dl may be seen as follows:

d) and sequentially traversing the parent node folders from the first folder d1 upwards until the p of the parent node folder is less than or equal to 0, and adding the a of all the parent node folders to the s of the s1.

And finally obtaining the current attribute record of each father node folder.

In a possible implementation, if there is no historical attribute record of the first folder, the first attribute set of the first folder is taken as the second attribute set of the first folder; the method comprises the steps of assigning s in current attribute information of a first folder to a in a current attribute record of the first folder, obtaining the current attribute record of the first folder according to a current father node folder identifier of the first folder, a first attribute set and a second attribute set, and then adding the first attribute set in a second historical attribute set of each father node folder aiming at each father node folder of the first folder to obtain the current attribute record of each father node folder.

For example, if the first attribute information s1= { d: d1, p: p1,. s: { dn: dn1, in: in1, an: an1, vn: vn1} } of the first folder d1, i.e. the first set of attributes of the first folder d1 is s1.s = { dn: dn1, in: in1, an: an1, vn: vn1}, since the first folder d1 needs to generate the current attribute record s1= { d, p, s, a }, the first set s1.s { dn: dn1, in: in1, an: an1, vn: vn1} is used here as the current attribute record in the first folder d1, and the first set s = { dn: dn1, in: in1, an: n1, vn: 1} of the current attribute record is further generated, the current attribute information of the folder d1, a, and the current attribute information of the { d: { n: n1, n =, n1, n 72, n, 1, n, 1, is further generated.

Then, for each father node folder of the first folder, adding a first attribute set of the first folder d1 to the second attribute record set a of each father node folder as s1.s, and obtaining the current attribute record of each father node folder.

a) assigning s of s1 to s of a of s3 and s of s3, s3.a = s1.s, s3.s = s1. s;

b) and traversing the parent nodes from the first folder d1 upwards in sequence until p of the parent node folder is less than or equal to 0, and adding a of all the parent node folders to s of s1.

In a possible embodiment, when the operation mode of the first folder is a mobile operation mode, the current parent node folder identifier, the first attribute set and the historical attribute record of the first folder are obtained, the historical parent node folder identifier in the historical attribute record is replaced by the current parent node folder identifier for the first folder to obtain the current attribute record of the first folder, then the first attribute set is added to the first parent node folder identified by the current parent node folder in the second historical attribute set of the historical attribute records of the respective parent node folders of the first parent node folder and the first parent node folder to generate the respective current attribute records, and the second parent node folder identified by the historical parent node folder is added to the second historical attribute set of the respective historical attribute records of the respective parent node folders of the second parent node folder and the second parent node folder, and deleting the first attribute set to generate respective current attribute records.

For example, when the first folder d1 moves from the second parent node folder p1 to the first parent node folder p2, if the historical attribute record of the first folder d1 is s1= { d: d1, p: p1, s: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1, vs: vs1, os: os1, us: us1}, a: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1, vs: as: vs1, vs: vs1, os: us1, us: us1, the current attribute record of the first folder d record may be obtained by the following steps:

a) modifying p of s1 to be a first parent node folder p2, s1.p = p 2;

b) initializing attribute information info _ new = { d: d1, p: p2, s: s1.a }, info _ old = { d: d1, p: p1, s: -s1.a };

c) sequentially traversing the father node folders from the second father node folder p1 upwards until p traversing the father node folders is less than or equal to 0, and adding a of all the father node folders to s of info _ old to obtain the current attribute record of each father node;

d) and sequentially traversing the parent node folders from the first parent node folder p2 upwards until p traversing the parent node folders is less than or equal to 0, and adding a of info _ new to s of all the parent node folders to obtain the current attribute record of each parent node.

In a possible implementation manner, when the operation mode of the first folder is a deletion operation mode, obtaining a historical attribute record of the first folder, clearing a first attribute set and a second attribute set in the historical attribute record for the first folder, modifying a historical parent node folder identifier in the historical attribute record into a deletion identifier to obtain a current attribute record of the first folder, and then deleting the second historical attribute set of the first folder in a second historical set in each historical attribute record of each parent node folder for each parent node folder of the first folder to obtain a current attribute record of each parent node folder.

For example, if the historical attribute record of the first folder d1 is s1= { d: d1, p: p1, s: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1, vs: vs1, os: os1, us: us1}, a: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1, vs: vs1, os: os1, us: us1} } for a computing device, see the following steps for the current process record of the current attribute of the first folder d1 and the parent node record of the first folder d 1:

a) generating s3= { d: d1, p:1, s: { dn:0, in:0, an:0, vn:0, on:0, un:0, is:0, as:0, vs:0, os:0, us:0}, a: { dn:0, in:0, an:0, vn:0, on:0, un:0, is:0, as:0, vs:0, os:0, us:0} };

b) modifying s of s1, s1.s = s3.s-s1. a;

c) and sequentially traversing the parent node folders from the first folder d1 upwards until the p of the parent node folder is less than or equal to 0, and adding the a of all the parent node folders to the s of s1 to obtain the current attribute record of each parent node folder.

By the operation mode, the attribute records of the folder can be counted in real time, and the counting time of the file attribute information is reduced.

Further, after obtaining the current attribute record of the first folder and the current attribute record of each parent node folder of the first folder, the current attribute record of the first folder and the current attribute record of each parent node folder of the first folder need to be stored in a first storage space and a second storage space of a cache storage space, where a storage duration of each attribute record in the first storage space is shorter than a storage duration of the second storage space.

In this embodiment of the present application, the storage form of the attribute record in the first storage space may be a key value pair (key = value), where d of the attribute record is taken as a key, and a specific attribute record is taken as a value. Here, the key functions as an index, and when the user wants to acquire the attribute information of the folder, for example, the attribute information of the folder can be quickly acquired by indexing the key value of the folder.

Specifically, if any attribute record exists in the first storage space and the retention time of the attribute record in the first storage space is longer than a first preset retention time, the attribute record is deleted from the first storage space.

In this embodiment of the present application, the storage form of the attribute record in the second storage space may be a hash structure, all the attribute records are stored in the same hash, key is any fixed character string, field is d of the attribute record, and value is a specific attribute record. In the embodiment of the present application, the key value is used to represent a service identifier, for example, a file statistics service, a transcoding service, and the like.

Specifically, in the second storage space, if the storage time of any one attribute record stored in the second storage space is longer than a second preset storage time, all the attribute records stored in the second storage space are transferred to a third storage space; and if the total capacity of the attribute records stored in the second storage space is greater than a preset capacity threshold, transferring all the attribute records stored in the second storage space to a third storage space.

In summary, the current attribute record of the first folder and the current attribute record of each parent node folder of the first folder are stored in the cache storage space, so that the effect of reducing the storage pressure of the underlying space can be achieved.

Furthermore, the current attribute record of the first folder and the current attribute record of each parent node folder of the first folder can be transferred from the second storage space to the bottom storage space.

Specifically, the current attribute record of the first folder and the current attribute record of each parent node folder of the first folder are transferred from the second storage space to the third storage space of the cache storage space.

In this embodiment of the present application, atomicity exists between the second storage space and the third storage space of the cache storage space, that is, when the second storage space is in a state of transferring the cached current attribute record of the first folder and the current attribute record of each parent node folder of the first folder to the third storage space, the second storage space is in a suspended state, and during the suspended state, the second storage space is not used for saving the newly generated attribute record.

After the second storage space is used for transferring the cached current attribute records of the first folder and the cached current attribute records of each father node folder of the first folder to the third storage space, the third storage space is used for transferring all the received attribute records to the bottom storage space.

By the method, the accuracy of the attribute record stored in the third storage space can be ensured, and the time for unloading the attribute record from the second storage space to the third storage space is less than the time for directly saving the attribute record to the bottom layer module.

Based on the storage space structure, when the history attribute record of the first folder is obtained, the history attribute record can be obtained in the following manner: firstly, acquiring a historical attribute record of a first folder from a first storage space of a cache storage space; if the historical attribute record is not obtained from the first storage space, obtaining the historical attribute record from a second storage space of the cache storage space; and if the historical attribute record is not acquired from the second storage space, acquiring the historical attribute record from the bottom-layer storage space.

By this way of acquisition, the time to acquire the historical attribute record can be reduced.

By the technical scheme provided by the embodiment of the application, the problem that in the prior art, the time consumption of file information is counted in a traversing mode in response to a user request can be solved, the file information is counted in real time, the efficiency of counting the file information is improved, the time for counting the file information is saved, and the user experience is improved.

Further, an embodiment of the present application further provides a system for large-scale real-time statistics of file information, where the system may be a system as shown in fig. 4, and the system may include six modules: the system comprises a user operation module, an operation merging module, an operation caching module, an attribute storage module and an attribute updating module.

The user operation module is used for recording various operations of the folder and generating various operation records; the operation merging module is used for merging various operation records of the same folder; the operation cache module is used for storing the operation records merged by the operation merging module; the attribute caching module is used for temporarily storing the attribute information of the latest updated folder; the attribute storage module is used for recording attribute information of all folders.

Based on the system, the application provides a method for large-scale document information real-time statistics, which can be applied to the system, the inventive concept of the method is the same as that of the information statistics method, and the following describes in detail the method provided by the large-scale document information real-time statistics system according to the embodiment of the application with a specific embodiment.

In the embodiment of the application, the user operation module is used for recording various operations of the folder and generating various operation records. Wherein the operation types include: update operations, move operations, and delete operations. As shown in table 5 below:

TABLE 5

It should be noted that the update operation is an operation type of operation, and it represents that a file or a folder performs a specific update operation, and the performing of the update operation does not represent that attribute information corresponding to the corresponding file or folder is updated synchronously. Wherein, the updating operation specifically comprises: update files, update folders. Updating the files specifically comprises adding the files and deleting the files; updating the folders specifically includes adding folders and deleting folders. As shown in table 6:

TABLE 6

Based on the different operations, operation values corresponding to the different operations are preset, and are specifically shown in table 7:

TABLE 7

Wherein the operation values of adding, deleting, moving and deleting satisfy the left shift operation "<<"rule, the operation values of updating file, updating folder, updating operation are obtained by phase or calculation. For example, the add file operation value is set to op =1<<0, left shift operation is op 1 × 2 ⁰ 1, meaning left shift by 0; the delete file operation value is op =1<<1, according to the left shift operation, op is1 × 2 ¹ 2, left shift by 1.

Different operation records can be generated according to different operation values, and the operation records generated by the file have one or more of the following specific cases, for example.

And (3) deleting the file: deleting the files in the folder m, and if m is { d:2, p:1, name: "a 1}, generating 1 operation record { d:2, op:2, t:1648742400 }.

Deleting the folder: deleting the folder m, if m is { d:2, p:1, name:' a 1}, generating two records { d:1, op:8, t:1648742400}, { d:2, op:32, t:1648742400}, wherein the record { d:1, op:8, t:1648742400} represents that the folder m on the previous layer of the folder m executes the operation of deleting the folder m, and the record { d:2, op:32, t:1648742400} represents that the folder m is deleted.

The different operation records generated by the user operation module are transmitted to the operation merging module. On one hand, the operation merging module caches the received operation records for m seconds and forcibly caches for n seconds, wherein n is greater than m, the refresh retiming can be carried out during the m seconds, and the refresh retiming cannot be carried out during the n seconds. And when the existence time of the operation record exceeds m seconds or n seconds, the overtime operation record is transmitted to the operation caching module. On the other hand, each time the operation merging module receives a new operation record, the new operation record is compared with the historical operation record, when the new operation record is the same as the historical operation record d, the op of the two operation records is merged to generate a new operation record to replace the operation record d before merging, the cache time m is refreshed, and the timing is restarted.

Specifically, the examples of the combination are as follows: there are two operational records: { d:1, op:1, t:1648742400}, { d:1, op:4, t:1648742405}, op merger 1|4=5, and the operation after merger is recorded as { d:1, op:5, t:1648742405 }.

Then, the operation cooperation module transmits the merged operation record to the operation caching module, and the operation caching module caches the operation record transmitted by the operation merging module.

And the attribute updating module reads the operation records from the operation caching module in sequence for consumption, wherein the consumption refers to a process of updating the attribute information of the current folder and all the layer folders on the current folder by adopting the operation records of the current folder. The consumption of different operation records of the same folder is not sequentially distinguished and not interfered with each other, and the statistics of the s field and the a field are separated from each other. The specific operation record consumption method is as follows:

for example, the merged operation record { d: d1, op: n, t: s }, first, whether the merged operation record belongs to an update operation, a move operation or a delete operation is judged according to n: op = op &15>0, then the indication is of an update operation; and/or if op & move.op = op &16>0, then the representation belongs to a move operation; and/or if op & delete. op = op &32>0, then the indication is of a delete operation.

After the operation to which the operation record belongs is definitely merged, corresponding consumption is executed according to the operation to which the operation record belongs, wherein the consumption processes corresponding to the updating operation, the moving operation and the deleting operation are specifically as follows:

updating operation consumption process: in a user _ space, acquiring attribute information of a d1 node after updating, assuming that s1= { d: d1, p: p1, s: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1, vs: vs1, os: os1, us: us1} }, and extracting an attribute record with d being d1 from an attribute caching module or an attribute storage module;

if the record does not exist, indicating a newly added file or folder, the consuming steps are as follows:

a) assigning the value of s after the update of the d1 node to a, then s1= { d: d1, p: p1, s: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1, vs: vs1, os: os1, us: us1}, a: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1, vs: vs1, os: os1, us: us1} };

b) storing s1 in the attribute cache module;

c) and sequentially traversing the parent nodes from the current d1 node upwards until p of the parent node is less than or equal to 0, adding s of s1 to a of all the parent nodes, storing the modified parent node attributes into the attribute cache module, and replacing if the attributes exist.

If a record exists, for example, s2= { d: d1, p: p2, s: { dn: dn3, in: in3, an: an3, vn: vn3, on: on3, un: un3, is: is3, as: as3, vs: vs3, os: os3, us: us3}, a: { dn: dn4, in: in4, an: an4, vn: vn4, on: on4, un: un4, is: is4, as: as4, vs: vs4, os: os4, us: us4} }, the consuming steps are as follows:

a) modifying s of s1, s1.s = s1.s-s2. s;

b) s of modification s2, s2.s = s2.s + s1. s;

c) modify a of s2, s2.a = s2.a + s1. s;

d) storing s2 in the attribute cache module;

e) and traversing the father nodes from the current node upwards in sequence until p of the father node is less than or equal to 0, adding s of s1 to a of all the father nodes, storing the modified father node attribute into the attribute cache module, and replacing if the attribute exists.

A mobile operation consumption process: the move operation will only modify p, d of the moved file or folder and will not change. An attribute record with d1 is fetched from the attribute caching module or the attribute saving module, for example, s1= { d: d1, p: p1, s: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1, vs: vs1, os: os1, us: us1}, a: { dn: dn2, in: in2, an: an2, vn: vn2, on: on2, un: un2, is: is2, as: as2, vs: vs2, os: os2, us: us2} }, the consuming steps are as follows:

a) inquiring that the parent node of the d1 node after movement is p2 in the user _ space, and modifying p of s1, wherein s1.p = p 2;

b) storing s1 in the attribute cache module;

c) initializing attribute information info _ new = { d: d1, p: p2, s: s1.a }, info _ old = { d: d1, p: p1, s: -s1.a };

d) sequentially traversing the father nodes from the p1 node upwards until the p of the father node is less than or equal to 0, adding the a of all the father nodes and the s of info _ old, storing the modified father node attribute into an attribute cache module, and replacing if the attribute exists;

e) and traversing the father nodes from the p2 node in sequence until the p of the father node is less than or equal to 0, adding the info _ new s to the a of all the father nodes, storing the modified father node attribute into the attribute cache module, and replacing if the attribute exists.

And (3) deleting operation consumption process: an attribute record with d1 is fetched from the attribute caching module or the attribute saving module, for example, s1= { d: d1, p: p1, s: { dn: dn1, in: in1, an: an1, vn: vn1, on: on1, un: un1, is: is1, as: as1, vs: vs1, os: os1, us: us1}, a: { dn: dn2, in: in2, an: an2, vn: vn2, on: on2, un: un2, is: is2, as: as2, vs: vs2, os: os2, us: us2} }, the consuming steps are as follows:

a) generating s2= { d: d1, p:1, s: { dn:0, in:0, an:0, vn:0, on:0, un:0, is:0, as:0, vs:0, os:0, us:0}, a: { dn:0, in:0, an:0, vn:0, on:0, un:0, is:0, as:0, vs:0, os:0, us:0} };

b) storing s2 in the attribute cache module;

c) modifying s of s1, s1.s = s2.s-s1. a;

d) and traversing the father nodes from the current node upwards in sequence until p of the father node is less than or equal to 0, adding s of s1 to a of all the father nodes, storing the modified father node attribute into the attribute cache module, and replacing if the attribute exists.

In this embodiment of the present application, the attribute caching module caches the attribute information updated by the attribute updating module, where the attribute caching module may be divided into three layers.

Specifically, the first layer of the attribute caching module is configured to save the attribute record updated by the attribute updating module as a single key-value pair (key = value), where d of the attribute record is used as a key, and a specific attribute record is used as a value. The layer can set the expiration time of each attribute record, automatically clear the attribute records when the attribute records are expired, and reset the expiration time of the modified attribute records if the attribute records are modified within the expiration time, so that the attribute updating module can quickly acquire the required folder attribute records when the folder attributes are updated.

The second layer of the attribute cache module adopts a hash structure, the attribute records updated by the attribute updating module are stored in the same hash, key is any fixed character string, field is d of the attribute records, and value is a specific attribute record. And when the capacity of the attribute records stored in the layer is greater than the defined capacity threshold value and/or the time of the stored attribute records is greater than the defined time threshold value, moving all the updated attribute records stored in the second layer of the attribute cache module to the third layer of the attribute cache module.

The updated attribute records stored in the third layer of the attribute cache module need to be written into the attribute storage module completely. The second layer and the third layer of the attribute cache module have atomicity, and attribute record modification can be prevented when attribute records are deleted after being written into the attribute storage module, so that the attribute records are mistaken or not stored.

When acquiring the attribute record, the attribute updating module acquires the attribute record in the first layer of the attribute caching module, the attribute updating module acquires the attribute record in the second layer if the attribute record cannot be acquired in the first layer, and the attribute updating module acquires the attribute record in the attribute storage module if the attribute record cannot be acquired in the second layer. The attribute caching module can effectively reduce the pressure of the attribute storage module and improve the acquisition speed of the folder attribute records.

The attribute storage module is used for storing the attribute record of the third layer of the attribute cache module, and when p in the attribute record of the third layer is more than or equal to 0, the attribute record of the folder corresponding to the attribute storage module is updated; and when the attribute record p of the third layer is less than 0, deleting the attribute record of the folder corresponding to the attribute storage module. When the attribute of the appointed folder needs to be acquired, the attribute can be quickly acquired in the attribute storage module according to the d value of the folder, so that the real-time statistics of the folder attribute, namely the file information in the folder is achieved.

The method for counting the large-scale file information in real time is set forth above, and the method improves the counting rate of the file information under the folder and achieves the technical effect of counting the file information in real time by combining different operation records of the same folder and updating the attribute information of the current folder and all layers of folders on the current folder according to the combined operation records.

Based on the same inventive concept, the present application further provides an information statistics apparatus, which solves the problem that the time consumption of file information statistics is long when facing massive files, and effectively improves the speed of file information statistics, as shown in fig. 5, the apparatus includes:

an operation mode determining module 501, configured to obtain a target operation record generated by a first folder based on n operations, and determine an operation mode of the first folder according to the target operation record; wherein n is an integer greater than or equal to 1;

an attribute information obtaining module 502, configured to obtain a first attribute set of the first folder when the operation mode of the first folder is the update operation mode; wherein the first set of attributes comprises the latest attribute information of child node files of the first folder;

a determine difference attribute module 503, configured to determine, if there is a historical attribute record of the first folder, a difference attribute set between a first historical attribute set in the historical attribute record and the first attribute set; the historical attribute record comprises a first historical attribute set and a second historical attribute set, the first historical attribute set comprises historical attribute information of child node files of the first folder, and the second historical attribute set comprises historical attribute information of all files under the first folder;

a first adding module 504, configured to add, for the first folder, the difference attribute set to a second historical attribute set of the first folder to generate a second attribute set, and obtain a current attribute record of the first folder based on the second attribute set; wherein the second attribute set comprises current attribute information of all folders under the first folder;

the second adding module 505 is configured to, for each parent node folder of the first folder, add the difference attribute sets to the second history attribute sets of each parent node folder to generate respective second attribute sets, and obtain current attribute records of each parent node folder based on the respective second attribute sets.

In one possible design, the determine operation mode module 501 is further configured to: when the operation mode of the first folder is a mobile operation mode, acquiring a folder identifier of a current father node of the first folder, a first attribute set and a historical attribute record; replacing a history father node folder identifier in the history attribute record with the current father node folder identifier aiming at the first folder to obtain a current attribute record of the first folder; for a first father node folder identified by the current father node folder, adding the first attribute set to a second historical attribute set of respective historical attribute records of the first father node folder and each father node folder of the first father node folder to generate respective current attribute records; and deleting the first attribute set in a second history attribute set of the history attribute record of each father node folder of the second father node folder and the second father node folder aiming at the second father node folder identified by the history father node folder, and generating the current attribute record of each father node folder.

In one possible design, the determine operation mode module 501 is further configured to: when the operation mode of the first folder is a deleting operation mode, acquiring a historical attribute record of the first folder; for the first folder, emptying a first attribute set and a second attribute set in the historical attribute record, and modifying a historical father node folder identifier in the historical attribute record into a deletion identifier to obtain a current attribute record of the first folder; and deleting the second history attribute set of the first folder from the second history set in the history attribute record of each father node folder aiming at each father node folder of the first folder to obtain the current attribute record of each father node folder.

In one possible design, the determining operation mode module 501 is further configured to: acquiring a first operation record and a second operation record generated by a first folder in a preset time period; judging whether the generation time difference between the first operation record and the second operation record is smaller than a preset threshold value or not; if not, replacing the first operation record with the second operation record; if so, merging the first operation record and the second operation record to generate a third operation record; and in response to the generation time of the third operation record being greater than the preset threshold value or the third operation record being the latest generated operation record in the preset time period, taking the third operation record as a target operation record.

In one possible design, the determine operation mode module 501 is further configured to: determining a first operation value in the first operation record and a second operation value in the second operation record; wherein, the operation value is used for representing the operation type of the operation recorded in the operation record; in response to that the phase of the binary number of the operation value corresponding to the first operation type is1, the phase of the binary number of the operation value corresponding to the other operation types is 0; responding to that the first operation type comprises a plurality of sub-operation types, accumulating the operation values corresponding to the plurality of sub-operation types into the operation value corresponding to the first operation type, wherein the phase of the binary number of the operation value corresponding to any sub-operation type is1, and the phase of the binary number of the operation value corresponding to other sub-operation types is 0; performing bit-wise or calculation of binary numbers on the first operation value and the second operation value to obtain a third operation value; and replacing the first operation value in the first operation record by using the third operation value, and obtaining a third operation record after replacement.

In one possible design, the apparatus is further configured to: and unloading the current attribute record of the first folder and the current attribute record of each father node folder of the first folder from the second storage space to the bottom storage space, including: in response to the second storage space being in a suspended state, transferring the current attribute records of the first folder and the current attribute records of each parent node folder of the first folder from the second storage space to a third storage space of the cache storage space; wherein the second storage space is not used to save newly generated attribute records during the suspended state; and transferring the current attribute record of the first folder in the second storage space and the current attribute record of each father node folder of the first folder from the third storage space to a bottom storage space.

In one possible design, the apparatus is further configured to: saving the current attribute record of the first folder and the current attribute record of each father node folder of the first folder to a first storage space of a cache storage space, including: storing the current attribute record of the first folder and the current attribute record of each father node folder of the first folder into a first storage space of a cache storage space; and if the storage time of any attribute record in the first storage space is longer than a first preset storage time, deleting the attribute record from the first storage space.

The device based on the file information statistics can count the file information in real time, solves the problem that the file information statistics consumes a long time when facing massive files, and improves the speed of the file information real-time statistics.

Based on the same inventive concept, an embodiment of the present application further provides an electronic device, where the electronic device can implement the function of the foregoing device for file information statistics, and with reference to fig. 6, the electronic device includes:

at least one processor 601 and a memory 602 connected to the at least one processor 601, in this embodiment, a specific connection medium between the processor 601 and the memory 602 is not limited, and fig. 6 illustrates an example where the processor 601 and the memory 602 are connected through a bus 600. The bus 600 is shown in fig. 6 by a thick line, and the connection manner between other components is only for illustrative purpose and is not limited thereto. The bus 600 may be divided into an address bus, a data bus, a control bus, etc., and is shown with only one thick line in fig. 6 for ease of illustration, but does not represent only one bus or type of bus. Alternatively, the processor 601 may also be referred to as a controller, without limitation to name a few.

In the embodiment of the present application, the memory 602 stores instructions executable by the at least one processor 601, and the at least one processor 601 can execute the file information statistics method discussed above by executing the instructions stored in the memory 602. The processor 601 may implement the functions of the respective units in the apparatus shown in fig. 5.

The processor 601 is a control center of the apparatus, and may connect various parts of the entire control device by using various interfaces and lines, and perform various functions of the apparatus and process data by operating or executing instructions stored in the memory 602 and calling data stored in the memory 602, thereby performing overall monitoring of the apparatus.

In one possible design, processor 601 may include one or more processing units, and processor 601 may integrate an application processor, which primarily handles operating systems, user interfaces, application programs, and the like, and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 601. In some embodiments, the processor 601 and the memory 602 may be implemented on the same chip, or in some embodiments, they may be implemented separately on separate chips.

The processor 601 may be a general-purpose processor, such as a Central Processing Unit (CPU), digital signal processor, application specific integrated circuit, field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like, that may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the file information statistical method disclosed in the embodiments of the present application may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

The memory 602, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory 602 may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charge Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and the like. The memory 602 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 602 in the embodiments of the present application may also be circuitry or any other device capable of performing a storage function for storing program instructions and/or data.

The processor 601 is programmed to solidify the code corresponding to the file information statistics method described in the foregoing embodiment into the chip, so that the chip can execute the steps of the file information statistics method of the embodiment shown in fig. 3 when running. How to program the processor 601 is well known to those skilled in the art and will not be described herein.

Based on the same inventive concept, the embodiment of the present application further provides a storage medium, where the storage medium stores computer instructions, and when the computer instructions run on a computer, the computer is caused to execute the statistical method for the file information discussed above.

In some possible embodiments, the various aspects of the document information statistical method provided by the present application may also be implemented in the form of a program product, which includes program code for causing the control apparatus to perform the steps in the document information statistical method according to various exemplary embodiments of the present application described above in this specification, when the program product is run on a device.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to encompass such modifications and variations.

Claims

1.A method for file information statistics, characterized in that the method comprises:

2. The method of claim 1, wherein the method further comprises:

3. The method of claim 1, wherein after obtaining a target operation record generated by the first folder based on n operations and determining an operation mode of the first folder according to the target operation record, the method further comprises:

4. The method of claim 1, wherein after obtaining a target operation record generated by the first folder based on n operations and determining an operation mode of the first folder according to the target operation record, the method further comprises:

5. The method of claim 1, wherein prior to said obtaining the target operation record generated by the first folder based on the n operations, further comprising:

if not, replacing the first operation record with the second operation record;

6. The method of claim 5, wherein said merging the first operational record and the second operational record comprises:

determining a first operation value in the first operation record and a second operation value in the second operation record; wherein, the operation value is used for representing the operation type of the operation recorded in the operation record;

in response to that the phase of the binary number of the operation value corresponding to the first operation type is1, the phases of the binary numbers of the operation values corresponding to the other operation types are 0;

7. The method of claim 1, wherein after said obtaining the current property record for the respective parent node folder, further comprising:

8. The method of claim 7, wherein said unloading the current property record of the first folder, the current property record of each parent node folder of the first folder, from the second storage space to an underlying storage space comprises:

9. The method of claim 7, wherein saving the current property record of the first folder, the current property record of each parent node folder of the first folder to a first storage space of a cache storage space comprises:

10. The method of claim 8, wherein the dumping the current property record of the first folder, the current property record of each parent node folder of the first folder, from the second storage space into a third storage space of the cache storage space in response to the second storage space being in a suspended state comprises:

11. The method of any of claims 7-10, wherein the historical property record for the first folder is obtained by:

and if the historical attribute record is not acquired from the second storage space, acquiring the historical attribute record from the bottom-layer storage space.

12. An apparatus for statistics of file information, the apparatus comprising:

the attribute information acquisition module is used for acquiring a first attribute set of the first folder when the operation mode of the first folder is an updating operation mode; wherein the first set of attributes comprises the latest attribute information of child node files of the first folder;

the first adding module is used for adding the difference attribute set into a second historical attribute set of the first folder to generate a second attribute set aiming at the first folder, and obtaining a current attribute record of the first folder based on the second attribute set; the second attribute set comprises current attribute information of all folders under the first folder;

13. An electronic device, comprising:

a memory for storing a computer program;

a processor for implementing the method steps of any one of claims 1-11 when executing the computer program stored on the memory.

14. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 11.