CN112579526A - Data processing method, system and device - Google Patents
Data processing method, system and device Download PDFInfo
- Publication number
- CN112579526A CN112579526A CN201910941164.7A CN201910941164A CN112579526A CN 112579526 A CN112579526 A CN 112579526A CN 201910941164 A CN201910941164 A CN 201910941164A CN 112579526 A CN112579526 A CN 112579526A
- Authority
- CN
- China
- Prior art keywords
- data
- target data
- target
- index
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 28
- 238000004458 analytical method Methods 0.000 description 9
- 238000013500 data storage Methods 0.000 description 7
- 230000008878 coupling Effects 0.000 description 5
- 238000010168 coupling process Methods 0.000 description 5
- 238000005859 coupling reaction Methods 0.000 description 5
- 238000007405 data analysis Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000012217 deletion Methods 0.000 description 3
- 230000037430 deletion Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000013479 data entry Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/162—Delete operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure provides a data processing method, system and device, relating to the technical field of electronic information, and capable of solving the problem of low supervision efficiency in the data exchange process. The specific technical scheme is as follows: when the first data is acquired, on one hand, the first data is copied to generate target data and the target data is sent to a file system; on the other hand, the first data is sent to a database; after acquiring the target data, the file system stores the target data into a target file directory according to the type and the identification information corresponding to the target data, and generates an index of the target data according to the target file directory; and finally, generating an index file according to the index of the target data, and searching the target data and the information related to the target data by a user according to the index file. The present disclosure is for data processing.
Description
Technical Field
The present disclosure relates to the field of electronic information technologies, and in particular, to a data processing method, system, and apparatus.
Background
With the rapid development of science and technology, data exchange technology is also rapidly developed, and in order to better supervise data exchange, the prior art supervises data in a log file or database mode. However, when the data exchange is supervised based on the log file of the system, only the data operation record analysis can be realized, and the analysis and the tracing of the data exchange result cannot be realized. When the data is supervised by a database, the exchange result is analyzed and traced according to the data of each batch stored in the data, but a series of problems of low analysis efficiency and low accuracy and the like when a large amount of historical data is faced. Based on the analysis, the problems of low efficiency and low accuracy exist in the processing process of supervision data exchange in the prior art.
Disclosure of Invention
The embodiment of the disclosure provides a data processing method and device, which can solve the problem of low supervision efficiency in a data exchange process. The technical scheme is as follows:
according to a first aspect of embodiments of the present disclosure, there is provided a data processing method, including:
applied to a file system, the method comprises:
acquiring target data and identification information corresponding to the target data;
comparing the target data with the original data, and determining a target type corresponding to the target data in preset types according to a comparison result, wherein the preset types comprise: adding a type, deleting a type and modifying a type;
storing the target data to a corresponding target file directory in a preset file directory according to the target type and the identification information;
and generating an index creating instruction according to the target file directory, wherein the index creating instruction is used for indicating that the index of the target data is created according to the target file directory corresponding to the target data.
In one embodiment, the method of the present disclosure further includes, before acquiring the target data:
acquiring first data, wherein the first data comprises credit data;
copying the first data, generating the target data, and transmitting the target data to a file system;
the first data is stored to a first database or transmitted to a second database.
In one embodiment, before generating the create index instruction, the method of the present disclosure further includes:
and creating a preset index directory according to the preset file directory, wherein the preset file directory is generated according to the identification information, the type and the timestamp corresponding to the historical data.
In one embodiment, after creating the index of the target data, the method of the present disclosure further includes:
and generating an index file according to the index of the target data, wherein the index file comprises historical iteration information corresponding to the target data and classification statistical information corresponding to the target data.
In one embodiment, the disclosed method further comprises:
receiving a user query instruction, wherein the query instruction comprises mark information of target data, and the user query instruction is used for indicating to query the target data or at least one item of historical iterative information classification statistical information corresponding to the target data;
and inquiring the index file according to the inquiry instruction, acquiring the target data or at least one item of historical iterative information classification statistical information corresponding to the target data, and sending the target data or the historical iterative information classification statistical information.
According to the data processing method provided by the embodiment of the disclosure, when first data is acquired, on one hand, the first data is copied to generate target data and sent to a file system; on the other hand, the first data is sent to a database; after acquiring the target data, the file system stores the target data into a target file directory according to the type and the identification information corresponding to the target data, and generates an index of the target data according to the target file directory; and finally, generating an index file according to the index of the target data, and searching the target data and the information related to the target data by a user according to the index file.
According to a second aspect of the embodiments of the present disclosure, there is provided a data processing apparatus including:
the device comprises an acquisition module, a comparison module, a storage module and a creation module;
the acquisition module is used for acquiring target data and identification information corresponding to the target data;
the comparison module is used for comparing the target data with the original data and determining a target type corresponding to the target data in preset types according to a comparison result, wherein the preset types comprise: adding a type, deleting a type and modifying a type;
the storage module is used for storing the target data into a corresponding target file directory in a preset file directory according to the target type and the identification information and generating a newly-added file message;
the creating module is configured to generate an index creating instruction according to the new file message, where the index creating instruction is used to instruct to create an index of the target data according to a target file directory corresponding to the target data.
In one embodiment, the present disclosure further provides the data processing apparatus, which includes a processor and a memory, where the memory stores at least one computer instruction, and the instruction is loaded and executed by the processor to implement the steps performed in the processing method of the data processing method provided in the first aspect and any embodiment of the first aspect.
In one embodiment, the apparatus further includes a computer-readable storage medium, where at least one computer instruction is stored, and the computer instruction is loaded and executed by a processor to implement the steps performed in the data processing method provided in the first aspect and any embodiment of the first aspect.
According to a third aspect of embodiments of the present disclosure, there is provided a data processing system comprising:
the system comprises a first device and a second device, wherein the first device and the second device are connected;
the first device is used for acquiring first data, and the first data comprises credit data;
copying the first data, generating the target data, and transmitting the target data to the second device;
storing the first data to a first database or transmitting the first data to a second database;
the second device is used for acquiring target data and identification information corresponding to the target data through a file system;
comparing the target data with the original data, and determining a target type corresponding to the target data in preset types according to a comparison result, wherein the preset types comprise: adding a type, deleting a type and modifying a type;
storing the target data into a corresponding target file directory in a preset file directory according to the target type and the identification information, and generating a newly added file message;
and generating an index creating instruction according to the newly added file message, wherein the index creating instruction is used for indicating that the index of the target data is created according to the target file directory corresponding to the target data.
In one embodiment, the system further comprises: a third device for controlling the operation of the third device,
the third device is configured to generate an index file according to an index of the target data, where the index file includes historical iteration information corresponding to the target data and classification statistical information corresponding to the target data to obtain the index file;
receiving a user query instruction, wherein the query instruction comprises mark information of target data, and the user query instruction is used for indicating to query the target data or at least one item of historical iterative information classification statistical information corresponding to the target data;
and inquiring the index file according to the inquiry instruction, and acquiring at least one item of historical iterative information classification statistical information corresponding to the target data or the target data.
The embodiment of the disclosure provides a data processing system, when first data is acquired by first equipment, on one hand, the first data is copied to generate target data and sent to second equipment; on the other hand, the first data is sent to a database; after the second device acquires the target data, processing the target data through the file system: storing the target data into a target file directory according to the type and the identification information corresponding to the target data, and generating an index of the target data according to the target file directory; and the third equipment generates an index file according to the index of the target data.
According to the system, after the data are acquired, the data are respectively stored in the first device and the second device, and the programs of data storage and data analysis are separated by adopting distributed deployment, so that the coupling degree is reduced, and the processing efficiency is improved. Meanwhile, a search engine of the target data is established through the third equipment, and the index file is generated, so that the analysis result and the change result of the target data can be quickly and accurately inquired, and the data inquiry efficiency is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 is a flowchart of a data processing method provided by an embodiment of the present disclosure;
fig. 2 is a block diagram of a data processing apparatus provided in an embodiment of the present disclosure;
FIG. 3 is a block diagram of a data processing system according to an embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Example one
An embodiment of the present disclosure provides a data processing method, as shown in fig. 1, where the data processing method is applied to a file system, and includes the following steps:
101. and acquiring target data and identification information corresponding to the target data.
The identification information corresponding to the target data includes name information, service information and the timestamp corresponding to the target data. For example, the target data is business license data of enterprise a, and if the time for acquiring the target data is 09 minutes 09 seconds 09 months 09 days 09 months 2019, the name information in the target data is the name of the business license of enterprise a; the service information in the target data, i.e. the content information of the license, is the timestamp in the target data is 09 minutes and 09 seconds 09.09.09.09.09.09.9.2019.
In an alternative embodiment, the method provided by the present disclosure further includes, before acquiring the target data:
acquiring first data, wherein the first data comprises credit data, and the credit data comprises personal credit information data and market subject credit information data of enterprises, farmer professional cooperative agencies, individual industrial and commercial enterprises and the like.
Copying the first data, generating the target data, and transmitting the target data to a file system;
the first data is stored to a first database or transmitted to a second database.
In view of the above explanation, the method provided by the present disclosure, after acquiring the first data, on one hand, generates target data after copying the first data and sends the target data to the file system, and on the other hand, stores the first data in the first database, so as to process data storage and data analysis in the database and the file system, respectively, thereby reducing the storage pressure of the business database; furthermore, the file system and the first database can be arranged on one device or two devices, so that the data processing efficiency is further improved.
102. And comparing the target data with the original data, and determining a target type corresponding to the target data in preset types according to a comparison result.
The preset type comprises a new type, a modification type, a deletion type and the like.
In an optional embodiment, when comparing the target data with the original data, the method provided by the present disclosure may compare the target data with the original data according to identification information corresponding to the target data, where the identification information includes name information, service information, a timestamp, and other information of the target data; then, original data corresponding to the identification information is searched; and finally comparing the target data with the original data. Further, the query can be performed through a database technology SQL when the original data is searched.
In an optional embodiment, the method provided by the present disclosure further includes creating preset type information, where the preset type information may include a new type, a modified type, and a deleted type; the new type means that the target data is newly added based on the original data; the modification type means that the target data is modified based on the original data, and the deletion type means that a part of the original data is deleted based on the original data; when the target data is of a deletion type, the method provided by the present disclosure may further acquire identification information of the deleted data, for example, data ID information, a name to which the data belongs, and data permission information.
In an optional embodiment, after the target data and the original data are compared, an intermediate temporary table may be created according to a comparison result, and the target data and preset type information corresponding to the target data may be stored.
Further, the intermediate temporary table may be created in a server that stores the target data, or may be stored in another server. The service data and the data of the exchange process are stored separately, so that the storage pressure of a service database is reduced.
103. And storing the target data to a corresponding target file directory in a preset file directory according to the identification information and the target type.
In an optional embodiment, after the target data is stored in the target file directory corresponding to the preset file directory, the method provided by the present disclosure may generate a new file message, where the new file message includes target file directory information and target data corresponding to the new target data.
The preset file directory and the identification information have a mapping relation.
In an alternative embodiment, the method provided by the present disclosure, before generating the create index instruction, further includes:
and creating a preset index directory according to the preset file directory, wherein the preset file directory is generated according to the identification information, the type and the timestamp corresponding to the historical data.
In an alternative embodiment, the method provided by the present disclosure further comprises creating a preset file directory, which may include multiple levels of subdirectories.
When the preset file directory is created, the preset file directory can be created based on user requirements and identification information, types and time stamps corresponding to historical data, for example, the preset file directory is a primary directory based on province to which the data belongs, the preset file directory is a secondary directory based on business to which the data belongs, and the preset file directory is a tertiary directory based on type to which the data belongs.
If province information in the identification information of the target data is Shaanxi province, the business information is mining permit of an enterprise A, and the type of the data is a new type; then the target file directory corresponding to the target data is: a newly added data folder in a mining permission folder in a folder in shaxi province.
The method provided by the disclosure stores the target data into a preset folder, stores the data exchange process data by using a well-defined structure file according to a timestamp file directory, and stores the service data and the exchange process data separately, thereby reducing the storage pressure of a service database
104. And generating an index creating instruction according to the target file directory.
The index creating instruction is used for indicating the target file directory corresponding to the target data to create the index of the target data.
The creating rule of the preset index file may be to create an index directory according to the preset file directory, and create an index of the target data according to a type corresponding to the target data.
In an alternative embodiment, the method provided by the present disclosure may create an index of the target data when a new file message file is received, that is, the target data and the index are mapped.
In an optional embodiment, the method provided by the present disclosure, after creating the index of the target data, further includes:
and generating an index file according to the index of the target data, wherein the index file comprises historical iteration information corresponding to the target data and classification statistical information corresponding to the target data.
The enumerated example illustrates historical iteration information corresponding to the target data, for example, the target data includes credit data of a B enterprise, and then the historical iteration information of the target data may include change process information of the credit data of the B enterprise within a preset time period: credit rating in the credit data of the business B in 2018 was good; the credit rating in the credit data for business B in 2019 was excellent. When the user selects the credit data of the enterprise B, the change process of the credit data of the enterprise B can be checked, and the purpose of supervising data exchange is achieved.
The enumerated examples illustrate classification statistics corresponding to target data, for example, target data includes credit data of enterprise C, credit data of enterprise D, and credit data of individual E, then classification criteria of classification statistics corresponding to target data may be according to enterprise or individual, then target data may be classified into two categories: enterprise information and personal information, wherein the enterprise information comprises credit data of enterprise C and credit data of enterprise D; the personal information includes credit data for the E person.
The index file further includes: data exchange information, which may include: the data exchange information can be obtained according to the comparison result of the target data compared with the original data. If the target data comprises data A, data B and data C, and the original data matched with the target data is data A and data B; according to the comparison result, the data A is the same data, the data B is the same data, the data C is the new data, and according to the preset rule, if only the new data is received, the total data exchange information is that the data exchange at this time is as follows: the data exchange success information comprises the following data A, data B and data C: the data which is successfully exchanged is C data, and the information of data exchange failure is as follows: the data successfully exchanged are data A and data B.
The data exchange information may further generate data quality information, for example, according to a ratio of success to failure of receiving the target data this time. If the data quality information of some data shows that the quality is lower than the preset threshold value, warning information can be generated to prompt the data sending equipment.
In an optional embodiment, after creating the index file, the method provided by the present disclosure further includes:
receiving a query instruction, wherein the query instruction comprises mark information of target data, and the user query instruction is used for indicating to query the target data or at least one item of historical iterative information classification statistical information corresponding to the target data;
and inquiring the index file according to the inquiry instruction, acquiring the target data or at least one item of historical iterative information classification statistical information corresponding to the target data, and sending the target data or the historical iterative information classification statistical information.
The query instruction may include a precise search and a fuzzy search, for example, the user may simply input the name of enterprise a, i.e., the relevant data of enterprise a may be retrieved, and the relevant data may include the data of current enterprise a and the data of historical enterprise a. Further historical a business's data may include historical change times, stored according to data entry timestamps, and detailed information for each change.
According to the method provided by the disclosure, an index directory is created according to a file directory structure, and an index of target data is created according to a target file directory corresponding to the target data. The distributed deployment is adopted, so that the analysis of the data exchange result is separated from the program for recording the exchanged data, the coupling degree is reduced, and the processing efficiency is improved
According to the data processing method provided by the embodiment of the disclosure, when first data is acquired, on one hand, the first data is copied to generate target data and sent to a file system; on the other hand, the first data is sent to a database; after acquiring the target data, the file system stores the target data into a target file directory according to the type and the identification information corresponding to the target data, and generates an index of the target data according to the target file directory; and finally, generating an index file according to the index of the target data, and searching the target data and the information related to the target data by a user according to the index file.
According to the method, after the data are acquired, the data are respectively stored in the database and the file system, and the programs of data storage and data analysis are separated by adopting distributed deployment, so that the coupling degree is reduced, and the processing efficiency is improved. Meanwhile, the index file is generated by establishing a search engine of the target data, so that the analysis result and the change result of the target data can be quickly and accurately inquired, and the data inquiry efficiency is improved.
Example two
Based on the data processing method described in the embodiment corresponding to fig. 1, the following is an embodiment of the apparatus of the present disclosure, which can be used to execute an embodiment of the method of the present disclosure.
The embodiment of the present disclosure provides a data processing apparatus, as shown in fig. 2, the data processing apparatus 20 includes: the method comprises the following steps: the system comprises an acquisition module 201, a comparison module 202, a storage module 203 and a creation module 204;
the obtaining module 201 is configured to obtain target data and identification information corresponding to the target data;
the comparison module 202 is configured to compare the target data with the original data, and determine a target type corresponding to the target data in preset types according to a comparison result, where the preset types include: adding a type, deleting a type and modifying a type;
the storage module 203 is configured to store the target data into a corresponding target file directory in a preset file directory according to the target type and the identification information, and generate a new file message;
the creating module 204 is configured to generate an index creating instruction according to the new file message, where the index creating instruction is used to instruct to create an index of the target data according to a target file directory corresponding to the target data.
In one embodiment, the apparatus of the present disclosure further includes a data processing apparatus, wherein the data processing apparatus includes a processor and a memory, and the memory stores at least one computer instruction, which is loaded and executed by the processor to implement the steps performed in the data processing method according to the first embodiment and any example of the first embodiment.
In one embodiment, the apparatus of the present disclosure further includes a computer-readable storage medium, wherein at least one computer instruction is stored in the storage medium, and the computer instruction is loaded and executed by a processor to implement the steps performed in the data processing method according to the first embodiment and any one of the first to third embodiments.
When first data is acquired, on one hand, the data processing device copies the first data to generate target data and sends the target data to a file system; on the other hand, the first data is sent to a database; after acquiring the target data, the file system stores the target data into a target file directory according to the type and the identification information corresponding to the target data, and generates an index of the target data according to the target file directory; and generating an index file according to the index of the target data.
According to the device, after the data are acquired, the data are respectively stored in the database and the file system, and the programs of data storage and data analysis are separated by adopting distributed deployment, so that the coupling degree is reduced, and the processing efficiency is improved. Meanwhile, the index file is generated by establishing a search engine of the target data, so that the analysis result and the change result of the target data can be quickly and accurately inquired, and the data inquiry efficiency is improved.
EXAMPLE III
Based on the data processing method described in the embodiment corresponding to fig. 1, the following is an embodiment of the system of the present disclosure, which may be used to execute an embodiment of the method of the present disclosure.
An embodiment of the present disclosure provides a data processing system, as shown in fig. 3, the data processing system 30 includes: the first device 301 and the second device 302 are connected, and the connection mode may include a hardware connection and a software connection.
The first device 301 may be a terminal device, a network server, or the like, and the second device 302 may be a terminal device, a network server, or the like.
The first device 301, configured to obtain first data, where the first data includes credit data;
copying the first data, generating the target data, and transmitting the target data to the second device 302;
storing the first data to a first database or transmitting the first data to a second database;
the second device 302 is configured to obtain target data and identification information corresponding to the target data through a file system;
comparing the target data with the original data, and determining a target type corresponding to the target data in preset types according to a comparison result, wherein the preset types comprise: adding a type, deleting a type and modifying a type;
storing the target data into a corresponding target file directory in a preset file directory according to the target type and the identification information, and generating a newly added file message;
and generating an index creating instruction according to the newly added file message, wherein the index creating instruction is used for indicating that the index of the target data is created according to the target file directory corresponding to the target data.
In one embodiment, the present system 30 further comprises: a third device 303, which may be a terminal device or a network server or the like,
the third device 303 is configured to generate an index file according to an index of the target data, where the index file includes historical iteration information corresponding to the target data and classification statistical information corresponding to the target data;
receiving a user query instruction, wherein the query instruction comprises mark information of target data, and the user query instruction is used for indicating to query the target data or at least one item of historical iterative information classification statistical information corresponding to the target data;
and inquiring the index file according to the inquiry instruction, and acquiring at least one item of historical iterative information classification statistical information corresponding to the target data or the target data.
In alternative embodiments, the third device 303 may be the same device as the second device 302, or may not be the same device as the second device 302, that is, the target data and the index file are not stored in the same device, so as to improve the processing efficiency of data and reduce the pressure of data storage.
The embodiment of the disclosure provides a data processing system, when first data is acquired by first equipment, on one hand, the first data is copied to generate target data and sent to second equipment; on the other hand, the first data is sent to a database; after the second device acquires the target data, processing the target data through the file system: storing the target data into a target file directory according to the type and the identification information corresponding to the target data, and generating an index of the target data according to the target file directory; and the third equipment generates an index file according to the index of the target data.
According to the system, after the data are acquired, the data are respectively stored in the first device and the second device, and the programs of data storage and data analysis are separated by adopting distributed deployment, so that the coupling degree is reduced, and the processing efficiency is improved. Meanwhile, a search engine of the target data is established through the third equipment, and the index file is generated, so that the analysis result and the change result of the target data can be quickly and accurately inquired, and the data inquiry efficiency is improved.
Based on the data processing method described in the embodiment corresponding to fig. 1, an embodiment of the present disclosure further provides a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a Read Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The storage medium stores computer instructions for executing the data processing method described in the embodiment corresponding to fig. 1, which is not described herein again.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
Claims (10)
1. A data processing method, applied to a file system, the method comprising:
acquiring target data and identification information corresponding to the target data;
comparing the target data with the original data, and determining a target type corresponding to the target data in preset types according to a comparison result, wherein the preset types comprise: adding a type, deleting a type and modifying a type;
storing the target data to a corresponding target file directory in a preset file directory according to the target type and the identification information;
and generating an index creating instruction according to the target file directory, wherein the index creating instruction is used for indicating that the index of the target data is created according to the target file directory corresponding to the target data.
2. The method of claim 1, prior to acquiring the target data, further comprising:
acquiring first data, wherein the first data comprises credit data;
copying the first data, generating the target data, and transmitting the target data to a file system;
and storing the first data to a first database or transmitting the first data to a second database.
3. The method of claim 2, wherein prior to generating the create index instruction, the method further comprises:
and creating a preset index directory according to the preset file directory, wherein the preset file directory is generated according to the identification information, the type and the timestamp corresponding to the historical data.
4. The method of claim 1, after creating the index of target data, further comprising:
and generating an index file according to the index of the target data, wherein the index file comprises historical iteration information corresponding to the target data and classification statistical information corresponding to the target data.
5. The method of claim 4, further comprising:
receiving a user query instruction, wherein the query instruction comprises mark information of target data, and the user query instruction is used for indicating to query the target data or at least one item of historical iterative information classification statistical information corresponding to the target data;
and inquiring the index file according to the inquiry instruction, acquiring target data or at least one item of historical iterative information classification statistical information corresponding to the target data, and sending the target data or the historical iterative information classification statistical information.
6. A data processing system, comprising: a first device and a second device, wherein,
the first equipment is connected with the second equipment;
the first device is configured to obtain first data, where the first data includes credit data;
copying the first data, generating the target data, and transmitting the target data to a second device;
storing the first data to a first database or transmitting the first data to a second database;
the second device is used for acquiring target data and identification information corresponding to the target data through a file system;
comparing the target data with the original data, and determining a target type corresponding to the target data in preset types according to a comparison result, wherein the preset types comprise: adding a type, deleting a type and modifying a type;
storing the target data into a corresponding target file directory in a preset file directory according to the target type and the identification information, and generating a newly added file message;
and generating an index creating instruction according to the newly added file message, wherein the index creating instruction is used for indicating that the index of the target data is created according to the target file directory corresponding to the target data.
7. The system of claim 6, further comprising: a third device for controlling the operation of the third device,
the third device is used for generating an index file according to the index of the target data, wherein the index file comprises historical iteration information corresponding to the target data and classification statistical information corresponding to the target data to obtain the index file;
receiving a user query instruction, wherein the query instruction comprises mark information of target data, and the user query instruction is used for indicating to query the target data or at least one item of historical iterative information classification statistical information corresponding to the target data;
and inquiring the index file according to the inquiry instruction to obtain at least one item of historical iterative information classification statistical information corresponding to the target data or the target data.
8. A data processing apparatus, comprising: the device comprises an acquisition module, a comparison module, a storage module and a creation module;
the acquisition module is used for acquiring target data and identification information corresponding to the target data;
the comparison module is configured to compare the target data with the original data, and determine a target type corresponding to the target data in preset types according to a comparison result, where the preset types include: adding a type, deleting a type and modifying a type;
the storage module is used for storing the target data into a corresponding target file directory in a preset file directory according to the target type and the identification information and generating a newly-added file message;
and the creating module is used for generating an index creating instruction according to the newly added file message, and the index creating instruction is used for indicating that the index of the target data is created according to the target file directory corresponding to the target data.
9. A data processing apparatus comprising a processor and a memory, said memory having stored therein at least one computer instruction, said instruction being loaded and executed by said processor to perform the steps performed in the data processing method of any one of claims 1 to 5.
10. A computer-readable storage medium having stored thereon at least one computer instruction, which is loaded and executed by a processor to perform the steps performed in the data processing method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910941164.7A CN112579526A (en) | 2019-09-30 | 2019-09-30 | Data processing method, system and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910941164.7A CN112579526A (en) | 2019-09-30 | 2019-09-30 | Data processing method, system and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112579526A true CN112579526A (en) | 2021-03-30 |
Family
ID=75116268
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910941164.7A Pending CN112579526A (en) | 2019-09-30 | 2019-09-30 | Data processing method, system and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112579526A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100145909A1 (en) * | 2008-12-10 | 2010-06-10 | Commvault Systems, Inc. | Systems and methods for managing replicated database data |
| CN102868765A (en) * | 2012-10-09 | 2013-01-09 | 乐视网信息技术（北京）股份有限公司 | Method and system for uploading files |
| WO2015117426A1 (en) * | 2014-07-24 | 2015-08-13 | 中兴通讯股份有限公司 | File management method and device |
| CN105630847A (en) * | 2014-11-21 | 2016-06-01 | 深圳市腾讯计算机***有限公司 | Data storage method as well as data query method, apparatus and system |
| CN106815370A (en) * | 2017-01-24 | 2017-06-09 | 腾讯科技（深圳）有限公司 | Update the methods, devices and systems of resource file |
| CN109710614A (en) * | 2018-12-28 | 2019-05-03 | 深圳市同行者科技有限公司 | A kind of method and device of real-time data memory and inquiry |
| CN110275864A (en) * | 2019-06-11 | 2019-09-24 | 武汉深之度科技有限公司 | Index establishing method, data query method and calculating equipment |
-
2019
- 2019-09-30 CN CN201910941164.7A patent/CN112579526A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100145909A1 (en) * | 2008-12-10 | 2010-06-10 | Commvault Systems, Inc. | Systems and methods for managing replicated database data |
| CN102868765A (en) * | 2012-10-09 | 2013-01-09 | 乐视网信息技术（北京）股份有限公司 | Method and system for uploading files |
| WO2015117426A1 (en) * | 2014-07-24 | 2015-08-13 | 中兴通讯股份有限公司 | File management method and device |
| CN105630847A (en) * | 2014-11-21 | 2016-06-01 | 深圳市腾讯计算机***有限公司 | Data storage method as well as data query method, apparatus and system |
| CN106815370A (en) * | 2017-01-24 | 2017-06-09 | 腾讯科技（深圳）有限公司 | Update the methods, devices and systems of resource file |
| CN109710614A (en) * | 2018-12-28 | 2019-05-03 | 深圳市同行者科技有限公司 | A kind of method and device of real-time data memory and inquiry |
| CN110275864A (en) * | 2019-06-11 | 2019-09-24 | 武汉深之度科技有限公司 | Index establishing method, data query method and calculating equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10824758B2 (en) | System and method for managing enterprise data | |
| US10073919B2 (en) | System and method of search validation | |
| CN110908997A (en) | Data blood margin construction method and device, server and readable storage medium | |
| EP1513065A1 (en) | File system and file transfer method between file sharing devices | |
| CN109740129B (en) | Report generation method, device and equipment based on blockchain and readable storage medium | |
| CN111737227B (en) | Data modification method and system | |
| CN103026334A (en) | Data classification | |
| JP4627539B2 (en) | Load test system, load test data creation method, and program thereof | |
| CN115660610B (en) | Decentralized cooperative office system and method | |
| CN112346950A (en) | Database index performance estimation system and method based on query log analysis | |
| CN113434588B (en) | Data mining analysis method and device based on mobile communication ticket | |
| CN111177481A (en) | User identifier mapping method and device | |
| CN110705297A (en) | Enterprise name-identifying method, system, medium and equipment | |
| US20080033995A1 (en) | Identifying events that correspond to a modified version of a process | |
| CN111932076A (en) | Rule configuration and release method and device and computing equipment | |
| CN112800127B (en) | Data mining analysis method and device based on transaction bill | |
| KR101415528B1 (en) | Apparatus and Method for processing data error for distributed system | |
| CN112579526A (en) | Data processing method, system and device | |
| CN114817340B (en) | Data tracing method and device, computer equipment and storage medium | |
| CN115455059A (en) | Method, device and related medium for analyzing user behavior based on underlying data | |
| CN112446011B (en) | Watermark identification and error code copyright judgment method | |
| CN114371870A (en) | Code scanning and submitting method, code scanning server, client and server | |
| CN110502675B (en) | Voice dialing user classification method based on data analysis and related equipment | |
| CN110689326A (en) | Supplier information auditing and recommending method | |
| CN109582720A (en) | Date storage method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |