CN114091409A - Method, system, equipment and storage medium for distributed asynchronous Excel analysis - Google Patents
Method, system, equipment and storage medium for distributed asynchronous Excel analysis Download PDFInfo
- Publication number
- CN114091409A CN114091409A CN202111387463.4A CN202111387463A CN114091409A CN 114091409 A CN114091409 A CN 114091409A CN 202111387463 A CN202111387463 A CN 202111387463A CN 114091409 A CN114091409 A CN 114091409A
- Authority
- CN
- China
- Prior art keywords
- file
- import
- export
- request
- excel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the field of Excel data analysis, in particular to a distributed asynchronous Excel analysis method, system, equipment and storage medium. The method comprises the following steps: acquiring a file interaction request and feeding back an asynchronous processing identifier; according to the acquired interaction request configuration template information, analyzing the data of the imported file or the exported service information; updating and caching the import and export progress, and storing progress information and a result file; and periodically inquiring the corresponding file import or export progress according to the asynchronous processing identifier, and once the import or export operation is completed, inquiring a file download address of the import or export result so as to download the import or export result file according to the download address. According to the Excel data processing method, the Excel data can be analyzed in a distributed batch mode in an asynchronous processing mode without blocking a processing link, and the processing progress can be checked in real time. The parallel processing capability is increased while the memory consumption of the single machine is saved, the transverse extension cluster is supported, and the user experience is enhanced in an asynchronous data processing mode.
Description
Technical Field
The invention relates to the field of Excel data analysis, in particular to a distributed asynchronous Excel analysis method, system, equipment and storage medium.
Background
There are many Excel parsing tools available in the market, such as Apache poi, jxl, which is well known in Java, such as Openpyxl in Python. However, the common problems of the methods are that the memory consumption is large, and each analysis is to acquire the content of Excel in full quantity and store the content in the memory, so that the maximum number of the input and output at one time is limited by the size of the memory. The Excel data can reach the bottleneck when reaching more than 6w stand-alone server processing. The method for analyzing Excel data in a distributed batch mode can meet the requirement of lateral expansion, and the concurrency of the analyzed Excel data is greatly improved. However, if a synchronization mode is adopted to process a large amount of Excel files, adverse consequences such as poor user experience, even error reporting and overtime can be caused.
Disclosure of Invention
Based on the requirement for fast and efficient processing of a large number of Excel files, the invention provides a method, a system, equipment and a storage medium for distributed asynchronous Excel analysis, aiming at solving the problems of poor user experience, even error reporting overtime and other adverse consequences caused by distributed synchronous batch analysis of Excel data in the prior art.
The invention is realized by adopting the following technical scheme:
a method for distributed asynchronous Excel resolution, the method comprising:
acquiring a file interaction request and feeding back an asynchronous processing identifier, wherein the interaction request comprises an import request or an export request of an Excel file;
configuring template information according to the acquired interactive request, when the interactive request is an import request, analyzing data of an import file according to the import request, setting the quantity of batch analysis, and circularly importing when the quantity of batch analysis is reached; when the interactive request is an export request, downloading a template file according to the export request, and acquiring service information needing to be exported;
updating and caching the import and export progress, storing progress information and a result file, and taking an asynchronous processing identifier as a query mark;
and periodically inquiring the file import or export progress corresponding to the interactive request according to the asynchronous processing identifier, and once the import or export operation is completed, inquiring a file download address of the import or export result so as to download the import or export result file according to the download address.
As a further scheme of the invention, the template information is configured according to the acquired interaction request, and the configuration template information comprises a template name, a template service code, a template file address and a service domain of the configuration template.
Preferably, when the interactive request is an import request, analyzing the data in the import file, returning an analysis import result when the set quantity of batch analysis per time is reached, and circularly sending the next batch analysis request until all analysis data import is completed; and when the import fails, returning a failure reason as a return analysis import result.
Preferably, when the interactive request is an export request, the corresponding service cluster is queried according to the service domain, the template file corresponding to the service is downloaded, and the service information to be exported is acquired.
As a further scheme of the invention, the processing method for updating and caching the import and export progress, saving the progress information and the result file and using the asynchronous processing identifier as the query mark comprises the following steps:
taking the proportion of the processing data amount as an import and export progress;
after the import and export are completed, acquiring result data of the import and export, and writing the result data into an import and export template file to generate a result file;
and uploading a result file, updating the progress information and the address information of the result file into a cache, and taking the asynchronous processing identifier as a query mark.
The invention also comprises a distributed asynchronous Excel analysis system, wherein the distributed asynchronous Excel analysis system adopts the distributed asynchronous Excel analysis method to analyze Excel files in large batch; the distributed asynchronous Excel analysis system comprises a front-end display module, a file processing module, a template configuration module, a service cluster module, a file storage module and a data storage module.
The front-end display module is an interactive inlet of a file processing function and is used for sending an interactive request comprising an import request or an export request of an Excel file to the file processing module;
the template configuration module is used for configuring a template name, a template service code, a template file address or a service domain of a template, and is used for inquiring a corresponding service cluster according to the service domain and downloading a template file corresponding to a service;
the service cluster module is used for providing relevant data of the corresponding service cluster during export and importing the analyzed data into the data storage module during import;
the file storage module is used for providing uploading and downloading of the imported and exported template files and uploading and downloading of the imported and exported result files;
the data storage module is used for storing file processing progress, file addresses of file processing results and business data storage and processing log data storage;
the file processing module is used for acquiring the interactive request sent by the front-end display module, and feeding back an asynchronous processing identifier to the acquired front-end display module to serve as an identifier of the import or export operation of the interactive request; the system is used for configuring template information, inquiring a service cluster, analyzing data of an imported file, exporting service information, uploading result data, updating progress information and result file address information and storing log information.
Preferably, the data storage module includes a buffer and a database, the buffer is used for buffering file processing progress and file processing result file addresses, and the database is used for storing service data and processing log data.
Preferably, the front-end display module is further configured to periodically query the cache for a file import or export progress corresponding to the interaction request according to the asynchronous processing identifier, query the database for service data and processing log data, and query, once the import or export operation is completed, a file download address of the import or export result so as to download the import or export result file according to the download address.
The invention also comprises a distributed asynchronous Excel analyzing device which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the steps of the distributed asynchronous Excel analyzing method when executing the computer program.
The invention also comprises a storage medium storing a computer program which, when executed by a processor, performs the steps of the method of distributed asynchronous resolution Excel.
The technical scheme provided by the invention has the following beneficial effects:
according to the Excel data processing method, the Excel data can be analyzed in a distributed batch mode in an asynchronous processing mode without blocking a processing link, and the processing progress can be checked in real time. The parallel processing capability is increased while the memory consumption of the single machine is saved, the transverse extension cluster is supported, and the user experience is enhanced in an asynchronous data processing mode.
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or technical solutions in the related art, the drawings, which are needed to be used in the description of the exemplary embodiments or related art, will be briefly described below, and are used for providing further understanding of the present invention and are a part of the specification, and together with the embodiments of the present invention, serve to explain the present invention without limiting the present invention. In the drawings:
fig. 1 is a flowchart of a method for asynchronously analyzing Excel in a distributed manner according to an embodiment of the present invention.
Fig. 2 is a flowchart of a process of importing a requested distributed asynchronous analysis Excel in a method for asynchronously analyzing Excel according to an embodiment of the present invention.
Fig. 3 is a flowchart of a distributed asynchronous Excel parsing process of an export request in a method for asynchronously parsing Excel according to an embodiment of the present invention.
Fig. 4 is a system block diagram of a distributed asynchronous analysis Excel system according to an embodiment of the present invention.
Fig. 5 is a block diagram of a distributed asynchronous resolution Excel device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In some of the flows described in the present specification and claims and in the above figures, a number of operations are included that occur in a particular order, but it should be clearly understood that these operations may be performed out of order or in parallel as they occur herein, with the order of the operations being indicated as 101, 102, etc. merely to distinguish between the various operations, and the order of the operations by themselves does not represent any order of performance. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.
The distributed asynchronous Excel analyzing method, system, equipment and storage medium provided by the invention have the advantages that in an asynchronous processing mode, the Excel data can be analyzed in a distributed mode in batches without blocking a processing link, the processing progress can be checked in real time, and the problems of poor user experience, even error reporting timeout and other adverse consequences caused by synchronous processing of Excel files with large data volumes are solved.
The technical solutions in the exemplary embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the exemplary embodiments of the present invention, and it is apparent that the described exemplary embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1:
fig. 1 is a flowchart of a method for distributed asynchronous resolution of Excel according to an embodiment of the present invention. As shown in fig. 1, an embodiment of the present invention provides a distributed asynchronous Excel parsing method, which is applied to a large-batch Excel file parsing process, and includes the following steps:
s1, obtaining a file interaction request and feeding back an asynchronous processing identifier, wherein the interaction request comprises an import request or an export request of the Excel file.
In this embodiment, the configuration template information includes a template name, a template service code, a template file address, and a service domain of the configuration template, and when the interaction request is received, a processing identifier logId is asynchronously returned, and at this time, the processing identifier logId is used as a query flag for querying the analysis progress and the analysis result at a later stage, and a process of synchronously waiting for the processing of the interaction request is not needed, so that the sending of the interaction request is asynchronously performed separately from the analysis progress and the analysis result, and only the progress and the result corresponding to the query are queried through the returned processing identifier logId at regular time, so that the user experience is good, and adverse consequences such as error reporting, overtime and the like cannot occur.
S2, configuring template information according to the obtained interactive request, when the interactive request is an import request, analyzing the data of an import file according to the import request, setting the quantity of batch analysis, and circularly importing when the quantity of batch analysis is reached; and when the interactive request is an export request, downloading the template file according to the export request to acquire the service information needing to be exported.
Referring to fig. 2, fig. 2 is a flowchart of when the interactive request is an import request. When the interactive request is an import request, analyzing the data in the import file, returning an analysis import result when the set quantity of batch analysis per time is reached, and circularly sending the next batch analysis request until all analysis data are imported; and when the import fails, returning a failure reason as a return analysis import result.
Referring to fig. 3, fig. 3 is a flowchart of the interactive request being an export request. When the interactive request is a exporting request, the corresponding service cluster is inquired according to the service domain, the template file corresponding to the service is downloaded, and the service information needing to be exported is obtained.
And S3, updating and caching the import and export progress, saving progress information and a result file, and taking an asynchronous processing identifier as a query mark.
In this embodiment, a processing method for updating and caching an import/export progress, saving progress information and a result file, and using an asynchronous processing identifier as a query token includes:
taking the proportion of the processing data amount as an import and export progress;
after the import and export are completed, acquiring result data of the import and export, and writing the result data into an import and export template file to generate a result file;
and uploading a result file, updating the progress information and the address information of the result file into a cache, and taking the asynchronous processing identifier as a query mark.
S4, regularly inquiring the file import or export progress corresponding to the interactive request according to the asynchronous processing identifier, and inquiring the file download address of the import or export result once the import or export operation is completed so as to download the import or export result file according to the download address.
In this embodiment, during query, a processing identifier logId is asynchronously returned, so that the progress and the result of the analysis processing of the Excel file can be regularly queried, the progress and the result of the analysis processing of the Excel file can be continuously uploaded, and after the analysis processing operation is completed, the import or export result file can be downloaded according to the file download address of the import or export result obtained by query and the download address.
The method for analyzing Excel asynchronously in a distributed mode can analyze Excel data in a distributed mode in a batch mode without blocking a processing link and can check the processing progress in real time. The parallel processing capability is increased while the memory consumption of the single machine is saved, the transverse extension cluster is supported, and the user experience is enhanced in an asynchronous data processing mode.
Example 2:
as shown in fig. 4, in the embodiment of the present invention, a distributed asynchronous Excel parsing system is provided, where the distributed asynchronous Excel parsing system parses Excel files in a large batch by using the distributed asynchronous Excel parsing method described above; the distributed asynchronous Excel analysis system comprises a front-end display module 11, a file processing module 12, a template configuration module 13, a service cluster module 14, a file storage module 15 and a data storage module 16.
The front-end display module 11 is an interactive entry of a file processing function, and is configured to send an interactive request including an import request or an export request of an Excel file to the file processing module 12, and the front-end display module 11 is further configured to periodically query the cache 161 for a file import or export progress corresponding to the interactive request according to an asynchronous processing identifier, query the database 162 for service data and processing log data, and query a file download address of the import or export result once the import or export operation is completed, so as to download the import or export result file according to the download address.
The template configuration module 13 is configured to configure a template name, a template service code, a template file address, or a service domain of a template, so as to query a corresponding service cluster according to the service domain and download a template file corresponding to a service.
The service cluster module 14 is configured to provide relevant data of a corresponding service cluster during export, and import the parsed data into the data storage module 16 during import.
The file storage module 15 is used for providing upload and download of an import/export template file and upload and download of an import/export result file.
The data storage module 16 includes a buffer 161 and a database 162, where the buffer 161 is used to store a file processing progress and a file processing result file address, and the database 162 stores service data and processing log data.
The file processing module 12 is configured to obtain the interaction request sent by the front-end display module 11, and feed back an asynchronous processing identifier to the obtained front-end display module 11, where the asynchronous processing identifier is used as an identifier of an import or export operation of the interaction request; the system is used for configuring template information, inquiring a service cluster, analyzing data of an imported file, exporting service information, uploading result data, updating progress information and result file address information and storing log information.
In this embodiment, according to the difference of the interactive requests, the processing procedure is divided into a distributed asynchronous analysis Excel procedure for importing the request and a distributed asynchronous analysis Excel procedure for exporting the request. Referring to fig. 2, the specific steps of the distributed asynchronous analysis Excel process of the import request are as follows:
1. the front-end display module 11 sends an import request of the Excel file to the file processing module 12;
2. the file processing module 12 returns an asynchronous processing identifier logId to the front-end display module 11;
3. the file processing module 12 analyzes the data of the imported file according to the import request, configures the template name, the template service code, the template file address and the service domain of the template, queries the corresponding service cluster according to the service domain, sets the number of batch analyses at each time, and fetches a number of times (the number of the set batch analyses is obtained by analysis), and circularly imports the service cluster module 14 when the number of the batch analyses is reached, and the service cluster module 14 stores the imported data in the database 162;
4. when the number of batch analysis is reached, the service cluster module 14 returns an analysis import result to the file processing module 12 until all analysis data import is completed, and when import fails, a failure reason is returned as a returned analysis import result;
5. the file processing module 12 uploads the returned analysis import result to the file storage module 15 to form a file storage cluster; meanwhile, the file processing module 12 further continuously buffers the import schedule in the buffer 161 of the data storage module 16, and stores the import schedule or the log in the database 162;
6. the front-end display module 11 may query the cache 162 for an import progress and a query result file address, and once the import operation is completed, may query a file download address for obtaining the import result of this time;
7. downloading the import result file in the file storage module 15 according to the download address;
8. the front-end presentation module 11 may also query the log in the database 161 by an asynchronous process identification logId.
Referring to fig. 3, the specific steps of the distributed asynchronous resolution Excel process for exporting requests are as follows:
1. the front-end display module 11 sends an export request of the Excel file to the file processing module 12;
2. the file processing module 12 returns an asynchronous processing identifier logId to the front-end display module 11;
3. the file processing module 12 downloads the template file in the file storage module 15 according to the export request, acquires the service information to be exported, configures the template name, the template service code, the template file address and the service domain of the downloaded template file from the template configuration module 13, and queries the service cluster;
4. the file processing module 12 fetches a number of times;
5. the service cluster module 14 returns the export result to the file processing module 12;
6. the file processing module 12 uploads the result to the file storage module 15 to form a file storage cluster; meanwhile, the file processing module 12 also continuously buffers the export progress in the buffer 161 of the data storage module 16, and stores the export progress or the log in the database 162;
7. the front-end display module 11 may query the cache 162 for the export progress and the query result file address according to the asynchronous processing identifier logId, and once the export operation is completed, may query for the file download address of the export result;
8. downloading the export result file in the file storage module 15 according to the download address; the front-end presentation module 11 may also query the log in the database 161 by an asynchronous process identification logId.
According to the Excel data processing method, the Excel data can be analyzed in a distributed batch mode in an asynchronous processing mode without blocking a processing link, and the processing progress can be checked in real time. The parallel processing capacity is increased while the memory consumption of a single machine is saved, the transverse expansion of the cluster is supported, and the user experience is enhanced in an asynchronous data processing mode.
As shown in fig. 5, in an embodiment of the present invention, an apparatus 1000 for distributed asynchronous resolution of Excel is provided, the apparatus 1000 includes a memory 1010 and a processor 1020, the memory 1010 stores a computer program, and the processor 1020 is configured to execute the computer program stored in the memory 1010. The memory 1010 is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 1020 to implement the steps of the above-described method embodiments:
acquiring a file interaction request and feeding back an asynchronous processing identifier, wherein the interaction request comprises an import request or an export request of an Excel file;
configuring template information according to the acquired interactive request, when the interactive request is an import request, analyzing data of an import file according to the import request, setting the quantity of batch analysis, and circularly importing when the quantity of batch analysis is reached; when the interactive request is a exporting request, downloading a template file according to the exporting request to acquire service information needing exporting;
updating and caching the import and export progress, storing progress information and a result file, and taking an asynchronous processing identifier as a query mark;
and periodically inquiring the file import or export progress corresponding to the interactive request according to the asynchronous processing identifier, and once the import or export operation is completed, inquiring a file download address of the import or export result so as to download the import or export result file according to the download address.
In an embodiment of the present invention, a storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the above-mentioned method embodiments:
acquiring a file interaction request and feeding back an asynchronous processing identifier, wherein the interaction request comprises an import request or an export request of an Excel file;
configuring template information according to the acquired interactive request, when the interactive request is an import request, analyzing data of an import file according to the import request, setting the quantity of batch analysis, and circularly importing when the quantity of batch analysis is reached; when the interactive request is a exporting request, downloading a template file according to the exporting request to acquire service information needing exporting;
updating and caching the import and export progress, storing progress information and a result file, and taking an asynchronous processing identifier as a query mark;
and regularly inquiring the file import or export progress corresponding to the interactive request according to the asynchronous processing identifier, and inquiring a file download address of the import or export result once the import or export operation is completed so as to download the import or export result file according to the download address.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory.
It should be noted that the technical features in the above embodiments can be arbitrarily combined, and the technical solutions obtained by combining the technical features belong to the scope of protection of the present application. In this document, terms such as "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. A method for analyzing Excel asynchronously in a distributed manner; the method for distributed asynchronous Excel analysis is characterized by comprising the following steps:
acquiring a file interaction request and feeding back an asynchronous processing identifier, wherein the interaction request comprises an import request or an export request of an Excel file;
configuring template information according to the acquired interactive request, when the interactive request is an import request, analyzing data of an import file according to the import request, setting the quantity of batch analysis, and circularly importing when the quantity of batch analysis is reached; when the interactive request is an export request, downloading a template file according to the export request, and acquiring service information needing to be exported;
updating and caching the import and export progress, storing progress information and a result file, and taking an asynchronous processing identifier as a query mark;
and periodically inquiring the file import or export progress corresponding to the interactive request according to the asynchronous processing identifier, and once the import or export operation is completed, inquiring a file download address of the import or export result so as to download the import or export result file according to the download address.
2. The method for distributed asynchronous resolution of Excel according to claim 1, characterized in that: the configuration template information comprises a template name, a template service code, a template file address and a service domain of the configuration template.
3. The method for distributed asynchronous resolution of Excel according to claim 2, characterized in that: when the interactive request is an import request, analyzing the data in the import file, returning an analysis import result when the set quantity of batch analysis per time is reached, and circularly sending the next batch analysis request until all analysis data are imported; and when the import fails, returning a failure reason as a return analysis import result.
4. The method for distributed asynchronous resolution of Excel according to claim 3, characterized in that: and when the interactive request is a derivation request, inquiring the corresponding service cluster according to the service domain to acquire the service information needing to be derived.
5. The method for distributed asynchronous resolution of Excel according to claim 1, characterized in that: the processing method for updating and caching the import and export progress, saving the progress information and the result file and taking the asynchronous processing identifier as the query mark comprises the following steps:
taking the proportion of the processing data amount as an import and export progress;
after the import and export are completed, acquiring result data of the import and export, and writing the result data into an import and export template file to generate a result file;
and uploading a result file, updating the progress information and the address information of the result file into a cache, and taking the asynchronous processing identifier as a query mark.
6. A system for distributed asynchronous Excel resolution is characterized in that: the system for analyzing Excel asynchronously in a distributed mode analyzes Excel files in a large batch mode by adopting the method for analyzing Excel asynchronously in a distributed mode according to any one of claims 1 to 5; the distributed asynchronous Excel analysis system comprises a front-end display module, a file processing module, a template configuration module, a service cluster module, a file storage module and a data storage module;
the front-end display module is an interactive inlet of a file processing function and is used for sending an interactive request comprising an import request or an export request of an Excel file to the file processing module;
the template configuration module is used for configuring a template name, a template service code, a template file address or a service domain of a template, and is used for inquiring a corresponding service cluster according to the service domain and downloading a template file corresponding to a service;
the service cluster module is used for providing relevant data of a corresponding service cluster during export and importing the analyzed data into the data storage module during import;
the file storage module is used for providing uploading and downloading of the imported and exported template files and uploading and downloading of the imported and exported result files;
the data storage module is used for storing file processing progress, file addresses of file processing results and business data storage and processing log data storage;
the file processing module is used for acquiring the interactive request sent by the front-end display module and feeding back an asynchronous processing identifier to the acquired front-end display module to serve as an identifier of import or export operation of the interactive request; the system is used for configuring template information, inquiring a service cluster, analyzing data of an imported file, exporting service information, uploading result data, updating progress information and result file address information and storing log information.
7. The system for distributed asynchronous resolution of Excel according to claim 6 and further comprising: the data storage module comprises a buffer and a database, wherein the buffer is used for buffering the file processing progress and the file processing result file address, and the database is used for storing business data and processing log data.
8. The system for distributed asynchronous resolution of Excel according to claim 7 and further characterized by: the front-end display module is also used for periodically inquiring the file import or export progress corresponding to the interaction request from the buffer according to the asynchronous processing identifier, inquiring the service data and the processing log data from the database, and inquiring the file download address of the import or export result once the import or export operation is completed so as to download the import or export result file according to the download address.
9. A distributed asynchronous Excel parsing apparatus comprising a memory and a processor, said memory storing a computer program, characterized in that said processor when executing said computer program realizes the steps of the method according to any of claims 1 to 5.
10. A storage medium storing a computer program, characterized in that the computer program, when being executed by a processor, realizes the steps of the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111387463.4A CN114091409A (en) | 2021-11-22 | 2021-11-22 | Method, system, equipment and storage medium for distributed asynchronous Excel analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111387463.4A CN114091409A (en) | 2021-11-22 | 2021-11-22 | Method, system, equipment and storage medium for distributed asynchronous Excel analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114091409A true CN114091409A (en) | 2022-02-25 |
Family
ID=80302843
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111387463.4A Pending CN114091409A (en) | 2021-11-22 | 2021-11-22 | Method, system, equipment and storage medium for distributed asynchronous Excel analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114091409A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114448495A (en) * | 2022-03-31 | 2022-05-06 | 四川安迪科技实业有限公司 | Equipment batch adding method and device based on TDMA satellite network management |
| CN114490088A (en) * | 2022-04-01 | 2022-05-13 | 北京锐融天下科技股份有限公司 | Multithreading asynchronous export method and system for large-data-volume excel file |
| CN115495496A (en) * | 2022-08-17 | 2022-12-20 | 昆仑数智科技有限责任公司 | Data export method, device, electronic equipment, storage medium and program product |
| CN117520432A (en) * | 2023-11-30 | 2024-02-06 | 广州方舟信息科技有限公司 | Template-based data transmission method, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140181240A1 (en) * | 2011-03-29 | 2014-06-26 | Michael Scott | System, method and computer program product for templated export of content |
| CN110795920A (en) * | 2019-09-09 | 2020-02-14 | 国网电子商务有限公司 | Document generation method and device |
| CN112788115A (en) * | 2020-12-30 | 2021-05-11 | 银盛支付服务股份有限公司 | Asynchronous processing-based file transmission method and system |
-
2021
- 2021-11-22 CN CN202111387463.4A patent/CN114091409A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140181240A1 (en) * | 2011-03-29 | 2014-06-26 | Michael Scott | System, method and computer program product for templated export of content |
| CN110795920A (en) * | 2019-09-09 | 2020-02-14 | 国网电子商务有限公司 | Document generation method and device |
| CN112788115A (en) * | 2020-12-30 | 2021-05-11 | 银盛支付服务股份有限公司 | Asynchronous processing-based file transmission method and system |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114448495A (en) * | 2022-03-31 | 2022-05-06 | 四川安迪科技实业有限公司 | Equipment batch adding method and device based on TDMA satellite network management |
| CN114448495B (en) * | 2022-03-31 | 2023-06-13 | 四川安迪科技实业有限公司 | Equipment batch adding method and device based on TDMA satellite network management |
| CN114490088A (en) * | 2022-04-01 | 2022-05-13 | 北京锐融天下科技股份有限公司 | Multithreading asynchronous export method and system for large-data-volume excel file |
| CN115495496A (en) * | 2022-08-17 | 2022-12-20 | 昆仑数智科技有限责任公司 | Data export method, device, electronic equipment, storage medium and program product |
| CN117520432A (en) * | 2023-11-30 | 2024-02-06 | 广州方舟信息科技有限公司 | Template-based data transmission method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114091409A (en) | Method, system, equipment and storage medium for distributed asynchronous Excel analysis | |
| US10114682B2 (en) | Method and system for operating a data center by reducing an amount of data to be processed | |
| US10839038B2 (en) | Generating configuration information for obtaining web resources | |
| WO2022126983A1 (en) | Electronic report file export method, apparatus and device, and storage medium | |
| CN111651468B (en) | SQL analysis-based data updating method and device, electronic equipment and storage medium | |
| CN112800095A (en) | Data processing method, device, equipment and storage medium | |
| CN110213380B (en) | Message pushing method and device, computer equipment and storage medium | |
| CN103034735A (en) | Big data distributed file export method | |
| CN105138312A (en) | Table generation method and apparatus | |
| CN111611207B (en) | State data processing method and device and computer equipment | |
| CN111190901A (en) | Business data storage method and device, computer equipment and storage medium | |
| CN108255735B (en) | Associated environment testing method, electronic device and computer readable storage medium | |
| CN112613271A (en) | Data paging method and device, computer equipment and storage medium | |
| CN114153910A (en) | Data acquisition method and device, electronic device and computer program product | |
| CN111258819A (en) | Data acquisition method, device and system for MySQL database backup file | |
| CN114398520A (en) | Data retrieval method, system, device, electronic equipment and storage medium | |
| CN106648722A (en) | Flume receiving side data processing method and device based on big data | |
| CN108874856B (en) | Page generation system, method and server | |
| CN116341514A (en) | File analysis method and device based on dynamic configuration | |
| CN110011845A (en) | Log collection method and system | |
| CN116132448A (en) | Data distribution method based on artificial intelligence and related equipment | |
| CN112699149B (en) | Target data acquisition method and device, storage medium and electronic device | |
| CN113128848A (en) | Data quality monitoring method for full-service index, electronic equipment and storage medium | |
| CN111382379A (en) | Method and terminal for importing configuration data with pictures | |
| CN108846112B (en) | Method and device for drawing data map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |