CN111611511A - Method for synchronously submitting large data scale list in packaging manner - Google Patents
Method for synchronously submitting large data scale list in packaging manner Download PDFInfo
- Publication number
- CN111611511A CN111611511A CN201910132625.6A CN201910132625A CN111611511A CN 111611511 A CN111611511 A CN 111611511A CN 201910132625 A CN201910132625 A CN 201910132625A CN 111611511 A CN111611511 A CN 111611511A
- Authority
- CN
- China
- Prior art keywords
- data
- submitted
- packaging
- value pair
- key value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004806 packaging method and process Methods 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000006835 compression Effects 0.000 claims abstract description 4
- 238000007906 compression Methods 0.000 claims abstract description 4
- 238000005538 encapsulation Methods 0.000 claims description 10
- 125000004122 cyclic group Chemical group 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 235000014276 Diplazium esculentum Nutrition 0.000 claims description 2
- 244000108321 Diplazium esculentum Species 0.000 claims description 2
- 238000013480 data collection Methods 0.000 claims description 2
- 230000001360 synchronised effect Effects 0.000 claims 6
- 238000004220 aggregation Methods 0.000 claims 1
- 230000002776 aggregation Effects 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 4
- 238000012360 testing method Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method for synchronously packaging and submitting large data sheet lists, which is characterized in that when a user submits form data, the form data is firstly subjected to collective subpackaging, and then the data which cannot be subjected to subpackaging is subjected to front-end compression, so that the data volume which needs to be submitted at one time can be reduced, and the pressure of a server is reduced; when data is received in the background, the compressed data is decompressed according to a fixed set mode, so that the pressure of a server can be avoided, and finally, the problems of data confusion and dirty reading easily generated under the condition of submitting a large-data-volume form are solved by adding a thread synchronization mode to each function processing process.
Description
Technical Field
The invention belongs to the technical field of internet data processing, and particularly relates to a method for synchronously submitting large data volume packaging.
Background
The web page development often encounters the problem of submitting a large amount of data forms, and a common method for submitting a large amount of form data online is to synchronously submit the form data at one time by using a post method and a get method, so that the problem of dirty data reading is often caused, the system is tired, the operation time is long, the system response is slow, the white page phenomenon is more likely to occur in the uploading process, and the system is not responded.
Disclosure of Invention
The invention aims to reduce the bandwidth pressure of a server in the operation process of a system, avoid the problems of overlong time of function operation calculation and real-time concurrency caused by large data volume of a single submitted form, and effectively avoid the problem of dirty reading of data in the data submission process. In order to achieve the above object, the present invention provides a method for synchronously submitting large data volume list packages, which comprises the following steps:
s1, performing collection packaging on the form data to be submitted to obtain n packaging key value pair data collections;
s2, after the encapsulation is finished, putting all the encapsulated key value pair data sets into a form to be submitted;
and S3, judging the size of the form data to be submitted, and synchronously submitting the form data to be submitted meeting the numerical value requirement to a server at one time.
Preferably, the aggregate package in step S1 includes the following steps:
s1.1: carrying out cyclic packaging on the classification of the form data to be submitted to obtain an array set;
s1.2: after the circulation is finished, the server calls a comparison data set of the same type as the array set in the database and puts the comparison data set into an encapsulation key value pair data set of the array set named by the form data type.
Preferably, the aggregate package described in step S1.1 further includes the following steps:
s1.1.1: circularly converting the data type adopted by the data to be submitted into a certain specific key value pair data type;
s1.1.2: after circulation is finished, all the obtained key value pair data sets are compressed, and the key value pair compressed data sets are obtained;
preferably, the cyclic conversion in step S1.1.1 comprises the following steps:
s1.1.1.1: packaging the form data to be submitted into n groups and packaging one by one according to the unit quantity that a certain specific quantity is a group;
s1.1.1.2: and all the form data to be submitted are completely packaged to obtain n data sets.
Preferably, the cyclic encapsulation described in step S1.1 includes the following steps:
s1.1.1: packaging the form data to be submitted into n groups and packaging one by one according to the unit quantity that a certain specific quantity is a group;
s1.1.2: and all the form data to be submitted are completely packaged to obtain n data sets.
Preferably, the key-value pair data types include, but are not limited to, json key-value pair data types.
Preferably, the compression method includes, but is not limited to, the gzip method of pako.
Preferably, the numerical requirements of step S3 are: less than or equal to 1 MB.
Advantageous effects
The implementation of the invention has the following beneficial effects:
when a system user submits form data, the form data is firstly subjected to collective subpackage, and then the data which cannot be subjected to subpackage is subjected to front-end compression, so that the data volume submitted once is reduced, and the bandwidth pressure of a server is greatly reduced; when data are received at the background, compressed data are decompressed according to a fixed set mode, the pressure of a server is avoided, and finally, the mode of adding thread synchronization to the process of processing each function solves the problem of data confusion easily generated under the condition of submitting a large-data-volume form, and meanwhile, the problem of dirty reading of data in the data submission process can be effectively avoided, and the problem of no response of a system caused by the phenomenon of white pages in the data uploading process is solved.
Drawings
FIG. 1 is a schematic data processing flow diagram according to the present invention;
FIG. 2 is a schematic flow chart of an embodiment of the present invention;
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the scope of protection of the present invention.
Example one
And acquiring question information and question type information of the test paper as a common question type according to the test ID20160003 of the test paper submitted by the examinee.
Circularly packaging the answering answers according to the number 8 of the questions contained in the common question type to obtain an array answering answer set;
after circulation is finished, all answer sets of a certain frequently-used question type are put into an answer packaging key value pair data set named by the frequently-used question type to obtain the answer packaging key value pair data set of the frequently-used question type.
The circulation packaging comprises the following steps:
1. according to the difference of the number of questions of the common question type, taking a certain specific number as a unit number of a group;
2. packaging answer answers of a specific number of questions into a group according to the sequence of the questions;
3. packaging the answer of the next group of questions after the packaging of one group of answers is finished;
4. and repeating the step 3 until all answers of all questions are packaged.
After the packaging is finished, packaging key value pair data sets of all answer answers of the frequently-used question types into a form to be submitted;
and judging the size of the form data to be submitted, and submitting the form data to be submitted of which the size is less than or equal to 1MB to a server at one time.
Example two
Acquiring question information and question type information of a test paper as an actual question type according to the test ID20160004 of the test paper submitted by an examinee;
carrying out set packaging on the actual exercise question type answer answers of the examinees according to the question types and 8 question numbers to obtain an actual exercise question type answer packaging key value pair data set;
the set encapsulation comprises the following steps:
1. judging the specific accounting step of each question according to the professional requirements of the financial accounting specialty;
2. acquiring worksheet information adopted in each accounting step;
3. circularly converting the data types of the worksheets adopted in each billing step into json key value pair data types according to the number of the worksheets adopted in each billing step;
(1) a unit quantity in a group of a specific quantity according to the difference of the quantity of worksheets in the billing step;
(2) converting the data types of a group of worksheets according to the sequence of the worksheets;
(3) after the conversion of the data types of one group of worksheets is finished, converting the data types of the next group of worksheets;
(4) step S2.2.3.3 is repeated until all the data types of the worksheets have been converted for this billing step.
4. After circulation is finished, all the worksheet key value pair data sets obtained in the step 3 are compressed through a gzip method, and a worksheet key value pair compressed data set adopted in each bookkeeping step is obtained;
5. putting the worksheet key-value pair compressed data set adopted by each accounting step obtained in the step 4 into answer key-value pair data sets named in each accounting step respectively;
6. and (5) putting the key value pair data sets of the answers of all the accounting steps obtained in the step (5) into the key value pair data set packaged by the answers named by the real exercise questions.
After the packaging is finished, packaging key value pair data sets of all answer answers of the actual exercise question types into a form to be submitted;
and judging the size of the form data to be submitted, and submitting the form data to be submitted of which the size is less than or equal to 1MB to a server at one time.
Example three:
a method for packaging and synchronously submitting large data volume of a financial accounting examination system comprises the following steps:
s1, according to the examination ID of the examination paper submitted by the examinee, obtaining the question information and question type information of the examination paper,
the question types comprise a common question type and an actual exercise question type, and the common question type comprises a single-choice question, a multi-choice question, a blank-filling question, a judgment question and a brief answer question.
S2, collecting and packaging answers of examinees according to different question types and question numbers, when the question types belong to the common question types, circularly packaging the answers according to the number of questions contained in the common question types, and taking a certain specific number as a unit number of a group according to the difference of the number of questions of the common question types; packaging answer answers of a specific number of questions into a group according to the sequence of the questions; packaging the answer of the next group of questions after the packaging of one group of answers is finished; repeating the steps until all answers to the questions are completely packaged; obtaining a plurality of groups of answer sets; after circulation is finished, all answer sets of a certain common question type are put into an answer packaging key value pair data set named by the common question type.
When the question type is an actual exercise question type, judging the specific accounting step of each question according to the professional requirements of the financial accounting profession; acquiring worksheet information adopted in each accounting step; circularly converting the data type of the worksheet adopted in each accounting step into a certain specific key value pair data type according to the number of the worksheets adopted in each accounting step; a unit quantity in which a specific quantity is a group, depending on the number of worksheets in the billing step; converting the data types of a group of worksheets according to the sequence of the worksheets; after the conversion of the data types of one group of worksheets is finished, converting the data types of the next group of worksheets; step S2.2.3.3 is repeated until all the data types of the worksheets have been converted for this billing step.
After circulation is finished, all the obtained worksheet key value pair data sets are compressed to obtain a worksheet key value pair compressed data set adopted by each accounting step; putting the obtained worksheet key-value pair compressed data set adopted by each accounting step into answer key-value pair data sets named in each accounting step respectively; and putting the key-value pair data sets of all the answers of the accounting step obtained in the step S2.2.5 into the key-value pair data set of the answer package named by the real exercise type.
Obtaining answer packaging key value pair data sets of each question type through the steps;
s3, after the encapsulation, putting the key value pair data set of the answer packaging of all question types into the form to be submitted;
and S4, judging the size of the form data to be submitted, and submitting the form data to be submitted, which is less than or equal to 1MB, to the server at one time.
The above embodiments are only for illustrating the invention and are not to be construed as limiting the invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention, so that all equivalent technical solutions also belong to the scope of the invention, and the scope of the invention is defined by the claims.
Claims (8)
1. A method for synchronously submitting large data scale list packages comprises the following steps:
s1, performing collection packaging on the form data to be submitted to obtain n packaging key value pair data collections;
s2, after the encapsulation is finished, putting all the encapsulated key value pair data sets into a form to be submitted;
and S3, judging the size of the form data to be submitted, and synchronously submitting the form data to be submitted meeting the numerical value requirement to a server at one time.
2. The method for encapsulating synchronous large data volume list submissions of claim 1, wherein the set encapsulation of step S1 includes the steps of:
s1.1: carrying out cyclic packaging on the classification of the form data to be submitted to obtain an array set;
s1.2: after the circulation is finished, the server calls a comparison data set of the same type as the array set in the database and puts the comparison data set into an encapsulation key value pair data set of the array set named by the form data type.
3. The method for package-based synchronous submission of large data volume sheets as claimed in claim 2, wherein the package aggregation at step S1.1 further includes the steps of:
s1.1.1: circularly converting the data type adopted by the data to be submitted into a certain specific key value pair data type;
s1.1.2: and after the circulation is finished, compressing all the obtained key value pair data sets, wherein the key value pair compressed data sets.
4. The mass-packaging synchronous submission method for the financial accounting examination system as claimed in claim 3, wherein the circular conversion in step S1.1.1 comprises the following steps:
s1.1.1.1: packaging the form data to be submitted into n groups and packaging one by one according to the unit quantity that a certain specific quantity is a group;
s1.1.1.2: and all the form data to be submitted are completely packaged to obtain n data sets.
5. The method for encapsulating synchronous large data volume sheets for submission of claim 2, wherein the cyclic encapsulation of step S1.1 includes the steps of:
s1.1.1: packaging the form data to be submitted into n groups and packaging one by one according to the unit quantity that a certain specific quantity is a group;
s1.1.2: and all the form data to be submitted are completely packaged to obtain n data sets.
6. The method of claim 1, wherein the key-value pair data types include, but are not limited to, json key-value pair data types.
7. A financial institution examination system large data volume encapsulation synchronous submission method as claimed in claim 6 or 8, in which the compression method includes but is not limited to the gzip method of pako.
8. The method for package-based synchronous submission of the large data volume table as claimed in claim 1, wherein the value requirement of step S3 is: less than or equal to 1 MB.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910132625.6A CN111611511A (en) | 2019-02-22 | 2019-02-22 | Method for synchronously submitting large data scale list in packaging manner |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910132625.6A CN111611511A (en) | 2019-02-22 | 2019-02-22 | Method for synchronously submitting large data scale list in packaging manner |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111611511A true CN111611511A (en) | 2020-09-01 |
Family
ID=72199874
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910132625.6A Pending CN111611511A (en) | 2019-02-22 | 2019-02-22 | Method for synchronously submitting large data scale list in packaging manner |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111611511A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101964152A (en) * | 2010-10-27 | 2011-02-02 | 昆明冶金高等专科学校 | Local area network-based network examination system and implementation method thereof |
| CN102523119A (en) * | 2011-12-16 | 2012-06-27 | 广东东研网络科技有限公司 | EPON network management system data transmission method based on SNMP protocol |
-
2019
- 2019-02-22 CN CN201910132625.6A patent/CN111611511A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101964152A (en) * | 2010-10-27 | 2011-02-02 | 昆明冶金高等专科学校 | Local area network-based network examination system and implementation method thereof |
| CN102523119A (en) * | 2011-12-16 | 2012-06-27 | 广东东研网络科技有限公司 | EPON network management system data transmission method based on SNMP protocol |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3859644A1 (en) | Block chain-based data check system and method, computing device, and storage medium | |
| CN108052665B (en) | Data cleaning method and device based on distributed platform | |
| CN110784419A (en) | Method and system for visualizing professional data of railway electric affairs | |
| CN109947641A (en) | Automated testing method and device | |
| CN107016132B (en) | Online question bank quality improving method and system and terminal equipment | |
| CN110750414B (en) | Mobile data monitoring and analyzing method and device, computer equipment and storage medium | |
| CN111610970A (en) | Method for asynchronously submitting large data scale list package | |
| CN111443969A (en) | Method and device for recording webpage | |
| CN104615765A (en) | Data processing method and data processing device for browsing internet records of mobile subscribers | |
| CN114300084A (en) | Method, device and equipment for automatically generating clinical test case report table | |
| CN109246219A (en) | A kind of working method and system of IoT data collection system | |
| CN110618911A (en) | Data monitoring method and device, storage medium and server | |
| CN101976394A (en) | Data acquiring and counting system and method | |
| CN113744819A (en) | Follow-up state management method and device | |
| CN111611511A (en) | Method for synchronously submitting large data scale list in packaging manner | |
| CN109543772A (en) | Data set automatic matching method, device, equipment and computer readable storage medium | |
| CN101888451A (en) | Telephone voice system test information generation method, server and equipment | |
| CN113948187A (en) | Method and device for calculating nutrient components of food formula, computer and storage medium | |
| EP3869382A3 (en) | Method and device for determining answer of question, storage medium and computer program product | |
| CN110245341B (en) | Identification code batch generation method and device | |
| CN111382336A (en) | Data acquisition method and system | |
| CN108959574A (en) | A kind of test data analysis method and system | |
| CN115509637A (en) | Form-based intelligent filling method, system, equipment and medium | |
| CN101593106A (en) | A kind of evaluation method of programming questions | |
| CN101593107A (en) | A kind of appraisal system of program design topic |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200901 |