CN112084196B - Method and system for processing flow data - Google Patents
Method and system for processing flow data Download PDFInfo
- Publication number
- CN112084196B CN112084196B CN202010950926.2A CN202010950926A CN112084196B CN 112084196 B CN112084196 B CN 112084196B CN 202010950926 A CN202010950926 A CN 202010950926A CN 112084196 B CN112084196 B CN 112084196B
- Authority
- CN
- China
- Prior art keywords
- data
- node
- target operation
- nodes
- edges
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
- G06F40/18—Editing, e.g. inserting or deleting of tables; using ruled lines of spreadsheets
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method and a system for processing flow data, wherein the method comprises the steps of receiving to-be-executed form data and target operation information which are sent by user terminal equipment and stored in a mode of a graph, wherein the graph comprises a plurality of nodes and a plurality of edges, and the nodes comprise target operation nodes and data source nodes. The method is developed aiming at the pain points such as difficult processing of form data with large data quantity, strong professional, low automation degree for solving the actual business problem and the like existing in the existing desktop or network form data processing software, and aims to solve the problems of complex data processing, large time consumption and labor consumption in industries such as accounting, telecommunication, banking, sales, scientific research and the like. The method has the characteristics and advantages of no installation, no threshold for data processing, online rapid processing of mass data, user-friendly flow editor, access to the flow editor, strong module function expansibility and the like.
Description
Technical Field
The present application relates to the field of table data processing technologies, and in particular, to a method and a system for processing flow data.
Background
Form data is a type of data that many industries need to perform routine processing analysis. For example, judicial audit timely performs lending behavior restriction and early warning through statistical analysis of the illegal behaviors of the bank card, discovers illegal economic problem clues and the like; the sales industry counts sales performance and the like of each company of each department, and market analysis obtains high-profit commodity information and the like by performing statistical analysis on commodity inventory sales conditions of a market and timely supplementing a commodity source.
For some industry form data processing services, existing desktop or web form data processing software or platforms (e.g., excel) have many limitations:
1. it is difficult to handle complex table formats. The form data of each business department has a specific format, such as the row of the header, the statistical row of the end, and the like, and is difficult to process by adopting the traditional Excel and other software.
2. It is difficult to handle a large data amount of tabular data processing traffic. When the data size of Excel is too large, the query and calculation speed of Excel is significantly reduced, even if the Excel is extremely easy to be blocked. Such as: in a small supermarket, 10 cashier receives 200 goods per person every day, excel efficiency is extremely reduced, and tasks are difficult to complete.
3. Has strong specialization and high threshold. The use of many existing functions requires a deeper specialized learning of the software, such as applying perspective (VLOOKUP), summation (SUM), AVERAGE (AVERAGE), maximum (MAX), minimum (MIN) etc. functions that cannot be used by people who have not learned such functions, programming is more headache for most people.
4. The degree of automation is not high. Each business analysis requires a user to design a calculation flow according to specific business pertinence, and various functions are manually called and combined.
5. Time and effort are consumed. Some works are difficult or impossible to process by adopting traditional software, and are manually compared or copied one by one, so that accuracy cannot be ensured, and a large amount of time and labor cost are consumed.
6. The operation is repeated periodically. Some businesses are periodic, such as daily, weekly, monthly statistics, etc., and traditional approaches require manual processing that is repeated each time we task, but in fact the business and manner of processing are consistent each time.
Disclosure of Invention
The application provides a method and a system for processing flow data. The method can replace complex Excel formulas and programming to perform data analysis processing, and can solve the problem that Excel cannot be processed or needs strong expertise to be processed.
The application provides the following scheme:
a method of processing data in a flow, comprising:
receiving to-be-executed form data and target operation information which are sent by user terminal equipment and stored in a mode of a graph, wherein the graph comprises a plurality of nodes and a plurality of edges, the nodes comprise target operation nodes and data source nodes, and the edges are used for pointing to one node from the other node and storing data flow information;
traversing all nodes to determine a starting node, wherein the starting node is the data source node;
reading form data contained in the starting node;
traversing all the edges to determine all edges related to the starting node;
determining a target operation node related to the starting node according to the determined all edges;
determining an execution algorithm according to the target operation information contained in the target operation node, and executing the execution algorithm to generate an execution result;
and returning the execution result to the user terminal equipment so that the user terminal equipment can display the execution result.
Preferably: the target operation information comprises an operation mode and/or an operation object.
Preferably: the operation mode and/or the operation object are determined by the user terminal equipment according to the result input by the user.
Preferably: and reading the table data contained in the starting node from a hard disk, and reading target operation information from the target operation node.
Preferably: determining a target operation node related to the starting node according to the determined all edges, including: and traversing all the determined edges to judge whether the starting node is the starting point of all the determined edges, and if so, judging that the node corresponding to the respective end point of all the determined edges is the target operation node related to the starting node.
Preferably: the starting node comprises two data which respectively store first form data and second form data, wherein the first form data comprises bank flow data, and the bank flow data comprises names, identity card numbers, amounts and dates; the second table data comprises personnel list data, wherein the personnel list data comprises names, identity card numbers and work units; the target operation node comprises target operation information of personnel records contained in the second table data in the first table data; the execution algorithm comprises the following steps:
reading all records of name fields contained in the second table data to obtain a list of people to be searched;
comparing the names of the people in the personnel list to be searched with the names contained in the first table data one by one, judging whether the names of the people in the personnel list to be searched appear in the first table data to obtain a matching result, if so, recording the matching result as successful, and if not, recording the matching result as failed;
the execution result includes the matching result.
Preferably: and storing the matching result in a table mode and returning the matching result to the user terminal equipment.
Preferably: the start node includes time table data for a time field; determining a theoretical interval time between uplink and downlink of the time field; the target operation node comprises target operation information for searching for the vacant time; the execution algorithm comprises the following steps:
reading all the time in the time table data, and calculating from the first row;
judging whether the interval time between the next line and the previous line is the theoretical interval time, if so, continuing the next line, and if the interval time is larger than the theoretical interval time, adding the theoretical interval time to the time of the current line to obtain the insertion time, storing the insertion time into a result list, and taking the insertion time as the time value of the current line; circularly executing the operation until the last row is calculated to be ended;
and all the insertion time contained in the result list is the vacant time to be searched.
Preferably: the table data to be executed is any one or a combination of a plurality of xlsx format data, xls format data and csv format data.
A streaming data processing system for use with a server, the system comprising:
the receiving unit is used for receiving to-be-executed form data and target operation information which are sent by user terminal equipment and stored in a form of a graph, wherein the graph comprises a plurality of nodes and a plurality of edges, the nodes comprise target operation nodes and data source nodes, and the edges are used for pointing to one node to the other node and storing data flow direction information;
the starting node determining unit is used for traversing all the nodes to determine a starting node, wherein the starting node is the data source node;
a data reading unit, configured to read table data included in the start node;
an edge determining unit, configured to traverse all the edges to determine all edges related to the start node;
a target operation node determining unit, configured to determine a target operation node related to the start node according to the determined all edges;
the execution unit is used for determining an execution algorithm according to the target operation information contained in the target operation node and executing the execution algorithm to generate an execution result;
and the result returning unit is used for returning the execution result to the user terminal equipment so that the user terminal equipment can display the execution result.
According to the specific embodiment provided by the application, the application discloses the following technical effects:
according to the application, a method and a system for processing flow data can be realized, and in one implementation mode, the method can comprise the steps of receiving to-be-executed form data and target operation information which are sent by user terminal equipment and stored in a graph mode, wherein the graph comprises a plurality of nodes and a plurality of edges, the nodes comprise target operation nodes and data source nodes, and the edges are used for pointing to one node from the other node and storing data flow information; traversing all nodes to determine a starting node, wherein the starting node is the data source node; reading form data contained in the starting node; traversing all the edges to determine all edges related to the starting node; determining a target operation node related to the starting node according to the determined all edges; determining an execution algorithm according to the target operation information contained in the target operation node, and executing the execution algorithm to generate an execution result; and returning the execution result to the user terminal equipment so that the user terminal equipment can display the execution result. The method is developed aiming at the pain points such as difficult processing of form data with large data quantity, strong professional, low automation degree for solving the actual business problem and the like existing in the existing desktop or network form data processing software, and aims to solve the problems of complex data processing, large time consumption and labor consumption in industries such as accounting, telecommunication, banking, sales, scientific research and the like. The method has the characteristics and advantages of no installation, no threshold for data processing, online rapid processing of mass data, user-friendly flow editor, access to the flow editor, strong module function expansibility and the like.
Of course, it is not necessary for any one product to practice the application to achieve all of the advantages set forth above at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for processing data in a flow chart according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a data processing system according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which are derived by a person skilled in the art based on the embodiments of the application, fall within the scope of protection of the application.
Example 1
Referring to fig. 1, a method for processing data in a flow manner according to a first embodiment of the present application, as shown in fig. 1, is applied to a server, and includes:
s101: receiving to-be-executed form data and target operation information which are sent by user terminal equipment and stored in a mode of a graph, wherein the graph comprises a plurality of nodes and a plurality of edges, the nodes comprise target operation nodes and data source nodes, and the edges are used for pointing to one node from the other node and storing data flow information; the target operation information comprises an operation mode and/or an operation object. The operation mode and/or the operation object are determined by the user terminal equipment according to the result input by the user. The table data to be executed is any one or a combination of a plurality of xlsx format data, xls format data and csv format data. The data source refers to a file uploaded to a server, supports xlsx, xls, csv format, each data is called a data source, and the data source can also refer to a data source node dragged into a working environment. The subsequent step can also support database connection, namely, the table of each database can be used as a data source to carry out flow analysis calculation in the same way. A node refers to an operation in a flow, and both a data source and a calculation method can be nodes, and a plurality of nodes are connected in series to form a flow. The process of connecting two nodes by a line (edge) with a directional arrow is the connection. The arrow indicates the direction of data flow, the starting node must start from the data source, and the system will calculate from the data source node to the last node in connection order.
S102: and traversing all the nodes to determine a starting node, wherein the starting node is the data source node.
S103: reading form data contained in the starting node; and reading the table data contained in the starting node from a hard disk, and reading target operation information from the target operation node. The hard disk is a server hard disk.
S104: traversing all the edges to determine all edges related to the starting node; it should be noted that the edges provided by the present application are all directed edges, i.e. one edge includes a start point and an end point. The edges connected with the starting node are all the edges related to the starting node, and the starting node is the starting point of the edge.
S105: determining a target operation node related to the starting node according to the determined all edges; specifically, traversing all the determined edges to determine whether the starting node is a starting point of all the determined edges, if yes, the node corresponding to each end point of all the determined edges is the target operation node related to the starting node. It can be understood that the number of the target operation nodes to be found may be one or more, and when a plurality of target operation nodes are found, calculation is performed one by one.
S106: determining an execution algorithm according to the target operation information contained in the target operation node, and executing the execution algorithm to generate an execution result; the execution algorithm is edited by a developer and stored in a hard disk of the server, and the execution algorithm is directly called when the execution algorithm is needed, so that the user does not need to edit the execution algorithm.
S107: and returning the execution result to the user terminal equipment so that the user terminal equipment can display the execution result.
The method provided by the application can provide a corresponding operation interface for the user terminal equipment, and a user can realize online procedural chain type processing of arbitrary complex form data (xlsx, xls, csv, dat and the like) only by dragging a mouse without applying formulas and programming, so that the difficult problems of manual complexity, repeatability, periodicity, complexity and the like in data processing are solved, and the method has the functions of data source management, procedure editing, result output and the like.
The method is developed aiming at the pain points such as difficult processing of form data with large data quantity, strong professional, low automation degree for solving the actual business problem and the like existing in the existing desktop or network form data processing software, and aims to solve the problems of complex data processing, large time consumption and labor consumption in industries such as accounting, telecommunication, banking, sales, scientific research and the like. The method has the characteristics and advantages of no installation, no threshold for data processing, online rapid processing of mass data, user-friendly flow editor, access to the flow editor, strong module function expansibility and the like.
Specifically, the user inputs the to-be-executed form data and the target operation information through the user terminal device, and the user terminal device stores the to-be-executed form data and the target operation information which are needed to be executed by the user in a Graph mode. The Graph is divided into nodes (Node) and edges (Edge), wherein the Node stores operation information such as connection, aggregation and the like, and relevant parameters of the operation such as fields of connection and the like; edge has a direction, i.e. from one Node to another Node, where Edge stores data flow information, e.g. a table performs connection first and then aggregation.
The stored Graph is transmitted to a server, and the server analyzes and executes the Graph after obtaining the Graph, and the specific steps are as follows:
1. the method comprises the steps of finding out initial nodes from all nodes, wherein the initial nodes are data source nodes (in which table information is stored, storage positions, table heads are in rows and the like), the initial nodes are the start of one-time flow calculation, the system starts follow-up calculation from the initial nodes, the initial conditions of calculation are the initial conditions of calculation, and the mode of determining the initial nodes is to find out data nodes, namely, the nodes containing excel and other data are found out by circulating through all nodes.
2. The table data is read according to the Node configuration (data is read from the hard disk if it is a data source Node, and (calculation result) is read from the memory if it is an operation Node).
3. And finding the Edge related to the Node from all the edges.
4. Finding another Node from the Edge, wherein the Node is the Node to be executed (the searching mode of the two steps is to circularly traverse all the Edge, judging whether the starting Node of one Edge is the Node, if so, the ending Node is the Node to be searched).
5. Executing the Node, wherein the executing parameters need to be read, and the specific executing steps are as follows: checking the operation names stored in the Node, if the Node is connected, the program automatically executes the connection operation, the connection operation needs parameters such as two data and a connection field name, the two data are respectively the calculation results of two nodes at the other end of the two edges connected with the two data, and the parameters are taken out from the Node.
6. And executing 2-5 steps circularly until all nodes are executed.
7. And saving the last executed node to obtain a final execution result.
The implementation of the method provided by the application will be described below with specific examples.
The data names of the tables to be executed shown in the tables 1 and 2 are two data sources of Chinese score and mathematic score, and the field compositions are the number and score respectively. The target operation information is a connection field with the 'number' as two data, and the number, the Chinese score and the mathematic score are connected to generate new data.
TABLE 1
| A | B | |
| 1 | Number of school | Chinese score |
| 2 | 1 | 80 |
| 3 | 2 | 79 |
| 4 | 3 | 78 |
TABLE 2
| A | B | |
| 1 | Number of school | Mathematical performance |
| 2 | 1 | 90 |
| 3 | 2 | 89 |
| 4 | 3 | 88 |
The first step, newly-built flow is named, and the newly-built flow is: after receiving the operation request sent by the user terminal equipment, the server provides an operation interface for the user terminal equipment so as to display the operation interface on the user terminal equipment, and certainly, the operation interface information can also be stored on the user terminal equipment, so that the operation interface can be provided for the user at the user terminal equipment. The user clicks on the flow menu through the operation interface to create a new flow, named in this example as quick start of analysis.
And step two, uploading data. The user clicks the upload data to upload the data to be processed. Tables 1 and 2 are uploaded.
And thirdly, setting data parameters. And the user clicks and switches to the data menu in the operation interface, and clicks and views the parameter setting interface. There is a need to set a header, ignore the number of rows, and delimiter. And finally determining target operation information through parameters input by a user.
And step four, receiving the data of the table to be executed and the target operation information which are sent by the user terminal equipment and stored in a picture mode.
And fifthly, determining that the initial node comprises two data sources of Chinese score and mathematic score, wherein the target operation node is target operation information of the two data sources, namely, a 'number' is used as a connection field of the two data, and connecting the number, the Chinese score and the mathematic score to generate new data.
And step six, calling a pre-stored execution algorithm in an execution algorithm database according to the target operation information, wherein the specific execution algorithm can be set in various modes, and finally, the execution result required by the target operation information can be obtained. And a table containing correspondence relation of the number, the language score, and the mathematical score is calculated and obtained as shown in table 3.
TABLE 3 Table 3
| Index | Number of school | Chinese language | Mathematics |
| 0 | 1 | 80 | 90 |
| 1 | 2 | 79 | 89 |
| 2 | 3 | 78 | 88 |
And seventh, returning the table containing the corresponding relation of the academic score, the Chinese score and the mathematical score to the user terminal equipment, and displaying the table in the user terminal equipment.
Therefore, one analysis operation is completed, if the result data is required to be continuously operated, the corresponding operation is only required to be dragged into the same working area, and the connecting node is connected with the newly dragged operation in the same mode.
In practical application, in order to facilitate calculation and ensure that accurate calculation results are obtained at the same time, the embodiment of the application can also provide a plurality of execution algorithms for calculating different execution results. Specifically, the initial node comprises two initial nodes and respectively stores first form data and second form data, the first form data comprises bank flow data, and the bank flow data comprises names, identity card numbers, amounts and dates; the second table data comprises personnel list data, wherein the personnel list data comprises names, identity card numbers and work units; the target operation node comprises target operation information of personnel records contained in the second table data in the first table data; the execution algorithm comprises the following steps:
and reading all records of name fields contained in the second table data to obtain a list of people to be searched.
And comparing the names of the people in the personnel list to be searched with the names contained in the first table data one by one, judging whether the names of the people in the personnel list to be searched appear in the first table data to obtain a matching result, if so, recording the matching result as successful, and if not, recording the matching result as failed.
The execution result includes the matching result. And storing the matching result in a table mode and returning the matching result to the user terminal equipment.
Specifically, there are two excel files, if the first excel is the running data provided by a certain bank, including name, identification number, amount, date, etc., and the second excel is a person list including name, identification number, work unit, etc., it is desired to find the person record included in the second excel in the first excel, i.e. see which persons in the second excel have the running data record in the first excel. Then our matching operation can be used, with the specific execution algorithm as follows:
1. and reading all records of the name field, namely all personnel lists to be searched, from the second excel.
2. And comparing the first excel line by line, judging whether the name of the person in the current line of the flow record appears in the list, if so, recording the name as successful matching, and if not, recording the name as failed matching.
3. And saving the result as a new excel, namely the execution result.
The execution algorithm may further include time table data in which the start node includes a time field; determining a theoretical interval time between uplink and downlink of the time field; the target operation node comprises target operation information for searching for the vacant time; the execution algorithm comprises the following steps:
all times in the time table data are read out, and the calculation is started from the first row.
Judging whether the interval time between the next line and the previous line is the theoretical interval time, if so, continuing the next line, and if the interval time is larger than the theoretical interval time, adding the theoretical interval time to the time of the current line to obtain the insertion time, storing the insertion time into a result list, and taking the insertion time as the time value of the current line; the above operation is cyclically performed until the end of the last line is calculated.
And all the insertion time contained in the result list is the vacant time to be searched.
Specifically, the find free time operation is used to find missing items in a fixed time interval record. If an excel data includes a time field, the values of the time field are theoretically arranged at intervals of 10 seconds between the upper row and the lower row, and if the first row is 8:00:00, the second row is 8:00:10, and the values of the time field are arranged downwards in sequence. But for some reason some of the rows are missing or not collected, i.e. the interval between the rows up and down is 20 seconds or 30 seconds, etc. This operation can be used to find lines of time slots at this time, the system calculation process is as follows:
1. reading out all time in excel, and calculating from the first row;
2. and judging whether the next row is separated from the previous row by 10 seconds, if so, continuing the next row, if so, adding 10 seconds to the time of the current row, storing the value into a result list, and taking the value as the time value of the current row.
3. The second step is performed in a loop until the end of the last line is calculated.
4. All times in the results list are the missing times in excel for this purpose.
Example two
Corresponding to the method for processing the flow data provided in the first embodiment of the present application, the second embodiment of the present application further provides a system for processing the flow data; referring to fig. 2, applied to a server, the system includes:
a receiving unit 201, configured to receive to-be-executed table data and target operation information sent by a user terminal device and stored in a graph manner, where the graph includes a plurality of nodes and a plurality of edges, the nodes include a target operation node and a data source node, and the edges are configured to point from one node to another node and store data flow information.
The starting node determining unit 202 is configured to determine a starting node by traversing all nodes, where the starting node is the data source node.
And a data reading unit 203, configured to read the table data contained in the start node.
An edge determining unit 204, configured to traverse all the edges to determine all edges related to the start node.
A target operation node determining unit 205, configured to determine a target operation node related to the start node according to the determined edge.
And the execution unit 206 is configured to determine an execution algorithm according to the target operation information included in the target operation node, and execute the execution algorithm to generate an execution result.
And a result returning unit 207, configured to return the execution result to the user terminal device, so that the user terminal device displays the execution result.
For system or apparatus embodiments, the description is relatively simple as it is substantially similar to method embodiments, with reference to the description of method embodiments in part.
From the above description of embodiments, it will be apparent to those skilled in the art that the present application may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present application.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for a system or system embodiment, since it is substantially similar to a method embodiment, the description is relatively simple, with reference to the description of the method embodiment being made in part. The systems and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present application without undue burden.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.
Claims (7)
1. A method of processing data in a flow, for application to a server, the method comprising:
receiving to-be-executed form data and target operation information which are sent by user terminal equipment and stored in a mode of a graph, wherein the graph comprises a plurality of nodes and a plurality of edges, the nodes comprise target operation nodes and data source nodes, and the edges are used for pointing to one node from the other node and storing data flow information; the target operation information comprises an operation mode and/or an operation object; the operation mode and/or the operation object are determined by the user terminal equipment according to the result input by the user;
traversing all nodes to determine a starting node, wherein the starting node is the data source node; the starting node comprises two starting nodes and respectively stores first form data and second form data, the first form data comprises bank flow data, and the bank flow data comprises names, identity card numbers, amounts and dates; the second table data comprises personnel list data, wherein the personnel list data comprises names, identity card numbers and work units; the target operation node comprises target operation information of personnel records contained in the second table data in the first table data;
reading form data contained in the starting node;
traversing all the edges to determine all edges related to the starting node;
determining a target operation node related to the starting node according to the determined all edges;
determining an execution algorithm according to the target operation information contained in the target operation node, and executing the execution algorithm to generate an execution result; the execution algorithm comprises the following steps:
reading all records of name fields contained in the second table data to obtain a list of people to be searched;
comparing the names of the people in the personnel list to be searched with the names contained in the first table data one by one, judging whether the names of the people in the personnel list to be searched appear in the first table data to obtain a matching result, if so, recording the matching result as successful, and if not, recording the matching result as failed;
the execution result comprises the matching result;
and returning the execution result to the user terminal equipment so that the user terminal equipment can display the execution result.
2. The method according to claim 1, wherein table data included in the start node is read from a hard disk, and target operation information is read from the target operation node.
3. The method of claim 1, wherein determining a target operational node associated with the starting node based on the determined all edges, comprises: and traversing all the determined edges to judge whether the starting node is the starting point of all the determined edges, and if so, judging that the node corresponding to the respective end point of all the determined edges is the target operation node related to the starting node.
4. The method according to claim 1, wherein the matching result is stored in a table manner and returned to the user terminal device.
5. The method of claim 1, wherein the start node includes a time table data of a time field; determining a theoretical interval time between uplink and downlink of the time field; the target operation node comprises target operation information for searching for the vacant time; the execution algorithm comprises the following steps:
reading all the time in the time table data, and calculating from the first row;
judging whether the interval time between the next line and the previous line is the theoretical interval time, if so, continuing the next line, and if the interval time is larger than the theoretical interval time, adding the theoretical interval time to the time of the current line to obtain the insertion time, storing the insertion time into a result list, and taking the insertion time as the time value of the current line; circularly executing the operation until the last row is calculated to be ended;
and all the insertion time contained in the result list is the vacant time to be searched.
6. The method according to claim 1, wherein the table data to be executed is any one or a combination of xlsx format data, xls format data, csv format data.
7. A streaming data processing system for application to a server, the system comprising:
the receiving unit is used for receiving to-be-executed form data and target operation information which are sent by user terminal equipment and stored in a form of a graph, wherein the graph comprises a plurality of nodes and a plurality of edges, the nodes comprise target operation nodes and data source nodes, and the edges are used for pointing to one node to the other node and storing data flow direction information; the target operation information comprises an operation mode and/or an operation object; the operation mode and/or the operation object are determined by the user terminal equipment according to the result input by the user;
the starting node determining unit is used for traversing all the nodes to determine a starting node, wherein the starting node is the data source node; the starting node comprises two starting nodes and respectively stores first form data and second form data, the first form data comprises bank flow data, and the bank flow data comprises names, identity card numbers, amounts and dates; the second table data comprises personnel list data, wherein the personnel list data comprises names, identity card numbers and work units; the target operation node comprises target operation information of personnel records contained in the second table data in the first table data;
a data reading unit, configured to read table data included in the start node;
an edge determining unit, configured to traverse all the edges to determine all edges related to the start node;
a target operation node determining unit, configured to determine a target operation node related to the start node according to the determined all edges;
the execution unit is used for determining an execution algorithm according to the target operation information contained in the target operation node and executing the execution algorithm to generate an execution result; the execution algorithm comprises the following steps:
reading all records of name fields contained in the second table data to obtain a list of people to be searched;
comparing the names of the people in the personnel list to be searched with the names contained in the first table data one by one, judging whether the names of the people in the personnel list to be searched appear in the first table data to obtain a matching result, if so, recording the matching result as successful, and if not, recording the matching result as failed;
the execution result comprises the matching result;
and the result returning unit is used for returning the execution result to the user terminal equipment so that the user terminal equipment can display the execution result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010950926.2A CN112084196B (en) | 2020-09-11 | 2020-09-11 | Method and system for processing flow data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010950926.2A CN112084196B (en) | 2020-09-11 | 2020-09-11 | Method and system for processing flow data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112084196A CN112084196A (en) | 2020-12-15 |
| CN112084196B true CN112084196B (en) | 2023-10-17 |
Family
ID=73736220
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010950926.2A Active CN112084196B (en) | 2020-09-11 | 2020-09-11 | Method and system for processing flow data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112084196B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113360559A (en) * | 2021-06-04 | 2021-09-07 | 武汉一格空间科技有限公司 | Configurable one-stop large-screen visualization method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010122969A (en) * | 2008-11-20 | 2010-06-03 | Obic Co Ltd | Business management device and method, and business management program |
| CN108090198A (en) * | 2017-12-22 | 2018-05-29 | 浙江创邻科技有限公司 | Chart database creates and data load method, equipment and medium |
| US10002154B1 (en) * | 2017-08-24 | 2018-06-19 | Illumon Llc | Computer data system data source having an update propagation graph with feedback cyclicality |
| CN109101575A (en) * | 2018-07-18 | 2018-12-28 | 广东惠禾科技发展有限公司 | Calculation method and device |
| CN110347708A (en) * | 2019-06-28 | 2019-10-18 | 深圳市元征科技股份有限公司 | A kind of data processing method and relevant device |
| CN110738389A (en) * | 2019-09-03 | 2020-01-31 | 深圳壹账通智能科技有限公司 | Workflow processing method and device, computer equipment and storage medium |
| CN111209309A (en) * | 2020-01-13 | 2020-05-29 | 腾讯科技（深圳）有限公司 | Method, device and equipment for determining processing result of data flow graph and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7657549B2 (en) * | 2005-07-07 | 2010-02-02 | Acl Services Ltd. | Method and apparatus for processing XML tagged data |
| WO2016183544A1 (en) * | 2015-05-14 | 2016-11-17 | Walleye Software, LLC | System performance logging |
-
2020
- 2020-09-11 CN CN202010950926.2A patent/CN112084196B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010122969A (en) * | 2008-11-20 | 2010-06-03 | Obic Co Ltd | Business management device and method, and business management program |
| US10002154B1 (en) * | 2017-08-24 | 2018-06-19 | Illumon Llc | Computer data system data source having an update propagation graph with feedback cyclicality |
| CN108090198A (en) * | 2017-12-22 | 2018-05-29 | 浙江创邻科技有限公司 | Chart database creates and data load method, equipment and medium |
| CN109101575A (en) * | 2018-07-18 | 2018-12-28 | 广东惠禾科技发展有限公司 | Calculation method and device |
| CN110347708A (en) * | 2019-06-28 | 2019-10-18 | 深圳市元征科技股份有限公司 | A kind of data processing method and relevant device |
| CN110738389A (en) * | 2019-09-03 | 2020-01-31 | 深圳壹账通智能科技有限公司 | Workflow processing method and device, computer equipment and storage medium |
| CN111209309A (en) * | 2020-01-13 | 2020-05-29 | 腾讯科技（深圳）有限公司 | Method, device and equipment for determining processing result of data flow graph and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112084196A (en) | 2020-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7475062B2 (en) | Apparatus and method for selecting a subset of report templates based on specified criteria | |
| US10685319B2 (en) | Big data sourcing simulator | |
| US20160132828A1 (en) | Real-time continuous realignment of a large-scale distributed project | |
| US20200074509A1 (en) | Business data promotion method, device, terminal and computer-readable storage medium | |
| CN115423289B (en) | Intelligent plate processing workshop data processing method and terminal | |
| CN113361954A (en) | Attribution analysis method, attribution analysis device, attribution analysis equipment and storage medium | |
| CN110471945A (en) | Processing method, system, computer equipment and the storage medium of alive data | |
| CN112084196B (en) | Method and system for processing flow data | |
| Clausen et al. | Big data driven order-up-to level model: Application of machine learning | |
| US10055469B2 (en) | Method and software for retrieving information from big data systems and analyzing the retrieved data | |
| CN111967521A (en) | Cross-border active user identification method and device | |
| US8335759B2 (en) | Work analysis device and recording medium recording work analysis program | |
| Pika et al. | Configurable batch-processing discovery from event logs | |
| EP3785197A1 (en) | Business insight generation system | |
| CN114385705A (en) | Data importance identification method, device, equipment and medium | |
| Prakashbhai et al. | Inference patterns from Big Data using aggregation, filtering and tagging-A survey | |
| CN110765100B (en) | Label generation method and device, computer readable storage medium and server | |
| CN107203506A (en) | A kind of report form generation method and device | |
| CN113139835A (en) | Data processing method, data processing device, nonvolatile storage medium and processor | |
| US20160232470A1 (en) | Automated Generation of Process Flow Charts | |
| US20200371999A1 (en) | System and computer program for providing automated actions and content to one or more web pages relating to the improved management of a value chain network | |
| Aisha et al. | Optimization of economic order quantity (EOQ) with dynamic programming | |
| CN112580915A (en) | Project milestone determination method and device, storage medium and electronic equipment | |
| Koenecke et al. | Tutorial: Sequential Pattern Mining in R for Business Recommendations | |
| US11727002B2 (en) | Segment trend analytics query processing using event data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |