CN117787902A - Flow batch integration-based distribution control early warning system and method - Google Patents
Flow batch integration-based distribution control early warning system and method Download PDFInfo
- Publication number
- CN117787902A CN117787902A CN202311804177.2A CN202311804177A CN117787902A CN 117787902 A CN117787902 A CN 117787902A CN 202311804177 A CN202311804177 A CN 202311804177A CN 117787902 A CN117787902 A CN 117787902A
- Authority
- CN
- China
- Prior art keywords
- data
- early warning
- control
- task
- distribution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000010354 integration Effects 0.000 title claims abstract description 19
- 238000004590 computer program Methods 0.000 claims description 9
- 230000007246 mechanism Effects 0.000 claims description 8
- 239000004744 fabric Substances 0.000 claims description 4
- 238000012545 processing Methods 0.000 abstract description 12
- 230000004927 fusion Effects 0.000 abstract description 3
- 238000012423 maintenance Methods 0.000 abstract description 3
- 230000008520 organization Effects 0.000 description 7
- 230000009286 beneficial effect Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000012216 screening Methods 0.000 description 4
- 125000002015 acyclic group Chemical group 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a distribution control early warning system and method based on a flow batch integration, wherein the system comprises the following components: the data source management module is used for extracting the distributed control data from different data sources; the distribution management module is used for creating a distribution task according to the distribution data; and the approval module is used for approving the control task, and transmitting the early warning message generated by the control task to the early warning module for display after the control task passes the approval. The invention realizes the fusion of data stream processing and batch processing through the DAG (directed acyclic graph) model, reduces the redundancy of a data chain, relieves the maintenance difficulty of developers and avoids errors caused by inconsistent settlement of stream batch results.
Description
Technical Field
The invention belongs to the technical field of big data processing, and particularly relates to a distribution control early warning system and method based on a flow batch integration.
Background
With the continuous development of the Internet and the mobile Internet, various industries accumulate massive business data. In order to improve user experience, enterprises improve the competitiveness of products in the market, and all adopt a real-time mode to process big data. The success of the products is described as an overwhelming trend in real-time big screen of social media, real-time recommendation of electronic commerce, real-time traffic prediction of urban brain and real-time anti-fraud of financial industry. Under the general trend of real-time, how to integrate real-time data processing with batch processing in historical business and solve pain points of developers and users is the key to solve the problems currently.
The existing method for processing the distributed control early warning is divided into two types, wherein the first type of the method is used for processing the early warning in batch, namely, the early warning result is generated by analyzing comprehensive historical data; with the development of technology, users are increasingly pursuing timeliness of data at present, so that a second type of distributed early warning for high-timeliness data processing is generated. The two modes operate independently, and from the perspective of users and developers, the original flow and batch independent scheme has some disadvantages: 1. since streams and batches are two sets of systems, the same logic requires two teams to develop twice, and thus the labor cost is high. 2. And redundancy of the data link. In many scenarios, the flow and batch content are identical in nature, but because of the two systems, the same logic still needs to run twice, resulting in some wastage of resources. 3. The data apertures are inconsistent. Two systems, two operators, and two sets of UDFs (user-defined functions) must generate errors of different degrees, and these errors bring great trouble to the business party.
Disclosure of Invention
In order to solve the above problems, the present invention aims to provide a system and a method for controlling and early warning based on integration of flow batch.
In order to achieve the above purpose, the invention adopts the following technical scheme:
flow batch integration-based cloth control early warning system comprises:
the data source management module is used for extracting the distributed control data from different data sources;
the distribution management module is used for creating a distribution task according to the distribution data;
and the approval module is used for approving the control task, and transmitting the early warning message generated by the control task to the early warning module for display after the control task passes the approval.
Preferably, the deployment data includes real-time trace class data stored in a message queue and dimension table data stored in a sub-deployment storage database.
Preferably, the data source management module is configured to implement a data source plug-in mechanism by using a JAVA SPI and a custom class loader, so as to dynamically load and adapt to various data sources.
Preferably, the control management module is configured to construct a directed acyclic graph in a dragging manner, and create a control task based on the directed acyclic graph and the control data.
Preferably, the approval module is configured to perform approval according to a preset organization hierarchy.
Preferably, the approval module is further configured to parse the approved task into a flink task.
The invention also provides a distribution control early warning method based on the integration of the flow batch, which comprises the following steps:
step 1: extracting the distributed control data from different data sources;
step 2: creating a distribution task according to the distribution data;
step 3: and examining and approving the control task, and transmitting an early warning message generated by the control task to an early warning module for display after the control task passes the examination and approval.
The invention also provides an electronic device, which comprises a bus, a transceiver, a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the transceiver, the memory and the processor are connected through the bus.
The invention also provides a computer readable storage medium, on which a computer program is stored, which is characterized in that the computer program, when executed by a processor, realizes the steps in the batch integrated-based distributed control early warning method.
The flow batch integration-based control early warning system and method provided by the invention have the beneficial effects that: compared with the prior art, the invention realizes the fusion of data stream processing and batch processing through the DAG (directed acyclic graph) model, reduces the redundancy of a data chain, relieves the maintenance difficulty of developers, and avoids errors caused by inconsistent settlement of stream batch results.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a flow batch integration-based distributed control early warning system provided by an embodiment of the invention;
FIG. 2 is a flow chart of creating a task for controlling according to an embodiment of the present invention;
fig. 3 is an approval flowchart provided in an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical schemes and beneficial effects to be solved more clear, the invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, in order to achieve the above objective, the present invention adopts the following technical scheme: flow batch integration-based cloth control early warning system comprises:
the data source management module is used for extracting the distributed control data from different data sources; the deployment data includes real-time trace class data stored in a message queue and dimension table data stored in a deployment store database.
The data source management module can be added for different data sources, test connection, table structure information mapping can be carried out for a database table system, and message data is tiled, analyzed and mapped for real-time track class database (kafka) data, so that support is provided for selection of track class tables and dimension tables in distributed control management. The plug-in mechanism of the data source is realized by adopting a JAVA SPI (JAVA service providing discovery mechanism) and a custom class loader, and various data sources can be dynamically loaded and adapted. The following is a basic implementation procedure:
1. defining an interface: firstly, defining an interface, describing the operation method and the function of a data source;
2. the realization interface is as follows: specific classes for realizing interfaces are created according to different data sources;
3. creating a service provider interface: creating an interface for providing data source instances;
4. implementing a service provider interface: creating a specific service provider, and returning corresponding data source examples according to the need;
5. creating an enumeration class: configuring the interface of the service provider realized in the last step into an enumeration class according to the type;
6. the SPI is used to load the data source: dynamically loading by using a ServiceLoader class according to different data sources configured in an enumeration class through an SPI mechanism of java to obtain a data source example;
in this way the above-mentioned manner of operation,the invention is thatThe plug-in mechanism for dynamically loading and adapting various data sources can be realized.
The distribution management module is used for creating a distribution task according to the distribution data;
referring to fig. 2, in the present invention, the management module is mainly configured to create a management task for a person, a vehicle or an article, and manage the management task for the management task. The operation of creating the control task is mainly to drag and edit configuration of operator nodes through the front end, and specifically, the simple steps of forming a DAG (directed acyclic graph) in the process of creating the task are as follows:
1. the front end uses an AntV G6 open source component to realize the dragging and editing of operators, defines the types of operator nodes and basic data structures, such as data sources, index operators and the like, and the types of the data source operators are consistent with the types of the dynamic loading data sources mentioned above;
2. the front end uses the DargeLayout layout algorithm provided by AntV G6 to control the generation of the DAG orchestration tasks. DagreLayout is a graph-based Directed Acyclic Graph (DAG) layout algorithm that uses Dagre's library to calculate the location of nodes. The algorithm is suitable for displaying a complex directed acyclic graph, maintaining the dependency relationship between nodes, and reducing the intersection of edges as much as possible;
3. the front end stores the page configuration flow chart to the background in a json character string mode, and secondary verification is carried out on canvas content during storage to judge the integrity of the canvas content and the reliability of the DAG. The back-end uses a topology ordering algorithm to determine whether the currently stored canvas content is a directed acyclic graph. The specific operation steps are as follows:
1) Constructing a data structure of a graph: and constructing a data structure of the directed graph according to the relation between the nodes and the edges.
The graph may be represented using an adjacency list or adjacency matrix.
2) Counting the degree of entering of the nodes: for each node, statistics of its degree of ingress (i.e., the number of edges pointing to the node) may be recorded using an array or hash table.
3) Initializing a queue: creating a queue, and adding a node with an admission degree of 0 to the queue.
4) Topology ordering traversal: one node is taken out of the queue, added to the result set, and the degree of penetration of all its neighbors is subtracted by 1. If the ingress of the adjacent node is reduced to 0, the adjacent node is added into the queue.
5) Judging result: if all nodes are added to the result set and there is no loop (i.e., all nodes are 0 in), then the graph is a directed acyclic graph. Otherwise, loops exist in the graph, not the directed acyclic graph.
The above is a checking method for whether the DAG graph is created when the task is created, and the following steps are the business operation steps for creating the task:
(1) Selecting dimension table data, such as dimension tables of a personnel basic information table, a vehicle basic information table and the like, connecting a screening operator at the downstream, and filtering personnel needing to be controlled according to specific conditions, wherein the main problem to be solved is to reduce the data size of the dimension table and remove invalid data;
(2) And selecting a real-time track class data table, such as a vehicle passing by a gate, ticket purchasing information and the like, and filling key data between the screened dimension data and the track class data through an intersection operator. For example, only the data of the identification card number exists in the real-time data, no personnel basic attribute information exists, the identification card number of the real-time track type data and the identification card number in the personnel dimension information table after the first step filtering can be intersected, basic information data such as personnel name, sex, age and the like in the dimension information are supplemented into the track data, and at the moment, other basic attribute information except the identification card can be obtained by the connected downstream operator nodes;
(3) Configuring a distribution rule of the data after intersection and association, wherein a screening operator node can be connected to the downstream of an intersection operator at the moment, screening is carried out in the screening operator node according to the position coordinate range of track occurrence and the time interval of track occurrence, and track data needing early warning generation is filtered out;
(4) The downstream of track data needing to generate early warning is connected with an early warning message formatting operator, and a well-defined message template is selected;
(5) The downstream of the message operator is connected with an early warning receiver operator, so that the configuration of the early warning receiver can be carried out;
and the approval module is used for approving the control task, and transmitting the early warning message generated by the control task to the early warning module for display after the control task passes the approval.
In practical application, the approval module needs to approve according to the level of the organization, for example, the organization is divided into three levels, namely, a first level organization, a second level organization and a third level organization in sequence from large to small, and if the third level organization submits a control task, the specific flow is shown in fig. 3. All layers of approval are completed and the DAG (directed acyclic graph) analysis of the control task is automatically submitted to the big data environment for operation after passing.
The early warning module mainly displays a display interface of early warning messages, and comprises display of early warning messages generated by a management and control task created by a creator and display of early warning messages received by an early warning receiver. The creator early warning message interface can check early warning signing feedback details, and the early warning receiver early warning message interface can conduct signing feedback operation.
The invention mainly aims at a distributed control early warning scene, uses the unification of the page configuration of dragging on business operation, and uses a flink calculation engine in calculation logic to complete the integration of flow and batch. The task configuration interface nodes are mainly divided into several categories including data sources, index operators, operation operators, branches, merging, output and the like. The flink tasks are dynamically built by page configuration directed acyclic graphs forming DAG (directed acyclic graphs) flowcharts on the business system. The distribution control early warning based on a big data system is realized, and the data value is embodied for business personnel.
The invention provides a method for constructing a directed acyclic graph by dragging, which can perform data collision between personnel, vehicles, articles and track data, set a space and time dimension distribution control rule and establish an omnibearing and multidimensional three-dimensional distribution control rule. Setting a control approval mechanism at the upstream to ensure the control stringency; the accurate pushing is arranged at the downstream, so that each early warning is well-known, a feedback overtime reminding mechanism is set up, and omission is avoided.
The invention also provides a distribution control early warning method based on the integration of the flow batch, which comprises the following steps:
step 1: extracting the distributed control data from different data sources;
step 2: creating a distribution task according to the distribution data;
step 3: and examining and approving the control task, and transmitting an early warning message generated by the control task to an early warning module for display after the control task passes the examination and approval.
The flow batch integration-based control early warning method provided by the invention has the beneficial effects that: compared with the prior art, the invention realizes the fusion of data stream processing and batch processing through the DAG (directed acyclic graph) model, reduces the redundancy of a data chain, relieves the maintenance difficulty of developers, and avoids errors caused by inconsistent settlement of stream batch results.
The invention also provides an electronic device, which comprises a bus, a transceiver, a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the transceiver, the memory and the processor are connected through the bus, and the computer program realizes the steps in the batch-based integrated control and early warning method when being executed by the processor.
Compared with the prior art, the electronic equipment provided by the invention has the same beneficial effects as the flow batch integration-based control early warning method in the technical scheme, and is not repeated herein.
The invention also provides a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, realizes the steps in the batch integrated-based distributed control early warning method.
Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the invention are the same as those of the flow batch integration-based control early warning method in the technical scheme, and the detailed description is omitted.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
Claims (9)
1. Flow batch integration-based cloth control early warning system, which is characterized by comprising:
the data source management module is used for extracting the distributed control data from different data sources;
the distribution management module is used for creating a distribution task according to the distribution data;
and the approval module is used for approving the control task, and transmitting the early warning message generated by the control task to the early warning module for display after the control task passes the approval.
2. The distributed control early warning system based on the integration of the flow batch is characterized in that the distributed control data comprise real-time track class data stored in a message queue and dimension table data stored in a distributed deployment storage database.
3. The method for integrating stream batch based distributed control and early warning according to claim 2, wherein the data source management module is configured to implement a data source plug-in mechanism by using a JAVA SPI and a custom class loader to dynamically load and adapt to various data sources.
4. The system of claim 3, wherein the control management module is configured to construct a directed acyclic graph by using a drag mode, and to create a control task based on the directed acyclic graph and the control data.
5. The system of claim 4, wherein the approval module is configured to approve according to a preset organizational hierarchy.
6. The system of claim 5, wherein the approval module is further configured to parse the approved task into a flink task.
7. The cloth control early warning method based on the integration of the flow batch is characterized by comprising the following steps:
step 1: extracting the distributed control data from different data sources;
step 2: creating a distribution task according to the distribution data;
step 3: and examining and approving the control task, and transmitting an early warning message generated by the control task to an early warning module for display after the control task passes the examination and approval.
8. An electronic device comprising a bus, a transceiver, a memory, a processor and a computer program stored on the memory and executable on the processor, the transceiver, the memory and the processor being connected by the bus, characterized in that the computer program when executed by the processor implements the steps of the batch-based integrated distributed pre-warning method according to any one of claims 7.
9. A computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements the steps of the batch integration based pilot warning method of any one of claims 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311804177.2A CN117787902A (en) | 2023-12-26 | 2023-12-26 | Flow batch integration-based distribution control early warning system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311804177.2A CN117787902A (en) | 2023-12-26 | 2023-12-26 | Flow batch integration-based distribution control early warning system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117787902A true CN117787902A (en) | 2024-03-29 |
Family
ID=90395911
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311804177.2A Pending CN117787902A (en) | 2023-12-26 | 2023-12-26 | Flow batch integration-based distribution control early warning system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117787902A (en) |
Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105677752A (en) * | 2015-12-30 | 2016-06-15 | 深圳先进技术研究院 | Streaming computing and batch computing combined processing system and method |
| CN106873945A (en) * | 2016-12-29 | 2017-06-20 | 中山大学 | Data processing architecture and data processing method based on batch processing and Stream Processing |
| US20180129713A1 (en) * | 2016-11-10 | 2018-05-10 | Sap Se | Efficient execution of data stream processing systems on multi-core processors |
| CN111597200A (en) * | 2019-02-21 | 2020-08-28 | 阿里巴巴集团控股有限公司 | Data processing method, device and storage medium |
| CN113407617A (en) * | 2021-06-25 | 2021-09-17 | 交控科技股份有限公司 | Real-time and off-line service unified processing method and device based on big data technology |
| CN114490763A (en) * | 2022-01-24 | 2022-05-13 | 全知科技(杭州)有限责任公司 | Single-machine high-performance batch flow integrated data processing system |
| CN114707961A (en) * | 2022-03-31 | 2022-07-05 | 中国建设银行股份有限公司 | Pending task execution method, device, equipment, medium and program product based on approval flow configuration |
| CN115018616A (en) * | 2022-04-25 | 2022-09-06 | 北京邮电大学 | Credit investigation intelligent evaluation method and system based on batch-flow integrated computing engine |
| CN115328626A (en) * | 2022-07-29 | 2022-11-11 | 浙江网安信创电子技术有限公司 | Big data engine distributed task scheduling method supporting stream and batch integration |
| CN115496130A (en) * | 2022-08-29 | 2022-12-20 | 广发银行股份有限公司 | Training and approving method, device and storage medium for new card issuing real-time approving model |
| CN115599524A (en) * | 2022-10-27 | 2023-01-13 | 中国兵器工业计算机应用技术研究所（Cn） | Data lake system based on cooperative scheduling processing of streaming data and batch data |
| CN116166728A (en) * | 2022-12-23 | 2023-05-26 | 航天信息股份有限公司 | Batch-flow integrated real-time data acquisition system |
| CN116301755A (en) * | 2023-03-23 | 2023-06-23 | 同济大学 | Automatic batch flow data marking framework construction method based on directed calculation graph |
| CN116629805A (en) * | 2023-06-07 | 2023-08-22 | 浪潮智慧科技有限公司 | Water conservancy index service method, equipment and medium for distributed flow batch integration |
| CN116680315A (en) * | 2023-06-09 | 2023-09-01 | 网易有道信息技术（北京）有限公司 | Data offline processing method and device, electronic equipment and storage medium |
| CN116910161A (en) * | 2023-09-14 | 2023-10-20 | 杭州三汇数字信息技术有限公司 | Collaborative analysis system, collaborative analysis method, electronic equipment and computer readable medium |
| CN117149873A (en) * | 2023-08-30 | 2023-12-01 | ***数智科技有限公司 | Data lake service platform construction method based on flow batch integration |
| CN117149383A (en) * | 2023-08-17 | 2023-12-01 | 招商银行股份有限公司 | Data processing method, device, terminal equipment and storage medium |
-
2023
- 2023-12-26 CN CN202311804177.2A patent/CN117787902A/en active Pending
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105677752A (en) * | 2015-12-30 | 2016-06-15 | 深圳先进技术研究院 | Streaming computing and batch computing combined processing system and method |
| US20180129713A1 (en) * | 2016-11-10 | 2018-05-10 | Sap Se | Efficient execution of data stream processing systems on multi-core processors |
| CN106873945A (en) * | 2016-12-29 | 2017-06-20 | 中山大学 | Data processing architecture and data processing method based on batch processing and Stream Processing |
| CN111597200A (en) * | 2019-02-21 | 2020-08-28 | 阿里巴巴集团控股有限公司 | Data processing method, device and storage medium |
| CN113407617A (en) * | 2021-06-25 | 2021-09-17 | 交控科技股份有限公司 | Real-time and off-line service unified processing method and device based on big data technology |
| CN114490763A (en) * | 2022-01-24 | 2022-05-13 | 全知科技(杭州)有限责任公司 | Single-machine high-performance batch flow integrated data processing system |
| CN114707961A (en) * | 2022-03-31 | 2022-07-05 | 中国建设银行股份有限公司 | Pending task execution method, device, equipment, medium and program product based on approval flow configuration |
| CN115018616A (en) * | 2022-04-25 | 2022-09-06 | 北京邮电大学 | Credit investigation intelligent evaluation method and system based on batch-flow integrated computing engine |
| CN115328626A (en) * | 2022-07-29 | 2022-11-11 | 浙江网安信创电子技术有限公司 | Big data engine distributed task scheduling method supporting stream and batch integration |
| CN115496130A (en) * | 2022-08-29 | 2022-12-20 | 广发银行股份有限公司 | Training and approving method, device and storage medium for new card issuing real-time approving model |
| CN115599524A (en) * | 2022-10-27 | 2023-01-13 | 中国兵器工业计算机应用技术研究所（Cn） | Data lake system based on cooperative scheduling processing of streaming data and batch data |
| CN116166728A (en) * | 2022-12-23 | 2023-05-26 | 航天信息股份有限公司 | Batch-flow integrated real-time data acquisition system |
| CN116301755A (en) * | 2023-03-23 | 2023-06-23 | 同济大学 | Automatic batch flow data marking framework construction method based on directed calculation graph |
| CN116629805A (en) * | 2023-06-07 | 2023-08-22 | 浪潮智慧科技有限公司 | Water conservancy index service method, equipment and medium for distributed flow batch integration |
| CN116680315A (en) * | 2023-06-09 | 2023-09-01 | 网易有道信息技术（北京）有限公司 | Data offline processing method and device, electronic equipment and storage medium |
| CN117149383A (en) * | 2023-08-17 | 2023-12-01 | 招商银行股份有限公司 | Data processing method, device, terminal equipment and storage medium |
| CN117149873A (en) * | 2023-08-30 | 2023-12-01 | ***数智科技有限公司 | Data lake service platform construction method based on flow batch integration |
| CN116910161A (en) * | 2023-09-14 | 2023-10-20 | 杭州三汇数字信息技术有限公司 | Collaborative analysis system, collaborative analysis method, electronic equipment and computer readable medium |
Non-Patent Citations (1)
| Title |
|---|
| 张奋翔等: "HSSM:一种流数据分层次模最大化方法", 计算机研究与发展, no. 08, 15 August 2016 (2016-08-15), pages 1792 - 1805 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11093668B2 (en) | Modeling and simulation | |
| US7835932B2 (en) | Relational logic management system | |
| KR101644418B1 (en) | Visualizing relationships between data elements and graphical representations of data element attributes | |
| US20180349134A1 (en) | User interface that integrates plural client portals in plural user interface portions through sharing of one or more log records | |
| US7574379B2 (en) | Method and system of using artifacts to identify elements of a component business model | |
| US7974896B2 (en) | Methods, systems, and computer program products for financial analysis and data gathering | |
| US8175911B2 (en) | System and method for inferring and visualizing correlations of different business aspects for business transformation | |
| US20060224425A1 (en) | Comparing and contrasting models of business | |
| US7743071B2 (en) | Efficient data handling representations | |
| US20060095276A1 (en) | Role-oriented development environment | |
| US8027859B2 (en) | Analysis of a model of a complex system, based on two models of the system, wherein the two models represent the system with different degrees of detail | |
| US7926024B2 (en) | Method and apparatus for managing complex processes | |
| US9558215B2 (en) | Governing information | |
| US10417924B2 (en) | Visual work instructions for assembling product | |
| US20140025411A1 (en) | Automatic configuration of process definition metrics | |
| US7512451B2 (en) | System and method for interactive process management | |
| Esser et al. | Graph-based version control for asynchronous BIM collaboration | |
| Hauschildt et al. | WOFLAN: a Petri-net-based Workflow Analyzer | |
| US20130166892A1 (en) | Generating a runtime framework | |
| KR102355787B1 (en) | Methods to gather screen requirements and to manage processes for the software development including web based services and mobile services | |
| CN117787902A (en) | Flow batch integration-based distribution control early warning system and method | |
| Lê | Diagramming multi-level service-oriented enterprise architecture | |
| CN113867700B (en) | Model construction method, display platform, server and storage medium | |
| CN110502219B (en) | Enterprise-level IT architecture asset management system and method | |
| Behbahani Nejad et al. | A novel architecture based on business intelligence approach to exploit big data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |