CN105930494A - Multimode matching model based complex event detection method - Google Patents
Multimode matching model based complex event detection method Download PDFInfo
- Publication number
- CN105930494A CN105930494A CN201610296398.7A CN201610296398A CN105930494A CN 105930494 A CN105930494 A CN 105930494A CN 201610296398 A CN201610296398 A CN 201610296398A CN 105930494 A CN105930494 A CN 105930494A
- Authority
- CN
- China
- Prior art keywords
- mode
- detection
- matching model
- function
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24568—Data stream processing; Continuous queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a multimode matching model based complex event detection method. A plurality of complex event detection modes are fused to construct a finite state automata, so that the storage and search of numerous redundant automata states and transfer boundaries are greatly reduced, the repeated data operation matching and computing operations are avoided, the detection and matching of the complex event detection modes can be finished by scanning data streams once, and the efficiency of detecting complex events in the massive data streams is improved. According to the method, the shared detection of the complex events in the massive data streams is realized, a current automata-based complex event mode detection method is improved, and an existing complex event detection technology is extended for finishing detection of the complex events in the massive data streams more efficiently.
Description
Technical field
The present invention relates to mass data flow processing technology field, a kind of complex events detecting methods based on multi-mode matching model in processing more particularly, to mass data flow.
Background technology
The electronic information technologies such as network, embedded, RFID, sensor and executor are blended by technology of Internet of things with traditional production technique, realize containing product design, produce the full-range data perception with the link such as service, transmit, calculate, control and service etc., improve product technological additional-value, strengthen the management and control ability produced with service process, expedite the emergence of the important means of new modern production pattern.
In production environment, day by day maximize along with production-scale, the day by day complication of production procedure, the awareness apparatus such as production process spatial and temporal distributions and production environment multi-source are done melancholy and made many a large amount of such as RFID label tag, sensor node are deployed to production scene and go monitoring field situation to produce various magnanimity creation data stream.Owing to these magnanimity creation data stream exists: 1) data volume is the hugest, in magnanimity, per second can reach TB level even PB level scale；2), there is people, material, equipment, production process, product, the multiple data content such as service in data content multi-source；3) data structure is complicated, and structural data, semi-structured data and unstructured data coexist；4) data renewal speed is fast, and data every point are all producing and updating each second；5) data response requires height, and data need to process real-time the data characteristicses such as response, result in mass data flow process in Internet of Things and is faced with: magnanimity multi-source dynamic dataflow is difficult to the major issue processed in time.Owing to available data processing method is difficult to the real-time high-efficiency process of complete supporter networking mass data flow, it is difficult to rapidly find out information needed from above-mentioned mass data flow and make a response in time, thus affecting manufacturing enterprise to production process scheduling and Decision-making Function.Due to complicated event detection technique, it can utilize the association between event attribute, the magnanimity creation data stream of arrival continuously is constantly filtered by matched rule or algebraic manipulation, the sequence of events meeting certain interconnection constraint required for rapidly finding out enterprise, thus in all kinds of production industries, in recent years obtain increasingly extensive concern.
Currently, about the research of complex events detecting methods in data stream, mainly carry out and have based on automat, based on Petri network, based on coupling tree and complex events detecting methods based on aspects such as directed graphs and their some improved methods, as based on Time Petri Nets complex events detecting methods, tree complex events detecting methods based on compressed compositions, based on improving the methods such as the complex events detecting methods of figure, complex events detecting methods based on pushdown automata structure, the complex events detecting methods shared based on coupling tree-model matching result.But owing to what existing above-mentioned detection method can only isolate detects and query processing complicated event single in data stream, and cannot realize the multiple complicated events in data stream are inquired about carrying out sharing detection.And in actual life, it often is faced with inquiring about and detecting on a data stream multiple complicated event problem.If directly utilizing above-mentioned detection method when going to realize multiple complicated events detection in data stream, it would appear that there is state of automata and the transfer limit of bulk redundancy, mass data repeats storage, searches and calculate operation, thus occur that the detection time is long, consumption internal memory is big, the problem that detection efficiency is low, it is difficult to realize the multimodal real-time detection function of event in mass data flow.
Summary of the invention
The present invention is directed to current complex events detecting methods and occur that the detection time is long when realizing multiple complicated events detection on data stream, consumption internal memory is big, the problem that detection efficiency is low, the present invention is towards mass data flow, propose a kind of complex events detecting methods based on multi-mode matching model, the inventive method improves automat (NFA) sequence scanning and the sequence process of routine, extends existing complicated event detection technique, substantially increases complicated event power of test in mass data flow.
For solving above-mentioned technical problem, technical scheme is as follows:
A kind of complex events detecting methods based on multi-mode matching model, comprises the following steps:
S1: build multi-mode finite-state automata Matching Model, realizes especially by following little step:
S1.1: read each monotype event matches expression formula；
S1.2: build each self-corresponding monotype finite-state automata Matching Model according to each monotype event matches expression formula read；
S1.3: building state transition function according to the monotype finite-state automata Matching Model built, the function of state transition function is to realize the state transition path of finite automata；
S1.4: build unsuccessfully transfer function according to the transfer function implementation status built, the function of failure transfer function (failure) is to realize, for when there is mismatch, pointing to next state to continue to compare；
S1.5: build Output transfer function according to the transfer function built and failure letter implementation status, Output transfer function (output) for when certain pattern string obtains coupling, exports or records one the match is successful information, and carries out next state and continue to compare；
S1.6: output multi-mode finite-state automata Matching Model；
S2: multi-mode finite-state automata Matching Model is mated, and realizes especially by following little step:
S2.1: read in detection data one by one from data stream；
S2.2: the detection data according to reading determine multi-mode state shift direction；
S2.3: determine whether to perform multi-mode failure forwarding function according to state transition path；
S2.4: determine whether to perform multi-mode matching output function according to multi-mode state shift direction or multi-mode matching failure forwarding function；
S2.5: judge whether Detection task completes, if completing, jumping to S2.6 and performing；Otherwise jump to S2.1 perform；
S2.6: output detections result.
Further, in step S2.2, described detection data are atomic event.
In step sl, it is necessary first to build corresponding monotype finite-state automata Matching Model according to single pattern matching expression formula；Then recycling builds corresponding monotype finite-state automata Matching Model and builds three transfer functions successively: state transition function, failure transfer function and Output transfer function；Finally utilize the effect of cooperating of above three transfer function, it is achieved become to merge multi-mode matching model by single pattern matching Model Fusion；
In step s 2, it is necessary first to use above-mentioned generation multi-mode finite-state automata Matching Model, from mass data flow, read in atomic event one by one, state transition function, failure transfer function is used to carry out state transfer and mate；Finally use Output transfer function to export comparison match result each time, be finally completed the multi-mode event matches of mass data flow；
Compared with prior art, technical solution of the present invention provides the benefit that: the present invention discloses a kind of complex events detecting methods based on multi-mode matching model, multiple complicated events detection schema merging is built into a finite-state automata, greatly reduce state of automata and the storage of transfer limit of many redundancies and search, avoid repeating data manipulation coupling and calculating operation, realize run-down data stream can complete multiple complicated event detection pattern detection with mate, improve complicated event detection efficiency on data stream.This method achieves multiple complicated events in Internet of Things mass data flow and shares detection, improve the complicated event mode detection method being currently based on automat, existing complicated event detection technique is extended so that it is the detection completing multiple complicated event efficiently in mass data can be compared.
Accompanying drawing explanation
Fig. 1 is the multi-mode matching model construction process figure that the inventive method is carried.
Fig. 2 be detection mode expansions be SEQ(A, B, C, D) NFA illustraton of model.
Fig. 3 be detection mode expansions be SEQ(A, F, E) NFA illustraton of model.
Fig. 4 be detection mode expansions be SEQ(M, E, C, A) NFA illustraton of model.
Fig. 5 be the inventive method carried based on multi-mode matching model.
Fig. 6 is the complicated event detection process of the multi-mode matching that the inventive method is carried.
Fig. 7 is that the present invention compares schematic diagram with monotype detection method in terms of the detection time.
Fig. 8 is that the present invention compares schematic diagram with monotype detection method in terms of memory consumption
Fig. 9 is that the present invention compares schematic diagram with monotype detection method in terms of handling capacity.
Detailed description of the invention
Accompanying drawing being merely cited for property explanation, it is impossible to be interpreted as the restriction to this patent；With embodiment, technical scheme is described further below in conjunction with the accompanying drawings.
Embodiment 1
A kind of complex events detecting methods based on multi-mode matching model, comprises the following steps:
S1: build multi-mode finite-state automata Matching Model, realizes especially by following little step:
S1.1: read each monotype event matches expression formula；
S1.2: build each self-corresponding monotype finite-state automata Matching Model according to each monotype event matches expression formula read；
S1.3: building transfer function according to the monotype finite-state automata Matching Model built, the function of transfer function is to realize the state transition path of finite automata；
S1.4: build unsuccessfully function according to the transfer function implementation status built, the function of failure function is to realize, for when there is mismatch, pointing to next state to continue to compare；
S1.5: build output function according to the state transition function built and failure transfer function implementation status, output function for when certain pattern string obtains coupling, exports or records one the match is successful information；
S1.6: output multi-mode finite-state automata Matching Model；
S2: multi-mode finite-state automata Matching Model is mated, and realizes especially by following little step:
S2.1: read in detection data one by one from data stream, described detection data are atomic event；
S2.2: the detection data according to reading determine multi-mode state shift direction；
S2.3: determine whether to perform multi-mode failure forwarding function according to state transition path；
S2.4: determine whether to perform multi-mode Output transfer function according to multi-mode state shift direction or multi-mode failure forwarding function；
S2.5: judge whether Detection task completes, if completing, jumping to S2.6 and performing；Otherwise jump to S2.1 perform；
S2.6: output detections result.
The concrete detection process of a kind of complex events detecting methods based on multi-mode matching model is described in detail by the present embodiment.In this example, data generator module is utilized to go simulation to produce from all kinds of production industry mass data flows.The specification of event type is generated, the needs to realize requirement of experiment such as the probability distribution of flow of event by controlling data generator module parameter.The experimental tool of the present embodiment is: Visual C++ 6. 0, test index is: the detection time, memory consumption and handling capacity three aspect, experiment comparative approach is: monotype detection (Singpattern detection) method, i.e. realizes complicated event detection expression formula with existing respectively based on automat structure complex events detecting methods.Detection expression formula is respectively SEQ(A, B, C, D), SEQ(A, F, E) and SEQ(M, E, C, A).This example, as a example by detection above three detection expression formula, illustrates that the present invention is put forward the concrete application process of complex events detecting methods based on multi-mode matching model.
Fig. 1 is the multi-mode matching model construction process figure that the inventive method is carried.It mainly contains: reading single model matching expression, builds single pattern matching model, creates multi-mode state transition function, creates multi-mode failure transfer function, creates the most of function of multi-mode Output transfer function and multi-mode matching model output etc. five.Its major function realized is that multiple monotype event matches Model Fusion are built into a multi-mode event matches model, to realize sharing between multiple mode detection expression formula string, eliminates state of automata and the transfer limit of the many redundancies existed between them.Fig. 2 is to be SEQ(A by detection mode expansions, B, C, D) the NFA illustraton of model that generates；Fig. 3 is to be SEQ(A by detection mode expansions, F, E) the NFA illustraton of model that generates；Fig. 4 is to be SEQ(M by detection mode expansions, E, C, A) the NFA illustraton of model that generates.Fig. 5 is the multi-mode matching model of the inventive method, and it is to be SEQ(A by detection mode expansions, B, C, D), SEQ(A, F, E) and SEQ(M, E, C, A) jointly construct and form.
The complicated event detection process of the multi-mode matching that the inventive method is carried is as shown in Figure 6, it mainly contains: read in data one by one from data stream, multi-mode state shifts, and multi-mode unsuccessfully shifts, multi-mode Output transfer and multi-mode matching result output partial function.Its major function is to use the new multi-mode event matches model realization merged to detect the complicated event of data stream, and output detections result.
Fig. 7 present invention compares schematic diagram with monotype detection method in terms of the detection time.It will be seen in fig. 7 that under the conditions of same test, compare monotype detection method, the inventive method can be greatly enhanced the detection time, improves event detection efficiency.Analyze it and main reason is that the inventive method eliminates state of automata and the transfer limit that there is redundancy in many monotype detection methods owing to sharing, decrease the storage of many repetition data, search and calculate operation, thus save the much detection time.
Fig. 8 is that the present invention compares schematic diagram with existing monotype detection method in terms of internal memory uses consumption.As seen from Figure 8, under the conditions of same test, the inventive method is better than monotype detection method in terms of internal memory uses consumption.Analyze it main reason is that, under the conditions of same test, the inventive method uses and goes to detect dependent event in mass data flow based on multi-mode event model, eliminate state of automata and the transfer limit that there is redundancy in many monotype detection methods, decrease the storage of many repetition data, search and calculate operation, thus save many internal memories and use consumption.
Fig. 9 is that the present invention compares schematic diagram with existing monotype detection method in terms of event handling capacity.It will be seen from figure 9 that under the conditions of same test, the inventive method in terms of event handling capacity also superior to monotype detection method.Analyze it and main reason is that in the present invention, multi-mode event matches model uses.In the present invention, we use multi-mode event model to go to search dependent event, realize the quick lookup of dependent event in data stream, calculate and matching operation, state of automata and the transfer limit of reducing many redundancies in monotype detection method are searched and are calculated, decrease the most multidata repeated matching and calculate operation, and then improve system event processing speed.
Obviously, the above embodiment of the present invention is only for clearly demonstrating example of the present invention, and is not the restriction to embodiments of the present invention.For those of ordinary skill in the field, can also make other changes in different forms on the basis of the above description.Here without also cannot all of embodiment be given exhaustive.All any amendment, equivalent and improvement etc. made within the spirit and principles in the present invention, within should be included in the protection domain of the claims in the present invention.
Claims (2)
1. a complex events detecting methods based on multi-mode matching model, it is characterised in that said method comprising the steps of:
S1: build multi-mode finite-state automata Matching Model, realizes especially by following little step:
S1.1: read each monotype event matches expression formula；
S1.2: build each self-corresponding monotype finite-state automata Matching Model according to each monotype event matches expression formula read；
S1.3: building transfer function according to the monotype finite-state automata Matching Model built, the function of transfer function is to realize the state transition path of finite automata；
S1.4: build unsuccessfully function according to the transfer function implementation status built, the function of failure function is to realize, for when there is mismatch, pointing to next state to continue to compare；
S1.5: build output function according to the transfer function built and failure letter implementation status, output function for when certain pattern string obtains coupling, exports or records one the match is successful information；
S1.6: output multi-mode finite-state automata Matching Model；
S2: multi-mode finite-state automata Matching Model is mated, and realizes especially by following little step:
S2.1: read in detection data one by one from data stream；
S2.2: the detection data according to reading determine multi-mode state shift direction；
S2.3: determine whether to perform multi-mode matching failure forwarding function according to state transition path；
S2.4: determine whether to perform multi-mode matching output function according to multi-mode state shift direction or multi-mode matching failure forwarding function；
S2.5: judge whether Detection task completes, if completing, jumping to S2.6 and performing；Otherwise jump to S2.1 perform；
S2.6: output detections result.
Complex events detecting methods based on multi-mode matching model the most according to claim 1, it is characterised in that in step S2.2, described detection data are atomic event.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610296398.7A CN105930494A (en) | 2016-05-06 | 2016-05-06 | Multimode matching model based complex event detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610296398.7A CN105930494A (en) | 2016-05-06 | 2016-05-06 | Multimode matching model based complex event detection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105930494A true CN105930494A (en) | 2016-09-07 |
Family
ID=56834527
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610296398.7A Pending CN105930494A (en) | 2016-05-06 | 2016-05-06 | Multimode matching model based complex event detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105930494A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112613317A (en) * | 2020-12-30 | 2021-04-06 | 中国农业银行股份有限公司 | Text data cleaning method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100057735A1 (en) * | 2008-08-29 | 2010-03-04 | Oracle International Corporation | Framework for supporting regular expression-based pattern matching in data streams |
| CN101753316A (en) * | 2008-12-02 | 2010-06-23 | 北京启明星辰信息技术股份有限公司 | Method and system for intelligently extracting features |
| CN104361058A (en) * | 2014-10-31 | 2015-02-18 | 广东工业大学 | Hash structure complex event detection method for mass data flow |
-
2016
- 2016-05-06 CN CN201610296398.7A patent/CN105930494A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100057735A1 (en) * | 2008-08-29 | 2010-03-04 | Oracle International Corporation | Framework for supporting regular expression-based pattern matching in data streams |
| CN101753316A (en) * | 2008-12-02 | 2010-06-23 | 北京启明星辰信息技术股份有限公司 | Method and system for intelligently extracting features |
| CN104361058A (en) * | 2014-10-31 | 2015-02-18 | 广东工业大学 | Hash structure complex event detection method for mass data flow |
Non-Patent Citations (1)
| Title |
|---|
| 刘春晖 等: "一种改进的AC多模式匹配算法", 《计算机工程》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112613317A (en) * | 2020-12-30 | 2021-04-06 | 中国农业银行股份有限公司 | Text data cleaning method and device |
| CN112613317B (en) * | 2020-12-30 | 2023-12-08 | 中国农业银行股份有限公司 | Text data cleaning method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yadav et al. | A systematic literature review of blockchain technology in agriculture | |
| Zhang et al. | Towards new-generation human-centric smart manufacturing in Industry 5.0: A systematic review | |
| Luan et al. | Out-of-distribution detection for deep neural networks with isolation forest and local outlier factor | |
| Wu et al. | The Internet of Things enabled shop floor scheduling and process control method based on Petri nets | |
| Lei et al. | Large-scale dynamic scheduling for flexible job-shop with random arrivals of new jobs by hierarchical reinforcement learning | |
| Dohale et al. | A review of literature on industry 4.0 | |
| Jing et al. | Intelligent generation method of 3D machining process based on process knowledge | |
| CN116244333A (en) | Database query performance prediction method and system based on cost factor calibration | |
| Chen et al. | Extension data mining method for improving product manufacturing quality | |
| Mayr et al. | Abstracting process mining event logs from process-state data to monitor control-flow of industrial manufacturing processes | |
| CN104766108A (en) | Optimizing processing method for FID electronic tag in BIM model | |
| Liu et al. | A dynamic updating method of digital twin knowledge model based on fused memorizing-forgetting model | |
| CN105930494A (en) | Multimode matching model based complex event detection method | |
| Xu et al. | Dynamic scheduling model of rail-guided vehicle (RGV) based on genetic algorithms in the context of mobile computing | |
| Zhao et al. | Design concerns for industrial big data system in the smart factory domain: From product lifecycle view | |
| CN101697194A (en) | Data processing system and method for improving reliability of RFID application | |
| Liu et al. | A stacking denoising auto-encoder with sample weight approach for order remaining completion time prediction in complex discrete manufacturing workshop | |
| CN102541855B (en) | Information data handling system and method | |
| CN104361058A (en) | Hash structure complex event detection method for mass data flow | |
| CN116228103A (en) | Inventory life cycle real-time management method and system | |
| Jiang et al. | Fault Diagnosis of Gearbox in Multiple Conditions Based on Fine‐Grained Classification CNN Algorithm | |
| Cupek et al. | Data preprocessing, aggregation and clustering for agile manufacturing based on automated guided vehicles | |
| CN114186755A (en) | Visual intelligent logistics dynamic optimization management and control method and system | |
| CN106383863A (en) | Isomorphic sub-graph query optimization method | |
| CN104391950A (en) | Method for using hash B + tree structure to detect complex events in manufacturing Internet of Things massive data streams |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160907 |