SG11201804482UA - Search and retrieval data processing system for computing near real-time data aggregations - Google Patents

Search and retrieval data processing system for computing near real-time data aggregations

Info

Publication number
SG11201804482UA
SG11201804482UA SG11201804482UA SG11201804482UA SG11201804482UA SG 11201804482U A SG11201804482U A SG 11201804482UA SG 11201804482U A SG11201804482U A SG 11201804482UA SG 11201804482U A SG11201804482U A SG 11201804482UA SG 11201804482U A SG11201804482U A SG 11201804482UA
Authority
SG
Singapore
Prior art keywords
data
international
data records
processing system
records
Prior art date
Application number
SG11201804482UA
Inventor
John Maclean
Paul Veiser
Original Assignee
Ab Initio Technology Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ab Initio Technology Llc filed Critical Ab Initio Technology Llc
Publication of SG11201804482UA publication Critical patent/SG11201804482UA/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1448—Management of the data involved in backup or backup restore
    • G06F11/1451—Management of the data involved in backup or backup restore by selection of backup contents
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
    • G06F11/3082—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting the data filtering being achieved by aggregating or compressing the monitored data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3438—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment monitoring of user actions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466—Performance evaluation by tracing or monitoring
    • G06F11/3476—Data logging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24564—Applying rules; Deductive queries
    • G06F16/24565—Triggers; Constraints
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24568—Data stream processing; Continuous queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/40—Data acquisition and logging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/84—Using snapshots, i.e. a logical point-in-time copy of the data

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Computer Hardware Design (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Debugging And Monitoring (AREA)

Abstract

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) (19) World Intellectual Property Organization International Bureau 111111111M 1111111111111111111111111111111111111111111111111111111111111R111111111111111 (10) International Publication Number WO 2017/112697 Al WIPO I PCT (43) International Publication Date 29 June 2017 (29.06.2017) (81) Designated States (unless otherwise indicated, for every kind of national protection available): AE, AG, AL, AM, AO, AT, AU, AZ, BA, BB, BG, BH, BN, BR, BW, BY, BZ, CA, CH, CL, CN, CO, CR, CU, CZ, DE, DJ, DK, DM, DO, DZ, EC, EE, EG, ES, FI, GB, GD, GE, GH, GM, GT, HN, HR, HU, ID, IL, IN, IR, IS, JP, KE, KG, KH, KN, KP, KR, KW, KZ, LA, LC, LK, LR, LS, LU, LY, MA, MD, ME, MG, MK, MN, MW, MX, MY, MZ, NA, NG, NI, NO, NZ, OM, PA, PE, PG, PH, PL, PT, QA, RO, RS, RU, RW, SA, SC, SD, SE, SG, SK, SL, SM, ST, SV, SY, TH, TJ, TM, TN, TR, TT, TZ, UA, UG, US, UZ, VC, VN, ZA, ZM, ZW. (84) Designated States (unless otherwise indicated, for every kind of regional protection available): ARIPO (BW, GH, GM, KE, LR, LS, MW, MZ, NA, RW, SD, SL, ST, SZ, TZ, UG, ZM, ZW), Eurasian (AM, AZ, BY, KG, KZ, RU, TJ, TM), European (AL, AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, FI, FR, GB, GR, HR, HU, IE, IS, IT, LT, LU, LV, MC, MK, MT, NL, NO, PL, PT, RO, RS, SE, SI, SK, SM, TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, KM, ML, MR, NE, SN, TD, TG). Published: (51) International Patent Classification: GOOF 17/40 (2006.01) GOOF 11/30 (2006.01) GOOF 11/34 (2006.01) (21) International Application Number: PCT/US2016/067840 (22) International Filing Date: 20 December 2016 (20.12.2016) English English (25) Filing Language: (26) Publication Language: (30) Priority Data: 62/270,257 21 December 2015 (21.12.2015) 15/360,449 23 November 2016 (23.11.2016) US US (71) Applicant: AB INITIO TECHNOLOGY LLC [US/US]; 201 Spring Street, Lexington, Massachusetts 02421 (US). (72) Inventors: MACLEAN, John; 201 Spring Street, Lexing- ton, Massachusetts 02421 (US). VEISER, Paul; 201 Spring Street, Lexington, Massachusetts 02421 (US). (74) Agents: MCDONOUGH, Christina V. et al.; Fish & Richardson P.C., P.O. Box 1022, Minneapolis, Minnesota 55440-1022 (US). — with international search report (Art 21(3)) (54) Title: SEARCH AND RETRIEVAL DATA PROCESSING SYSTEM FOR COMPUTING NEAR REAL-TIME DATA AG- GREGATIONS (57) : A method performed by a data processing system for processing data, the method including: intermittently receiving data from one or more data streams, the received data including data records; detecting two or more particular data records in the received data re- cords, where the detected two or more particular data records each in- clude a particular identifier; for that particular identifier, creating a col- lection of data records; for at least one particular data record included in the collection of data records, searching data records for a historical ag- gregation of data; and computing combined data; modifying a data re- cord by inserting the combined data into a field of the data record and by inserting data from at least one of the data records in the collection into another field of the data record; based on applying the rules, writ- ing to memory one or more instructions for initiation of one or more ac - tions. 220 220 INTEMITTENTLY RECEIVE EVENTS 222 • DETECT PARTICULAR ) TYPES OF EVENTS 224 CREATE OR UPDATE J A COLLECTION WITH THE DETECTED 226 EVENTS K 228 RETRIEVE HISTORICAL AGGREGATION . COMPUTE COMBINED r EVENT INFORMATION 229 ETERMINE WHETHER RULES ARE SATISFIED? YES NO 230 TRIGGER TO A QUEUE PUBLISH ACTION r i W O 20 17 / 112697 Al YES DETERMINE WHETHER CAMPAIGN STILL EXECUTING? 232 NO STOP 234 FIG. 15
SG11201804482UA 2015-12-21 2016-12-20 Search and retrieval data processing system for computing near real-time data aggregations SG11201804482UA (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201562270257P 2015-12-21 2015-12-21
US15/360,449 US11989096B2 (en) 2015-12-21 2016-11-23 Search and retrieval data processing system for computing near real-time data aggregations
PCT/US2016/067840 WO2017112697A1 (en) 2015-12-21 2016-12-20 Search and retrieval data processing system for computing near real-time data aggregations

Publications (1)

Publication Number Publication Date
SG11201804482UA true SG11201804482UA (en) 2018-06-28

Family

ID=59065098

Family Applications (2)

Application Number Title Priority Date Filing Date
SG11201804482UA SG11201804482UA (en) 2015-12-21 2016-12-20 Search and retrieval data processing system for computing near real-time data aggregations
SG10202000071TA SG10202000071TA (en) 2015-12-21 2016-12-20 Search and retrieval data processing system for computing near real-time data aggregations

Family Applications After (1)

Application Number Title Priority Date Filing Date
SG10202000071TA SG10202000071TA (en) 2015-12-21 2016-12-20 Search and retrieval data processing system for computing near real-time data aggregations

Country Status (9)

Country Link
US (1) US11989096B2 (en)
EP (2) EP3798870B1 (en)
JP (2) JP6609052B2 (en)
CN (2) CN117194492A (en)
AU (1) AU2016378560B2 (en)
CA (1) CA3009062C (en)
DE (1) DE112016005870T5 (en)
SG (2) SG11201804482UA (en)
WO (1) WO2017112697A1 (en)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10963517B2 (en) 2017-02-02 2021-03-30 Kensho Technologies, Llc Graphical user interface for displaying search engine results
US11625437B2 (en) 2017-02-02 2023-04-11 Kensho Technologies, Llc Graphical user interface for displaying search engine results
US10726071B2 (en) * 2017-02-02 2020-07-28 Kensho Technologies, Llc Content search engine
US11947978B2 (en) 2017-02-23 2024-04-02 Ab Initio Technology Llc Dynamic execution of parameterized applications for the processing of keyed network data streams
US10831509B2 (en) 2017-02-23 2020-11-10 Ab Initio Technology Llc Dynamic execution of parameterized applications for the processing of keyed network data streams
US11016824B1 (en) * 2017-06-12 2021-05-25 Pure Storage, Inc. Event identification with out-of-order reporting in a cloud-based environment
US11288258B2 (en) 2018-09-25 2022-03-29 Ab Initio Technology Llc Dedicated audit port for implementing recoverability in outputting audit data
CN112579638B (en) * 2019-09-29 2024-02-13 北京国双科技有限公司 Behavior tag information processing method and device, computer equipment and storage medium
CN111078996A (en) * 2019-11-12 2020-04-28 北京币世界网络科技有限公司 Block chain digital currency real-time heat monitoring method, device and system
CN111291082B (en) * 2020-01-20 2023-10-31 北京百度网讯科技有限公司 Data aggregation processing method, device, equipment and storage medium
US11233640B2 (en) * 2020-05-13 2022-01-25 Ridgeline, Inc. Mutation processing for events
US11949784B2 (en) 2020-05-13 2024-04-02 Ridgeline, Inc. Auditing for events
DE102021200190A1 (en) 2021-01-11 2022-07-14 Geze Gmbh Method for providing a configuration record of an entity
WO2022177904A1 (en) * 2021-02-16 2022-08-25 Ab Initio Technology Llc Dynamic execution of parameterized applications for the processing of keyed network data streams
US20230300112A1 (en) * 2022-03-21 2023-09-21 Sophos Limited Aggregating security events

Family Cites Families (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5966072A (en) 1996-07-02 1999-10-12 Ab Initio Software Corporation Executing computations expressed as graphs
US6584581B1 (en) 1999-12-06 2003-06-24 Ab Initio Software Corporation Continuous flow checkpointing data processing
US6721749B1 (en) * 2000-07-06 2004-04-13 Microsoft Corporation Populating a data warehouse using a pipeline approach
US6965886B2 (en) * 2001-11-01 2005-11-15 Actimize Ltd. System and method for analyzing and utilizing data, by executing complex analytical models in real time
JP4025572B2 (en) * 2002-04-05 2007-12-19 日本電信電話株式会社 Structured document analysis device and method, and storage medium storing structured document analysis program and structured document analysis program
US7689616B2 (en) * 2005-04-15 2010-03-30 Microsoft Corporation Techniques for specifying and collecting data aggregations
US8112399B2 (en) * 2005-11-07 2012-02-07 International Business Machines Corporation Method and apparatus for configurable data aggregation in a data warehouse
US7831617B2 (en) * 2006-07-25 2010-11-09 Microsoft Corporation Re-categorization of aggregate data as detail data and automated re-categorization based on data usage context
US8069129B2 (en) 2007-04-10 2011-11-29 Ab Initio Technology Llc Editing and compiling business rules
US8374986B2 (en) 2008-05-15 2013-02-12 Exegy Incorporated Method and system for accelerated stream processing
US20130262035A1 (en) * 2012-03-28 2013-10-03 Michael Charles Mills Updating rollup streams in response to time series of measurement data
US8595234B2 (en) * 2010-05-17 2013-11-26 Wal-Mart Stores, Inc. Processing data feeds
WO2011163363A1 (en) 2010-06-22 2011-12-29 Ab Initio Technology Llc Processing related datasets
WO2012068557A1 (en) * 2010-11-18 2012-05-24 Wal-Mart Stores, Inc. Real-time analytics of streaming data
US8856807B1 (en) * 2011-01-04 2014-10-07 The Pnc Financial Services Group, Inc. Alert event platform
US8589375B2 (en) * 2011-01-31 2013-11-19 Splunk Inc. Real time searching and reporting
TW201322022A (en) * 2011-11-24 2013-06-01 Alibaba Group Holding Ltd Distributed data stream processing method
AU2013214801B2 (en) * 2012-02-02 2018-06-21 Visa International Service Association Multi-source, multi-dimensional, cross-entity, multimedia database platform apparatuses, methods and systems
US11003687B2 (en) * 2012-05-15 2021-05-11 Splunk, Inc. Executing data searches using generation identifiers
US9185235B2 (en) 2012-08-02 2015-11-10 Ab Initio Technology Llc Aggregating data in a mediation system
US9892026B2 (en) 2013-02-01 2018-02-13 Ab Initio Technology Llc Data records selection
US20140324530A1 (en) 2013-04-30 2014-10-30 Liveops, Inc. Method and system for detecting patters in data streams
TR201809901T4 (en) * 2013-05-22 2018-07-23 Striim Inc Apparatus and method for sequential event processing in a dispersed environment.
US9442963B2 (en) * 2013-08-27 2016-09-13 Omnitracs, Llc Flexible time-based aggregated derivations for advanced analytics
US20150120607A1 (en) 2013-10-24 2015-04-30 Vonage Network Llc System and method for customer event email consolidation and delivery
US9934279B2 (en) * 2013-12-05 2018-04-03 Oracle International Corporation Pattern matching across multiple input data streams
EP3114620A1 (en) * 2014-03-07 2017-01-11 Systema Systementwicklung Dip.-Inf. Manfred Austen Gmbh Real-time information systems and methodology based on continuous homomorphic processing in linear information spaces
US9244978B2 (en) 2014-06-11 2016-01-26 Oracle International Corporation Custom partitioning of a data stream
CA3001304C (en) * 2015-06-05 2021-10-19 C3 Iot, Inc. Systems, methods, and devices for an enterprise internet-of-things application development platform
US10657134B2 (en) * 2015-08-05 2020-05-19 Ab Initio Technology Llc Selecting queries for execution on a stream of real-time data
EP3128423A1 (en) * 2015-08-06 2017-02-08 Hewlett-Packard Enterprise Development LP Distributed event processing
US20170061315A1 (en) * 2015-08-27 2017-03-02 Sas Institute Inc. Dynamic prediction aggregation
US10324773B2 (en) * 2015-09-17 2019-06-18 Salesforce.Com, Inc. Processing events generated by internet of things (IoT)
US10756991B2 (en) * 2015-09-17 2020-08-25 Salesforce.Com, Inc. Simplified entity engagement automation
US9965330B2 (en) * 2015-09-18 2018-05-08 Salesforce.Com, Inc. Maintaining throughput of a stream processing framework while increasing processing load

Also Published As

Publication number Publication date
JP6609052B2 (en) 2019-11-20
EP3798870A1 (en) 2021-03-31
EP3394772B1 (en) 2020-08-19
CA3009062C (en) 2021-10-26
CN108780466B (en) 2023-08-11
US20170177446A1 (en) 2017-06-22
US11989096B2 (en) 2024-05-21
AU2016378560B2 (en) 2020-03-05
WO2017112697A1 (en) 2017-06-29
JP7082961B2 (en) 2022-06-09
SG10202000071TA (en) 2020-03-30
EP3394772A1 (en) 2018-10-31
JP2019507403A (en) 2019-03-14
JP2020030842A (en) 2020-02-27
CN108780466A (en) 2018-11-09
CN117194492A (en) 2023-12-08
EP3798870B1 (en) 2023-05-03
DE112016005870T5 (en) 2018-09-06
AU2016378560A1 (en) 2018-06-21
CA3009062A1 (en) 2017-06-29

Similar Documents

Publication Publication Date Title
SG11201804482UA (en) Search and retrieval data processing system for computing near real-time data aggregations
SG11201903551RA (en) Performing virtual reality input
SG11201904942YA (en) Blockchain-based service execution method and apparatus, and electronic device
SG11201906476TA (en) Login information processing method and device
SG11201901550WA (en) Method and apparatus for data processing
SG11201907125XA (en) Dynamic execution of parameterized applications for the processing of keyed network data streams
SG11201806541RA (en) Image classification and labeling
SG11201903108UA (en) Order information determination method and apparatus
SG11201903958SA (en) Intuitive occluded object indicator
SG11201908886TA (en) Consensus node selection method and apparatus, and server
SG11201808144SA (en) Computer systems and methods for creating asset-related tasks based on predictive models
SG11201901577SA (en) Method and system for fast tracking navigation of blockchains via data manipulation
SG11201804771WA (en) Systems and methods for providing financial data to financial instruments in a distributed ledger system
SG11201808474YA (en) Method and system for desychronization recovery for permissioned blockchains using bloom filters
CA3015496A1 (en) Voice control of a media playback system
SG11201903141QA (en) Business processing method and apparatus
SG11201901595PA (en) Aggregating service data for transmission and risk analysis
SG11201807307VA (en) System and method for aerial system discrimination and action
SG11201900116RA (en) Communication flow for verification and identification check
SG11201906372PA (en) Computer-implemented system and method for generating and extracting user related data stored on a blockchain
SG11201804807VA (en) Computer architecture and method for modifying data intake parameters based on a predictive model
SG11201803666WA (en) Single image detection
SG11201811492TA (en) Automation vending store
SG11201807001YA (en) Systems and methods for management of cloud computing resources for information systems
SG11202000495YA (en) Methods and systems for environmental credit scoring