SG11201907776WA - Replication lag-constrained deletion of data in a large-scale distributed data storage system - Google Patents

Replication lag-constrained deletion of data in a large-scale distributed data storage system

Info

Publication number
SG11201907776WA
SG11201907776WA SG11201907776WA SG11201907776WA SG11201907776WA SG 11201907776W A SG11201907776W A SG 11201907776WA SG 11201907776W A SG11201907776W A SG 11201907776WA SG 11201907776W A SG11201907776W A SG 11201907776WA SG 11201907776W A SG11201907776W A SG 11201907776WA
Authority
SG
Singapore
Prior art keywords
lag
international
deletion
constrained
replication
Prior art date
Application number
SG11201907776WA
Inventor
Mehant Baid
Bogdan Munteanu
Daniel Tahara
Original Assignee
Dropbox Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dropbox Inc filed Critical Dropbox Inc
Publication of SG11201907776WA publication Critical patent/SG11201907776WA/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2365—Ensuring data consistency and integrity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • G06F16/2315—Optimistic concurrency control
    • G06F16/2322—Optimistic concurrency control using timestamps
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/275—Synchronous replication

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) (19) World Intellectual Property Organization International Bureau (43) International Publication Date 29 November 2018 (29.11.2018) WIPO I PCT °Ens ono oIIIOH0100111DIDMDIIII1111101 (10) International Publication Number WO 2018/217244 Al (51) International Patent Classification: GOOF 17/30 (2006.01) (21) International Application Number: PCT/US2018/015803 (22) International Filing Date: 29 January 2018 (29.01.2018) (25) Filing Language: English (26) Publication Language: English (30) Priority Data: 15/601,094 22 May 2017 (22.05.2017) US (71) Applicant: DROPBOX, INC. [US/US]; 333 Brannan Street, San Francisco, CA 94107 (US). (72) Inventors: BAID, Mehant; 333 Brannan Street, San Fran- cisco, CA 94107 (US). MUNTEANU, Bogdan; 333 Bran- nan Street, San Francisco, CA 94107 (US). TAHARA, Daniel, K.; 333 Brannan Street, San Francisco, CA 94107 (US). (74) Agent: ORICH, Christine, E. et al.; Hickman Palermo Becker Bingham LLP, 1 Almaden Boulevard, San Jose, CA 95113 (US). (81) Designated States (unless otherwise indicated, for every kind of national protection available): AE, AG, AL, AM, AO, AT, AU, AZ, BA, BB, BG, BH, BN, BR, BW, BY, BZ, CA, CH, CL, CN, CO, CR, CU, CZ, DE, DJ, DK, DM, DO, DZ, EC, EE, EG, ES, FI, GB, GD, GE, GH, GM, GT, HN, HR, HU, ID, IL, IN, IR, IS, JO, JP, KE, KG, KH, KN, KP, KR, KW, KZ, LA, LC, LK, LR, LS, LU, LY, MA, MD, ME, MG, MK, MN, MW, MX, MY, MZ, NA, NG, NI, NO, NZ, OM, PA, PE, PG, PH, PL, PT, QA, RO, RS, RU, RW, SA, SC, SD, SE, SG, SK, SL, SM, ST, SV, SY, TH, TJ, TM, TN, TR, TT, TZ, UA, UG, US, UZ, VC, VN, ZA, ZM, ZW. (84) Designated States (unless otherwise indicated, for every kind of regional protection available): ARIPO (BW, GH, GM, KE, LR, LS, MW, MZ, NA, RW, SD, SL, ST, SZ, TZ, UG, ZM, ZW), Eurasian (AM, AZ, BY, KG, KZ, RU, TJ, TM), European (AL, AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, FI, FR, GB, GR, HR, HU, IE, IS, IT, LT, LU, LV, MC, MK, MT, NL, NO, PL, PT, RO, RS, SE, SI, SK, SM, TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, KM, ML, MR, NE, SN, TD, TG). Published: — with international search report (Art. 21(3)) (54) Title: REPLICATION LAG-CONSTRAINED DELETION OF DATA IN A LARGE-SCALE DISTRIBUTED DATA STORAGE = SYSTEM 500 OBTAIN DELETE TASK (57) : Computer-implemented techniques for replication-lag constrained deletion of data in a distributed data storage system. In some aspects, the techniques improve the operation of a computing system by preventing too high of a delete rate that causes severe replication lag while at the same time increasing and decreasing the delete rate over time to a maximum allowable delete rate constrained by measured repli- cation lag in terms of both local replication lag and geographic replica- tion lag. In one implementation, the delete rate is adjusted by increasing 506 or decreasing a pause interval that determines how long a database da- ta deletion process pauses between submitting database deletion com- mands to a database server. END W O 20 18/ 217 244 Al Figure 5
SG11201907776WA 2017-05-22 2018-01-29 Replication lag-constrained deletion of data in a large-scale distributed data storage system SG11201907776WA (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15/601,094 US10007695B1 (en) 2017-05-22 2017-05-22 Replication lag-constrained deletion of data in a large-scale distributed data storage system
PCT/US2018/015803 WO2018217244A1 (en) 2017-05-22 2018-01-29 Replication lag-constrained deletion of data in a large-scale distributed data storage system

Publications (1)

Publication Number Publication Date
SG11201907776WA true SG11201907776WA (en) 2019-09-27

Family

ID=61231321

Family Applications (1)

Application Number Title Priority Date Filing Date
SG11201907776WA SG11201907776WA (en) 2017-05-22 2018-01-29 Replication lag-constrained deletion of data in a large-scale distributed data storage system

Country Status (7)

Country Link
US (2) US10007695B1 (en)
EP (1) EP3612954B1 (en)
JP (1) JP6653789B1 (en)
AU (2) AU2018273541B2 (en)
CA (1) CA3058765C (en)
SG (1) SG11201907776WA (en)
WO (1) WO2018217244A1 (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10545988B2 (en) * 2015-02-26 2020-01-28 Red Hat, Inc. System and method for data synchronization using revision control
US10007695B1 (en) * 2017-05-22 2018-06-26 Dropbox, Inc. Replication lag-constrained deletion of data in a large-scale distributed data storage system
US10114857B1 (en) 2017-11-13 2018-10-30 Lendingclub Corporation Techniques for performing multi-system computer operations
US11354301B2 (en) 2017-11-13 2022-06-07 LendingClub Bank, National Association Multi-system operation audit log
US10884820B1 (en) * 2018-08-31 2021-01-05 Veritas Technologies Llc Intelligent and automatic replication load score based load balancing and resiliency of replication appliances
CN110287183B (en) * 2019-05-23 2024-02-02 中国平安人寿保险股份有限公司 Processing method and device for database table water level, computer equipment and storage medium
US11263237B2 (en) * 2019-06-04 2022-03-01 Veritas Technologies Llc Systems and methods for storage block replication in a hybrid storage environment
US11409711B2 (en) * 2019-12-03 2022-08-09 Western Digital Technologies, Inc. Barriers for dependent operations among sharded data stores
US11567899B2 (en) * 2019-12-03 2023-01-31 Western Digital Technologies, Inc. Managing dependent delete operations among data stores
US11301462B1 (en) * 2020-03-31 2022-04-12 Amazon Technologies, Inc. Real-time data validation using lagging replica databases
CN111475493B (en) * 2020-06-19 2020-12-04 阿里云计算有限公司 Data reading method and device
US20230134759A1 (en) * 2021-11-01 2023-05-04 Ic Manage Inc Heliotropic work from home time zone expedition server coordinates Evolving FileTile (EFT) updates among local computation centers (LCC) by selectively relaying indicia As Soon After Commitment (ASAC) into version control to cause inter-center EFT demands to be queued earlier than local application start

Family Cites Families (66)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5319752A (en) * 1992-09-18 1994-06-07 3Com Corporation Device with host indication combination
US6502106B1 (en) * 1999-03-25 2002-12-31 International Business Machines Corporation System, method, and program for accessing secondary storage in a network system
US7665111B1 (en) * 1999-10-20 2010-02-16 Tivo Inc. Data storage management and scheduling system
US6442569B1 (en) * 1999-04-26 2002-08-27 General Electric Company Apparatus and method for data transfer between databases
US7388839B2 (en) * 2003-10-22 2008-06-17 International Business Machines Corporation Methods, apparatus and computer programs for managing performance and resource utilization within cluster-based systems
US7323970B1 (en) * 2004-01-21 2008-01-29 Numerex Corporation Method and system for remote interaction with a vehicle via wireless communication
JP2005258850A (en) * 2004-03-12 2005-09-22 Hitachi Ltd Computer system
JP4452533B2 (en) * 2004-03-19 2010-04-21 株式会社日立製作所 System and storage system
US7706822B2 (en) * 2005-08-24 2010-04-27 Motorola, Inc. Timing synchronization and beacon generation for mesh points operating in a wireless mesh network
US20070061379A1 (en) * 2005-09-09 2007-03-15 Frankie Wong Method and apparatus for sequencing transactions globally in a distributed database cluster
JP4757109B2 (en) * 2006-06-23 2011-08-24 富士通株式会社 Data communication program
US7702871B1 (en) * 2007-08-31 2010-04-20 Emc Corporation Write pacing
EP2076068A1 (en) * 2007-12-26 2009-07-01 Alcatel Lucent Handover method and apparatus in a wireless telecommunications network
US7523213B1 (en) * 2008-05-20 2009-04-21 International Business Machines Corporation Efficient approach with the toleration of stale data to dynamically transform and unify data quality in client and server with continuous transaction flows
JP5343399B2 (en) * 2008-05-22 2013-11-13 富士通株式会社 Management program, management method, and management apparatus
US7962458B2 (en) * 2008-06-12 2011-06-14 Gravic, Inc. Method for replicating explicit locks in a data replication engine
US8301593B2 (en) * 2008-06-12 2012-10-30 Gravic, Inc. Mixed mode synchronous and asynchronous replication system
US8346958B2 (en) * 2008-12-04 2013-01-01 International Business Machines Corporation Rate control technique for a lightweight directory access protocol over MQSeries (LoM) server
US8635250B2 (en) * 2010-04-26 2014-01-21 Vinod Mehra Methods and systems for deleting large amounts of data from a multitenant database
US9137288B2 (en) * 2010-12-20 2015-09-15 Yahoo! Inc. Scalable push-based architecture for web applications
US10740353B2 (en) * 2010-12-23 2020-08-11 Mongodb, Inc. Systems and methods for managing distributed database deployments
US8843714B1 (en) * 2011-03-30 2014-09-23 Emc Corporation Write pacing simulation tool
US20120290529A1 (en) * 2011-05-09 2012-11-15 Honeywell International Inc. Systems and methods for updating a database and handling interruptions
US8751863B2 (en) * 2011-05-23 2014-06-10 Microsoft Corporation Implementing failover processes between storage stamps
US20130166505A1 (en) * 2011-12-22 2013-06-27 Noam Peretz Monitoring replication lag between geographically dispersed sites
US8805940B2 (en) * 2012-02-28 2014-08-12 Microsoft Corporation Enhanced replication for message services
US9256506B1 (en) * 2012-06-04 2016-02-09 Google Inc. System and method for performing operations on target servers
US9251194B2 (en) * 2012-07-26 2016-02-02 Microsoft Technology Licensing, Llc Automatic data request recovery after session failure
US10152398B2 (en) * 2012-08-02 2018-12-11 At&T Intellectual Property I, L.P. Pipelined data replication for disaster recovery
US10915549B2 (en) * 2012-09-28 2021-02-09 Oracle International Corporation Techniques for keeping a copy of a pluggable database up to date with its source pluggable database in read-write mode
US10635674B2 (en) * 2012-09-28 2020-04-28 Oracle International Corporation Migrating a pluggable database between database server instances with minimal impact to performance
US8838535B2 (en) * 2012-10-05 2014-09-16 Oracle International Corporation Providing services across systems that manage distributed replicas
US9417917B1 (en) * 2012-12-14 2016-08-16 Amazon Technologies, Inc. Equitable resource allocation for storage object deletion
US20140188801A1 (en) * 2012-12-28 2014-07-03 Verizon Patent And Licensing Inc. Method and system for intelligent load balancing
US9824132B2 (en) * 2013-01-08 2017-11-21 Facebook, Inc. Data recovery in multi-leader distributed systems
US10599671B2 (en) * 2013-01-17 2020-03-24 Box, Inc. Conflict resolution, retry condition management, and handling of problem files for the synchronization client to a cloud-based platform
US20140229544A1 (en) * 2013-02-12 2014-08-14 BackPeddle, LLC Sharing content in social networks
US9189510B2 (en) * 2013-02-26 2015-11-17 Facebook, Inc. System and method for implementing cache consistent regional clusters
US9418129B2 (en) * 2013-03-08 2016-08-16 Oracle International Corporation Adaptive high-performance database redo log synchronization
US9229999B2 (en) * 2013-03-15 2016-01-05 International Business Machines Corporation Using average replication latency to influence routing query transactions
IN2013CH01467A (en) * 2013-04-01 2015-10-02 Sanovi Technologies Pvt Ltd
EP2988220B1 (en) * 2013-04-16 2020-09-16 Hitachi, Ltd. Computer system, computer-system management method, and program
JP5748932B2 (en) * 2013-04-30 2015-07-15 株式会社日立製作所 Computer system and method for supporting analysis of asynchronous remote replication
WO2015072026A1 (en) * 2013-11-18 2015-05-21 株式会社日立製作所 Computer system
JP2015161999A (en) * 2014-02-26 2015-09-07 富士通株式会社 Storage device, control apparatus, and control program
US9507844B2 (en) * 2014-03-28 2016-11-29 International Business Machines Corporation Automatic adjustment of data replication based on data access
US9513839B2 (en) * 2014-04-03 2016-12-06 Hitachi, Ltd. Management system and management method for computer system comprising remote copy system for performing asynchronous remote copy
US9396052B2 (en) * 2014-04-30 2016-07-19 Oracle International Corporation Periodic validation and health reports of disaster recovery plan
CN105530273B (en) * 2014-09-29 2018-08-03 国际商业机器公司 method and apparatus for data migration
US20160140132A1 (en) * 2014-11-19 2016-05-19 Unisys Corporation Online redistribution
US10021120B1 (en) * 2015-11-09 2018-07-10 8X8, Inc. Delayed replication for protection of replicated databases
US10043026B1 (en) * 2015-11-09 2018-08-07 8X8, Inc. Restricted replication for protection of replicated databases
US20170139716A1 (en) * 2015-11-18 2017-05-18 Arm Limited Handling stalling event for multiple thread pipeline, and triggering action based on information access delay
US10275508B2 (en) * 2015-11-19 2019-04-30 Sap Se Query hints for caching and replication
US10176215B2 (en) * 2015-11-24 2019-01-08 International Business Machines Corporation Data currency improvement for cross-site queries
US10496672B2 (en) * 2015-12-30 2019-12-03 EMC IP Holding Company LLC Creating replicas at user-defined points in time
US20170206148A1 (en) * 2016-01-20 2017-07-20 Facebook, Inc. Cross-region failover of application services
US10540238B2 (en) * 2016-01-29 2020-01-21 Veritas Technologies Llc Adjusting replication operations in a distributed environment
US20170293540A1 (en) * 2016-04-08 2017-10-12 Facebook, Inc. Failover of application services
US10262002B2 (en) * 2016-08-11 2019-04-16 International Business Machines Corporation Consistent execution of partial queries in hybrid DBMS
US10592528B2 (en) * 2017-02-27 2020-03-17 Sap Se Workload capture and replay for replicated database systems
US10068002B1 (en) * 2017-04-25 2018-09-04 Palantir Technologies Inc. Systems and methods for adaptive data replication
EP3616084A1 (en) * 2017-04-26 2020-03-04 Telefonaktiebolaget LM Ericsson (PUBL) Robust data replication among data centers
US11573947B2 (en) * 2017-05-08 2023-02-07 Sap Se Adaptive query routing in a replicated database environment
US10007695B1 (en) * 2017-05-22 2018-06-26 Dropbox, Inc. Replication lag-constrained deletion of data in a large-scale distributed data storage system
US10698920B2 (en) * 2017-12-22 2020-06-30 Dropbox, Inc. Techniques for a linearizable primary-secondary database system that replicates database data with eventual consistency

Also Published As

Publication number Publication date
WO2018217244A1 (en) 2018-11-29
EP3612954A1 (en) 2020-02-26
CA3058765A1 (en) 2018-11-29
EP3612954B1 (en) 2022-11-30
US10007695B1 (en) 2018-06-26
JP2020510893A (en) 2020-04-09
JP6653789B1 (en) 2020-02-26
US20180336237A1 (en) 2018-11-22
CA3058765C (en) 2021-06-22
AU2018273541B2 (en) 2019-07-18
AU2019250229B2 (en) 2021-04-08
AU2019250229A1 (en) 2019-11-07
US11226954B2 (en) 2022-01-18

Similar Documents

Publication Publication Date Title
SG11201907776WA (en) Replication lag-constrained deletion of data in a large-scale distributed data storage system
SG11201909168VA (en) Combination comprising a ppar agonist such as elafibranor and an acetyl-coa carboxylase (acc) inhibitor
SG11201809857TA (en) Anti-CTLA-4 Antibodies
SG11201906373VA (en) Chemical compounds as inhibitors of interleukin-1 activity
SG11201900274VA (en) Event trigger for independent links
SG11201903304YA (en) IL15/IL15Ra HETERODIMERIC FC-FUSION PROTEINS
SG11201909949XA (en) Targeted immunotolerance
SG11201807252QA (en) Anti-lag-3 antibodies
SG11201906418PA (en) Blockchain-based data processing method and device
SG11201806650VA (en) Systems and methods for providing a personal distributed ledger
SG11201805422WA (en) Multivalent and multispecific ox40-binding fusion proteins
SG11201907210YA (en) Policy communication via control plane signaling
SG11201805186VA (en) Compositions and methods for immune cell modulation in adoptive immunotherapies
SG11201900201YA (en) Methods for quantitating individual antibodies from a mixture
SG11201806404SA (en) Systems and methods for storing and sharing transactional data using distributed computer systems
SG11201804696RA (en) Techniques for metadata processing
SG11201811426UA (en) Distributed electronic record and transaction history
SG11201901310QA (en) Positioning reference signal enhancements
SG11201908847TA (en) Apparatuses, systems and methods for imaging flow cytometry
SG11201804836RA (en) Treatment of fibrosis
SG11201805072PA (en) Nicotine powder delivery system
SG11201902667UA (en) Methods and systems for chromatography data analysis
SG11201807474SA (en) Compositions comprising coformulation of anti-pd-l1 and anti-ctla-4 antibodies
SG11201810280YA (en) Spiro-lactam nmda receptor modulators and uses thereof
SG11201808306PA (en) Treatment of cancer with tg02