SG11201708917SA - Data processing method and system - Google Patents

Data processing method and system

Info

Publication number
SG11201708917SA
SG11201708917SA SG11201708917SA SG11201708917SA SG11201708917SA SG 11201708917S A SG11201708917S A SG 11201708917SA SG 11201708917S A SG11201708917S A SG 11201708917SA SG 11201708917S A SG11201708917S A SG 11201708917SA SG 11201708917S A SG11201708917S A SG 11201708917SA
Authority
SG
Singapore
Prior art keywords
data processing
processing method
data
processing
Prior art date
Application number
SG11201708917SA
Other languages
English (en)
Inventor
Min Han
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Publication of SG11201708917SA publication Critical patent/SG11201708917SA/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2272—Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2219—Large Object storage; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24534—Query rewriting; Transformation
    • G06F16/24539—Query rewriting; Transformation using cached or materialised query results
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F7/06—Arrangements for sorting, selecting, merging, or comparing data on individual record carriers
    • G06F7/08—Sorting, i.e. grouping record carriers in numerical or other ordered sequence according to the classification of at least some of the information they carry
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Mobile Radio Communication Systems (AREA)
  • Telephonic Communication Services (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
SG11201708917SA 2015-05-04 2016-04-21 Data processing method and system SG11201708917SA (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510222356.4A CN106202092B (zh) 2015-05-04 2015-05-04 数据处理的方法及***
PCT/CN2016/079812 WO2016177279A1 (zh) 2015-05-04 2016-04-21 数据处理的方法及***

Publications (1)

Publication Number Publication Date
SG11201708917SA true SG11201708917SA (en) 2017-11-29

Family

ID=57218083

Family Applications (1)

Application Number Title Priority Date Filing Date
SG11201708917SA SG11201708917SA (en) 2015-05-04 2016-04-21 Data processing method and system

Country Status (9)

Country Link
US (2) US10592491B2 (pl)
EP (1) EP3293641B1 (pl)
JP (1) JP6779231B2 (pl)
KR (1) KR102134952B1 (pl)
CN (1) CN106202092B (pl)
ES (1) ES2808948T3 (pl)
PL (1) PL3293641T3 (pl)
SG (1) SG11201708917SA (pl)
WO (1) WO2016177279A1 (pl)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106202092B (zh) 2015-05-04 2020-03-06 阿里巴巴集团控股有限公司 数据处理的方法及***
WO2017107118A1 (en) * 2015-12-24 2017-06-29 Intel Corporation Facilitating efficient communication and data processing across clusters of computing machines in heterogeneous computing environment
CN107729353B (zh) * 2017-08-30 2020-04-07 第四范式（北京）技术有限公司 用于执行机器学习的分布式***及其方法
US11044091B1 (en) * 2018-03-15 2021-06-22 Secure Channels Inc. System and method for securely transmitting non-pki encrypted messages
CN110347513B (zh) * 2019-07-15 2022-05-20 中国工商银行股份有限公司 热点数据批量调度方法及装置
US11804955B1 (en) 2019-09-13 2023-10-31 Chol, Inc. Method and system for modulated waveform encryption
JP7397179B2 (ja) * 2019-12-20 2023-12-12 インテル・コーポレーション 階層化オブジェクトメモリ配置のためのランタイム装置の管理
CN114443176A (zh) * 2020-10-30 2022-05-06 武汉斗鱼鱼乐网络科技有限公司 一种设置第三方软件的热键的方法以及相关设备
CN112650752A (zh) * 2020-12-22 2021-04-13 深圳壹账通智能科技有限公司 热点key的确定方法、装置、设备及存储介质
CN116432903B (zh) * 2023-04-01 2024-06-11 国网新疆电力有限公司电力科学研究院 一种通信仿真数据管理***
CN116346827B (zh) * 2023-05-30 2023-08-11 中国地质大学（北京） 一种面向倾斜数据流的实时分组方法及***

Family Cites Families (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7756919B1 (en) * 2004-06-18 2010-07-13 Google Inc. Large-scale data processing in a distributed and parallel processing enviornment
US8726290B2 (en) * 2008-06-12 2014-05-13 Yahoo! Inc. System and/or method for balancing allocation of data among reduce processes by reallocation
CN101645067A (zh) 2008-08-05 2010-02-10 北京大学 一种讨论区集合中热点讨论区的预测方法
JP5229731B2 (ja) * 2008-10-07 2013-07-03 インターナショナル・ビジネス・マシーンズ・コーポレーション 更新頻度に基づくキャッシュ機構
US8370493B2 (en) * 2008-12-12 2013-02-05 Amazon Technologies, Inc. Saving program execution state
EP2325762A1 (en) * 2009-10-27 2011-05-25 Exalead Method and system for processing information of a stream of information
CN102141995B (zh) * 2010-01-29 2013-06-12 国际商业机器公司 简化并行计算***中的传输的***与方法
CN102236581B (zh) * 2010-04-30 2013-08-14 国际商业机器公司 用于数据中心的映射化简方法和***
CN102314336B (zh) * 2010-07-05 2016-04-13 深圳市腾讯计算机***有限公司 一种数据处理方法和***
CN102456031B (zh) * 2010-10-26 2016-08-31 腾讯科技（深圳）有限公司 一种MapReduce***和处理数据流的方法
JP5552449B2 (ja) * 2011-01-31 2014-07-16 日本電信電話株式会社 データ分析及び機械学習処理装置及び方法及びプログラム
US20120304186A1 (en) * 2011-05-26 2012-11-29 International Business Machines Corporation Scheduling Mapreduce Jobs in the Presence of Priority Classes
JP5637071B2 (ja) * 2011-05-27 2014-12-10 富士通株式会社 処理プログラム、処理方法及び処理装置
CN103019614B (zh) 2011-09-23 2015-11-25 阿里巴巴集团控股有限公司 分布式存储***管理装置及方法
CN102388374A (zh) * 2011-09-28 2012-03-21 华为技术有限公司 存储数据的方法和装置
EP2765510A4 (en) * 2011-10-06 2016-07-06 Fujitsu Ltd DATA PROCESSING, DISTRIBUTED PROCESSING SYSTEM AND PROGRAM
TWI461929B (zh) 2011-12-09 2014-11-21 Promise Tecnnology Inc 雲端數據儲存系統
JP5919825B2 (ja) * 2012-01-05 2016-05-18 富士通株式会社 データ処理方法、分散処理システムおよびプログラム
US9367601B2 (en) * 2012-03-26 2016-06-14 Duke University Cost-based optimization of configuration parameters and cluster sizing for hadoop
WO2013153620A1 (ja) * 2012-04-10 2013-10-17 株式会社日立製作所 データ処理システム及びデータ処理方法
TWI610166B (zh) 2012-06-04 2018-01-01 飛康國際網路科技股份有限公司 自動災難復原和資料遷移系統及方法
WO2014020735A1 (ja) * 2012-08-02 2014-02-06 富士通株式会社 データ処理方法、情報処理装置およびプログラム
US20150363467A1 (en) 2013-01-31 2015-12-17 Hewlett-Packard Development Company, L.P. Performing an index operation in a mapreduce environment
CN104077297B (zh) * 2013-03-27 2017-05-17 日电（中国）有限公司 基于本体的查询方法及装置
CN104142950A (zh) 2013-05-10 2014-11-12 中国人民大学 基于关键词提取和基尼系数的微博用户分类方法
US9424274B2 (en) * 2013-06-03 2016-08-23 Zettaset, Inc. Management of intermediate data spills during the shuffle phase of a map-reduce job
IN2013MU02918A (pl) * 2013-09-10 2015-07-03 Tata Consultancy Services Ltd
CN103838844B (zh) * 2014-03-03 2018-01-19 珠海市君天电子科技有限公司 一种键值对数据存储、传输方法及装置
CN103995882B (zh) 2014-05-28 2017-07-07 南京大学 基于MapReduce的概率频繁项集挖掘方法
CN104331464A (zh) * 2014-10-31 2015-02-04 许继电气股份有限公司 一种基于MapReduce的监控数据优先预取处理方法
CN104536830A (zh) 2015-01-09 2015-04-22 哈尔滨工程大学 一种基于MapReduce的KNN文本分类方法
CN106202092B (zh) 2015-05-04 2020-03-06 阿里巴巴集团控股有限公司 数据处理的方法及***
CN107193500A (zh) 2017-05-26 2017-09-22 郑州云海信息技术有限公司 一种分布式文件***分层存储方法及***

Also Published As

Publication number Publication date
KR20180002758A (ko) 2018-01-08
PL3293641T3 (pl) 2021-02-08
EP3293641A4 (en) 2018-10-17
CN106202092A (zh) 2016-12-07
JP2018515844A (ja) 2018-06-14
CN106202092B (zh) 2020-03-06
JP6779231B2 (ja) 2020-11-04
EP3293641A1 (en) 2018-03-14
WO2016177279A1 (zh) 2016-11-10
KR102134952B1 (ko) 2020-07-17
US20200192882A1 (en) 2020-06-18
US10592491B2 (en) 2020-03-17
US20180046658A1 (en) 2018-02-15
US10872070B2 (en) 2020-12-22
ES2808948T3 (es) 2021-03-02
EP3293641B1 (en) 2020-06-17

Similar Documents

Publication Publication Date Title
ZA201800870B (en) Data processing method and device
IL255644B (en) Data processing apparatus and method
GB201712642D0 (en) Order processing method and system
EP3353671A4 (en) Distributed data processing method and system
GB201613109D0 (en) Computer implemented method and system
SG11201709271SA (en) Interaction data processing method and apparatus
SG11201702373YA (en) System and method for information processing
SG11201703410YA (en) Data processing method, apparatus, and system
PL3376389T3 (pl) Sposób i urządzenie do przetwarzania danych
EP2988230A4 (en) DATA PROCESSING METHOD AND COMPUTER SYSTEM
EP3367250A4 (en) Information processing system and information processing method
HK1218360A1 (zh) 字幕數據的處理方法及系統
SG11201708917SA (en) Data processing method and system
GB201700081D0 (en) Data processing method and apparatus
HK1211122A1 (en) Method and system for processing periodic order information
EP3382573A4 (en) INFORMATION PROCESSING SYSTEM AND INFORMATION PROCESSING METHOD
EP3367249A4 (en) Information processing system and information processing method
EP3358836A4 (en) INFORMATION PROCESSING SYSTEM AND INFORMATION PROCESSING METHOD
SG11201710887RA (en) Information processing apparatus and information processing method
GB201520023D0 (en) Method and system for processing image data
EP3276618A4 (en) Information processing system and information processing method
EP3276484A4 (en) Information processing system and information processing method
EP3347828A4 (en) SYSTEM AND METHOD FOR DATA PROCESSING
EP3346453A4 (en) CONTROL PROCESSING SYSTEM AND CONTROL PROCESSING METHOD
SG11201709848SA (en) Transaction processing method and system