CN108268639A - A kind of index calculating method under big data environment - Google Patents
A kind of index calculating method under big data environment Download PDFInfo
- Publication number
- CN108268639A CN108268639A CN201810048169.2A CN201810048169A CN108268639A CN 108268639 A CN108268639 A CN 108268639A CN 201810048169 A CN201810048169 A CN 201810048169A CN 108268639 A CN108268639 A CN 108268639A
- Authority
- CN
- China
- Prior art keywords
- data
- index
- event
- information
- middle table
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/235—Update request formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2393—Updating materialised views
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses the index calculating methods under a kind of big data environment, include the following steps：Data are read from data storage server；Data are cleaned；SparkSQL is performed, carries out middle table calculating；By performing the SparkSQL finished writing in advance, game index is calculated；Judge to calculate whether the time is the same day, in this way the same day, then stop calculating, if not being the same day, then continue to calculate the index of next day.The logic that index calculates is write in sql, and sql is written database by SparkSQL functions, increases an index newly, it is only necessary to which several sql can just achieve the goal, and substantially carried out zero code and increased demand by the present invention completely；After index has been calculated, by extracting public field, achieve the purpose that compression index data, while also improve index inquiry velocity, this method can be widely applied in the system with simple statistics data function.
Description
Technical field
The invention belongs to the technical field of big data statistics, specifically, being related to the index meter under a kind of big data environment
Calculation method.
Background technology
With internet, Internet of Things, wireless sensor, cloud computing fast development, there is explosive increasing in global metadata amount
Long, human society enters the big data epoch.The feature of big data includes：
1. data volume is huge, from TB ranks, PB ranks are risen to；
2. data type is various, including network log, video, picture, geographical location information etc.；
3. value density is low, in the storage data of magnanimity, contain huge value, but practical significant data, just
An especially small part wherein；
For these features of big data, many major companies provide one-stop solution, from storage to calculating, such as
Ali's cloud and Amazon cloud；But Ali's cloud does not provide specific index numerical procedure, it merely provide one it is open
Service environment is exactly to sell server substantially；Amazon provides data warehouse for data statistics, but service is too expensive, not only needs
The interface of data warehouse is known quite well, and service code exploitation is also comparatively laborious, safeguarded relatively difficult.
Invention content
For deficiency above-mentioned in the prior art, the present invention proposes the index calculating method under a kind of big data environment, point
Scheduling code is calculated from index and business calculates SQL, the reading of data, pretreatment, calculating, the SQL of result storage is whole
Database is put into, in this way, can realize that zero code completes demand substantially.
In order to achieve the above object, the solution that uses of the present invention is：A kind of index calculating side under big data environment
Method includes the following steps：
S1, initialization obtain Job scheduling from database and index calculate SQL information, and to needing in SQL to be used
Parameter is initialized；
S2, according to initialization information, load the corresponding event data of game to memory from cloud storage service device, and create pair
The event view answered is used for subsequent step；
S3, such as event data have dirty data or wrong data, then using SparkSQL, to the event data of step S2 loadings
Carry out data cleansing；If event data does not have dirty data or wrong data, then step S4 is jumped directly to；
S4, middle table calculating is carried out, and create middle table view using SparkSQL；
S5, using event view and middle table view, carry out game index and calculate, after having been calculated, be inserted into database with
For displaying；
S6, according to time and system requirements are calculated, terminate this operation；Or continue update and calculate time, return to step S1, weight
Multiple operation.
Further, step S1 includes the following steps：
S11, according to database table data, tectonic event load information EventLoadMap, event load information
EventLoadMap includes the corresponding filename of event and event in cloud storage service device and is loaded into the view created after memory
Name；
S12, corresponding event number E is judged, if E<0, illustrate the event data that the game is loaded without needs, then terminate this
The entire calculation process of game；If E >=0 enters step S13；
S13, according to database data, tectonic event data cleansing information CleanDirtyDataMap, event data cleaning
Information CleanDirtyDataMap includes the view name for needing data cleansing and the SparkSQL for cleaning data；
S14, construction middle table calculate information MidDataMap, and middle table calculates information MidDataMap and regarded including middle table
The map title and the SparkSQL for calculating middle table；
S15, construction index calculate information MetricComputeMap, and index calculates information MetricComputeMap and includes
Index needs the table name for being inserted into database and the SpakrSQL for calculating the index after having been calculated.
Further, step S2 includes the following steps：
S21, zero-computing time ST is obtained；
The event load information EventLoadMap that S22, traversal step S11 are constructed, obtain event name EventName and
The view name ViewName that will be created；
S23, the event name EventName according to acquisition construct the incident file path of cloud storage service device
OssSource reads corresponding event data using Spark；
If the corresponding incident file of S24, view name exists, view is created according to view name ViewName；If file
It is not present, then enters step S25；
If S25, event load information EventLoadMap traversal terminate, S31 is entered step；Otherwise return to step S22,
Continue load events data.
Further, the data cleansing of step S3 includes the following steps：
The event data cleaning information CleanDirtyDataMap that S31, traversal step S13 are constructed, obtains event name
The ViewName and DeleteSQL that will be performed；
S32, the ViewName obtained according to step S31, initialize the parameter in DeleteSQL；
If S33, the corresponding event views of cleaning data SQL exist, SQL is performed, otherwise, enters step S34；
If S34, event data cleaning information CleanDirtyDataMap traversals terminate, S41 is entered step；Otherwise it returns
Step S31 is returned, continues purge event data.
Further, the middle table calculating of step S4 includes the following steps：
S41：The middle table that traversal step S14 is constructed calculates information MidDataMap, obtains middle table view name
MidDataViewName；
S42：According to middle table view name, middle table data are copied to memory from cloud storage service device；
S43：If there are the middle table data, view is created；If without the middle table data, middle table InitSql is performed,
Construct the middle table view structure；
S44：DeleteSql is performed, deletes the middle table data of ST periods；
S45：ComputeSql is performed, calculates middle table data, and re-create the middle table view；
S46：Using Spark, data are uploaded to oss；
S47：If middle table calculates information MidDataMap, traversal terminates, and enters step S51；If middle table calculates information
MidDataMap, which is not traversed, to be terminated, return to step S41, continues to calculate event data.
Further, the game index calculating of step S5 includes the following steps：
S51：The index that traversal step S15 is constructed calculates information MetricComputeMap, obtains index corresponding data
Library table name and the table include the calculating SQL set C of index；
S52：It traverses index and calculates SQL set C, and the SQL to traversing out carries out parameter initialization, then index of performance meter
It calculates；
S53：If having traversed set C, the index calculated is merged；If set C, return to step have not been traversed
S52；
S54：If desired more new historical achievement data then enters in next step, otherwise skips to step 57；
S55：History achievement data is read, and restores the index dimension of the data；
S56：New legacy data merging is carried out, updates the data the index dimensional information in library, and to the index number after merging
According to dimensional information mapped, replace with the id of dimensional information；
S57：According to sql is deleted, corresponding historical data is deleted, while be inserted into new achievement data；
S58：If index calculates information MetricComputeMap, traversal does not terminate, jumps to step S51, continues index
Calculating；Otherwise the index for terminating the ST periods calculates, and jumps to step S6.
The invention has the advantages that the logic that index calculates is write on sql by the present invention completely by SparkSQL functions
In, and database is written into sql, increase an index newly, it is only necessary to which several sql can just achieve the goal, and substantially carry out for zero generation
Code is increased demand；After index has been calculated, by extracting public field, achieve the purpose that compression index data, while also carry
High index inquiry velocity, this method can be widely applied in the system with simple statistics data function.
Description of the drawings
Fig. 1 is the computational methods flow chart of the present invention.
Fig. 2 is the computational methods initialization flowchart of the present invention.
Fig. 3 is that event loads flow chart in computational methods of the invention.
Fig. 4 is event data cleaning process figure in computational methods of the invention.
Fig. 5 is middle table calculation flow chart in computational methods of the invention.
Fig. 6 is index calculation flow chart in computational methods of the invention.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, the technical solution in the present invention is carried out below
It clearly and completely describes, it is clear that described embodiment is part of the embodiment of the present invention, instead of all the embodiments.Base
Embodiment in the present invention, those of ordinary skill in the art obtained without creative efforts it is all its
His embodiment, shall fall within the protection scope of the present invention.
Below in conjunction with attached drawing, the invention will be further described：
With reference to attached drawing 1- attached drawings 6, for the present invention by taking the technical field of online game as an example, the present invention provides a kind of big data ring
Index calculating method under border, includes the following steps：
S1, initialization obtain Job scheduling from database and index calculate SQL information, and to needing in SQL to be used
Parameter is initialized；
S2, according to initialization information, load the corresponding event data of game to memory from cloud storage service device, and create pair
The view answered is used for subsequent step；
S3, using SparkSQL, the event data of step S2 loadings is cleaned.The step is not required, if
Event data does not have dirty data or without wrong data, just jumps directly to step S4；
S4, middle table calculating is carried out, and create middle table view using SparkSQL, middle table is mainly stored some and gone through
History data, for calculating the index that some need to rely on historical data；
S5, using event view and middle table view, carry out game index and calculate, index calculates slightly more complex, packet
Some achievement datas are included with new, merging etc., after having been calculated, are inserted into database for displaying；
It is S6, last, according to time and system requirements are calculated, terminate whole flow process or continue update to calculate the time, again
Return to step S1, performs whole flow process again.
In the present embodiment, the initialization of step S1 includes following sub-step：
S11, according to database table data, tectonic event load information EventLoadMap, it includes two parts：Yun Cun
After the corresponding filename of event and event are loaded into memory in storage server, the view name of establishment；
S12, corresponding event number E is judged, if E<0, illustrate the event data that the game is loaded without needs, then terminate this
The entire calculation process of game；Otherwise, S13 is entered step；
S13, according to database data, tectonic event data cleansing information CleanDirtyDataMap, it includes two
Point：It needs the view name of data cleansing and cleans the SparkSQL of data, which can be carried out some parameter processings；
S14, construction middle table calculate information MidDataMap, it includes two parts：In middle table view name and calculating
Between table SparkSQL, the sql can by carry out some parameter processings；
S15, construction index calculate information MetricComputeMap, it includes two parts：After index has been calculated, need
It is inserted into the table name of database and calculates the SpakrSQL of the index, which can be carried out some parameter processings.
In the present embodiment, the game events data loading of step S2 includes following sub-step：
S21, zero-computing time ST is obtained, because this system is the file named according to the time to store the thing of game
Number of packages evidence, so firstly the need of the time is obtained；
The event load information EventLoadMap that S22, traversal step S11 are constructed, obtain event name EventName and
The view name ViewName that will be created；
S23, the event name EventName according to acquisition construct the incident file path of cloud storage service device
OssSource reads corresponding event data using Spark；
If the corresponding incident file of S24, view name exists, view is created according to view name ViewName；If file
It is not present, then enters step S25；
If S25, event load information EventLoadMap traversal terminate, S31 is entered step；Otherwise return to step S22,
Continue load events data.
In the present embodiment, the game events data cleansing of step S3 includes following sub-step：
The event cleaning information CleanDirtyDataMap that S31, traversal step S13 are constructed, obtains event name
The ViewName and DeleteSQL that will be performed；
S32, the ViewName obtained according to step S31, initialize the parameter in DeleteSQL；
If S33, the corresponding event views of cleaning data SQL exist, SQL is performed, otherwise, enters step S34；
If S25, event cleaning information CleanDirtyDataMap traversals terminate, S41 is entered step；Otherwise step is returned
Rapid S31 continues purge event data.
In the present embodiment, the game middle table calculating of step S4 includes following sub-step：
The middle table that S41, traversal step S14 are constructed calculates information MidDataMap, obtains middle table view name
MidDataViewName；
S42, according to middle table view name, copy middle table data to memory from cloud storage service device；
If S43, there are the middle table data, view is created；Otherwise, middle table InitSql is performed, the middle table is constructed and regards
Graph structure；
S44, deleteSql is performed, deletes the middle table data of ST periods；
S45, computeSql is performed, calculates middle table data, and re-create the middle table view；
S46, Spark, upload data to oss are utilized；
If S47, middle table calculate information MidDataMap, traversal terminates, and enters step S51；Otherwise return to step S41,
Continue to calculate event data.
In the present embodiment, the game index calculating of step S5 includes following sub-step：
The index that S51, traversal step S15 are constructed calculates information MetricComputeMap, obtains index corresponding data
Library table name and the table include the calculating SQL set C of index；
S52, traversal index calculate SQL set C, and the SQL to traversing out carries out parameter initialization, then index of performance meter
It calculates；
If S53, having traversed set C, the index calculated is merged；Otherwise, return to step S52；
S54, if desired more new historical achievement data then enter in next step, otherwise skip to step 57；
S55, history achievement data is read, and restores the index dimension of the data；
S56, new legacy data merging is carried out, updates the data the index dimensional information in library, and to the index number after merging
According to dimensional information mapped, replace with the id of dimensional information；
S57, according to delete sql, delete corresponding historical data, while be inserted into new achievement data；
If S58, index calculate information MetricComputeMap, traversal does not terminate, jumps to step S51, continues index
Calculating；Otherwise the index for terminating the ST periods calculates, and jumps to step S6.
Finally it should be noted that：The above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although
The present invention is described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that：It still may be used
To modify to the technical solution recorded in foregoing embodiments or carry out equivalent replacement to which part technical characteristic；
And these modification or replace, various embodiments of the present invention technical solution that it does not separate the essence of the corresponding technical solution spirit and
Range.
Claims (6)
1. the index calculating method under a kind of big data environment, it is characterized in that, include the following steps：
S1, initialization obtain Job scheduling from database and index calculate SQL information, and to needing the parameter in SQL to be used
It is initialized；
S2, according to initialization information, load the corresponding event data of game to memory, and create corresponding from cloud storage service device
Event view is used for subsequent step；
S3, such as event data have dirty data or wrong data, then using SparkSQL, the event data of step S2 loadings is carried out
Data cleansing；If event data does not have dirty data or wrong data, then step S4 is jumped directly to；
S4, middle table calculating is carried out, and create middle table view using SparkSQL；
S5, using event view and middle table view, carry out game index and calculate, after having been calculated, be inserted into database for exhibition
Show；
S6, according to time and system requirements are calculated, terminate this operation；Or continue update and calculate the time, return to step S1 repeats to grasp
Make.
2. the index calculating method under big data environment according to claim 1, it is characterized in that, the step S1 include with
Lower step：
S11, according to database table data, tectonic event load information EventLoadMap, event load information
EventLoadMap includes the corresponding filename of event and event in cloud storage service device and is loaded into the view created after memory
Name；
S12, corresponding event number E is judged, if E<0, illustrate the event data that the game is loaded without needs, then terminate the game
Entire calculation process；If E >=0 enters step S13；
S13, according to database data, tectonic event data cleansing information CleanDirtyDataMap, the event data cleaning
Information CleanDirtyDataMap includes the view name for needing data cleansing and the SparkSQL for cleaning data；
S14, construction middle table calculate information MidDataMap, and the middle table calculates information MidDataMap and regarded including middle table
The map title and the SparkSQL for calculating middle table；
S15, construction index calculate information MetricComputeMap, and the index calculates information MetricComputeMap and includes
Index needs the table name for being inserted into database and the SpakrSQL for calculating the index after having been calculated.
3. the index calculating method under big data environment according to claim 1, it is characterized in that, the step S2 include with
Lower step：
S21, zero-computing time ST is obtained；
The event load information EventLoadMap that S22, traversal step S11 are constructed obtains event name EventName and will
The view name ViewName of establishment；
S23, the event name EventName according to step S22 construct the incident file path of cloud storage service device
OssSource reads corresponding event data using Spark；
If the corresponding incident file of S24, view name exists, view is created according to view name ViewName；If file is not deposited
Then entering step S25；
If S25, the event load information EventLoadMap traversal terminate, S31 is entered step；Otherwise return to step S22,
Continue load events data.
4. the index calculating method under big data environment according to claim 1, it is characterized in that, the data of the step S3
Cleaning includes the following steps：
Described in S31, traversal step S13 event data clean information CleanDirtyDataMap, obtain event name ViewName and
The DeleteSQL that will be performed；
S32, the ViewName according to step S31 initialize the parameter in DeleteSQL；
If S33, the corresponding event views of cleaning data SQL exist, SQL is performed, otherwise, enters step S34；
If S34, the event data cleaning information CleanDirtyDataMap traversals terminate, S41 is entered step；Otherwise it returns
Step S31 is returned, continues purge event data.
5. the index calculating method under big data environment according to claim 1, it is characterized in that, the centre of the step S4
Meter includes the following steps：
S41：Middle table described in traversal step S14 calculates information MidDataMap, obtains middle table view name
MidDataViewName；
S42：According to middle table view name, middle table data are copied to memory from cloud storage service device；
S43：If there are the middle table data, view is created；If without the middle table data, middle table InitSql is performed,
Construct the middle table view structure；
S44：DeleteSql is performed, deletes the middle table data of ST periods；
S45：ComputeSql is performed, calculates middle table data, and re-create the middle table view；
S46：Using Spark, data are uploaded to oss；
S47：If the middle table calculates information MidDataMap, traversal terminates, and enters step S51；If the middle table calculates
Information MidDataMap, which is not traversed, to be terminated, return to step S41, continues to calculate event data.
6. the index calculating method under big data environment according to claim 1, it is characterized in that, the game of the step S5
Index calculating includes the following steps：
S51：Described in traversal step S15 index calculate information MetricComputeMap, obtain index correspondence database table name and
The table includes the calculating SQL set C of index；
S52：It traverses index and calculates SQL set C, and the SQL to traversing out carries out parameter initialization, then index of performance calculates；
S53：If having traversed set C, the index calculated is merged；If not traversed set C, return to step S52；
S54：If desired more new historical achievement data then enters in next step, otherwise skips to step 57；
S55：History achievement data is read, and restores the index dimension of the data；
S56：New legacy data merging is carried out, updates the data the index dimensional information in library, and to the achievement data after merging
Dimensional information is mapped, and replaces with the id of dimensional information；
S57：According to sql is deleted, corresponding historical data is deleted, while be inserted into new achievement data；
S58：If the index calculates information MetricComputeMap, traversal does not terminate, jumps to step S51, continues index
Calculating；Otherwise the index for terminating the ST periods calculates, and jumps to step S6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810048169.2A CN108268639B (en) | 2018-01-18 | 2018-01-18 | Index calculation method under big data environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810048169.2A CN108268639B (en) | 2018-01-18 | 2018-01-18 | Index calculation method under big data environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108268639A true CN108268639A (en) | 2018-07-10 |
| CN108268639B CN108268639B (en) | 2022-02-11 |
Family
ID=62776027
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810048169.2A Active CN108268639B (en) | 2018-01-18 | 2018-01-18 | Index calculation method under big data environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108268639B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109447443A (en) * | 2018-10-18 | 2019-03-08 | 阳光人寿保险股份有限公司 | Index calculating method and device |
| CN111104417A (en) * | 2019-12-05 | 2020-05-05 | 苏宁云计算有限公司 | Spark Sql external data source device, implementation method and system |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102194164A (en) * | 2011-04-18 | 2011-09-21 | 上海实时数据软件有限公司 | Index-system-model-based agile business intelligence data construction method |
| CN102289373A (en) * | 2011-07-28 | 2011-12-21 | 福建富士通信息软件有限公司 | Method for dynamically configuring index evaluation system |
| KR20130019674A (en) * | 2011-08-17 | 2013-02-27 | 삼성전자주식회사 | Terminal and method for performing application thereof |
| CN103455489A (en) * | 2012-05-28 | 2013-12-18 | 阿里巴巴集团控股有限公司 | Method and system for rapidly constructing key performance indicators (KPIs) of enterprises |
| JP2014130492A (en) * | 2012-12-28 | 2014-07-10 | Hitachi Ltd | Generation method for index and computer system |
| US20150248460A1 (en) * | 2012-10-31 | 2015-09-03 | Hitachi, Ltd. | Monitor system and monitor program |
| CN105373695A (en) * | 2015-09-18 | 2016-03-02 | 网易（杭州）网络有限公司 | Data communication method and apparatus for game servers |
| CN105426292A (en) * | 2015-10-29 | 2016-03-23 | 网易（杭州）网络有限公司 | Game log real-time processing system and method |
| CN106022651A (en) * | 2016-06-14 | 2016-10-12 | 深圳市迪博企业风险管理技术有限公司 | Risk early warning method based on business attribute and index system |
| CN106293891A (en) * | 2015-06-12 | 2017-01-04 | 交通银行股份有限公司 | Multidimensional investment target measure of supervision |
| CN107070716A (en) * | 2017-04-14 | 2017-08-18 | 腾讯科技（深圳）有限公司 | Game server state acquiring method, device, electronic equipment and storage medium |
| CN107122431A (en) * | 2017-04-14 | 2017-09-01 | 浙江数链科技有限公司 | A kind of real-time calculating platform and the method for computing data based on real-time calculating platform |
| CN107291806A (en) * | 2017-05-16 | 2017-10-24 | 浙江工业大学 | A kind of Data View copy alternative manner in Web visible environments |
-
2018
- 2018-01-18 CN CN201810048169.2A patent/CN108268639B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102194164A (en) * | 2011-04-18 | 2011-09-21 | 上海实时数据软件有限公司 | Index-system-model-based agile business intelligence data construction method |
| CN102289373A (en) * | 2011-07-28 | 2011-12-21 | 福建富士通信息软件有限公司 | Method for dynamically configuring index evaluation system |
| KR20130019674A (en) * | 2011-08-17 | 2013-02-27 | 삼성전자주식회사 | Terminal and method for performing application thereof |
| CN103455489A (en) * | 2012-05-28 | 2013-12-18 | 阿里巴巴集团控股有限公司 | Method and system for rapidly constructing key performance indicators (KPIs) of enterprises |
| US20150248460A1 (en) * | 2012-10-31 | 2015-09-03 | Hitachi, Ltd. | Monitor system and monitor program |
| JP2014130492A (en) * | 2012-12-28 | 2014-07-10 | Hitachi Ltd | Generation method for index and computer system |
| CN106293891A (en) * | 2015-06-12 | 2017-01-04 | 交通银行股份有限公司 | Multidimensional investment target measure of supervision |
| CN105373695A (en) * | 2015-09-18 | 2016-03-02 | 网易（杭州）网络有限公司 | Data communication method and apparatus for game servers |
| CN105426292A (en) * | 2015-10-29 | 2016-03-23 | 网易（杭州）网络有限公司 | Game log real-time processing system and method |
| CN106022651A (en) * | 2016-06-14 | 2016-10-12 | 深圳市迪博企业风险管理技术有限公司 | Risk early warning method based on business attribute and index system |
| CN107070716A (en) * | 2017-04-14 | 2017-08-18 | 腾讯科技（深圳）有限公司 | Game server state acquiring method, device, electronic equipment and storage medium |
| CN107122431A (en) * | 2017-04-14 | 2017-09-01 | 浙江数链科技有限公司 | A kind of real-time calculating platform and the method for computing data based on real-time calculating platform |
| CN107291806A (en) * | 2017-05-16 | 2017-10-24 | 浙江工业大学 | A kind of Data View copy alternative manner in Web visible environments |
Non-Patent Citations (3)
| Title |
|---|
| JINLIN LIU 等: "Research on Weight Calculation for Requirement Indexes of Marine Shaft System Based on Fuzzy Analytic Hierarchy Process", 《IEEE》 * |
| 周亮 等: "基于 Spark 的时态查询扩展与时态索引优化研究", 《计算机工程》 * |
| 张雷: "基于Spark***的查询分析及优化研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109447443A (en) * | 2018-10-18 | 2019-03-08 | 阳光人寿保险股份有限公司 | Index calculating method and device |
| CN111104417A (en) * | 2019-12-05 | 2020-05-05 | 苏宁云计算有限公司 | Spark Sql external data source device, implementation method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108268639B (en) | 2022-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9020802B1 (en) | Worldwide distributed architecture model and management | |
| US9280381B1 (en) | Execution framework for a distributed file system | |
| CN103136243B (en) | File system duplicate removal method based on cloud storage and device | |
| CN108595664B (en) | Agricultural data monitoring method in hadoop environment | |
| CN103106152B (en) | Based on the data dispatching method of level storage medium | |
| CN104933112A (en) | Distributed Internet transaction information storage and processing method | |
| CN104639658A (en) | Realization method for accessing object storage by file system mounting | |
| CN107766568A (en) | Effective query processing is carried out using the histogram in columnar database | |
| CN103473298B (en) | Data archiving method and device and storage system | |
| CN109766341A (en) | A kind of method, apparatus that establishing Hash mapping, storage medium | |
| CN106294745A (en) | Big data cleaning method and device | |
| CN102880709A (en) | Data warehouse management system and data warehouse management method | |
| CN107704203A (en) | It polymerize delet method, device, equipment and the computer-readable storage medium of big file | |
| CN103198027A (en) | Method and device for storing and providing files | |
| CN110727406A (en) | Data storage scheduling method and device | |
| CN105912698A (en) | Deletion method and system of data file in disk | |
| CN103198150A (en) | Big data indexing method and system | |
| CN108491163A (en) | A kind of big data processing method, device and storage medium | |
| CN108268639A (en) | A kind of index calculating method under big data environment | |
| CN104951464A (en) | Data storage method and system | |
| CN115964542A (en) | Space-time big data mining method based on multi-view algorithm | |
| CN106682061A (en) | Distributed system for collection and storage of origin data | |
| Karode et al. | Performance analysis of trustworthy online review system using blockchain | |
| CN104391947B (en) | Magnanimity GIS data real-time processing method and system | |
| CN105915595A (en) | Cluster storage system data accessing method and cluster storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 610041 Building No. 722, Building No. 1, Unit 12, Layer 121, Middle Section of Yizhou Avenue, Chengdu High-tech Zone, Sichuan Province Applicant after: CHENGDU HIFIVE TECHNOLOGY Co.,Ltd. Address before: 610000 Huayang Avenue Section 117 and 119, Huayang Street, Tianfu New District, Chengdu City, Sichuan Province Applicant before: CHENGDU HIFIVE CULTURE COMMUNICATION Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230627 Address after: Room 1210, 12 / F, unit 1, building 1, No. 722, middle section of Yizhou Avenue, high tech Zone, Chengdu, Sichuan 610000 Patentee after: Chengdu potential Artificial Intelligence Technology Co.,Ltd. Address before: 610041 Building No. 722, Building No. 1, Unit 12, Layer 121, Middle Section of Yizhou Avenue, Chengdu High-tech Zone, Sichuan Province Patentee before: CHENGDU HIFIVE TECHNOLOGY Co.,Ltd. |