CN109284337A - A kind of system based on big data analysis service - Google Patents
A kind of system based on big data analysis service Download PDFInfo
- Publication number
- CN109284337A CN109284337A CN201811156380.2A CN201811156380A CN109284337A CN 109284337 A CN109284337 A CN 109284337A CN 201811156380 A CN201811156380 A CN 201811156380A CN 109284337 A CN109284337 A CN 109284337A
- Authority
- CN
- China
- Prior art keywords
- data
- big data
- system based
- cloud computing
- analysis service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a kind of systems based on big data analysis service, including MPP database, data-mining module, cloud computing platform, wherein: MPP database realizes data acquisition, storage for collecting, integrating, storing received data, according to different data sources；Data-mining module is used to carry out deep layer excavation to the data of the MPP database, does deep layer excavation, the information implied in decomposition data to the information in data；Cloud computing platform is for the collected data stored of MPP database described in calculation processing.In the actual operation process, collecting data, integration and the movement of storage can be completed at the same time, and the movement of data mining carries out after the completion of storage acts.
Description
Technical field
The present invention relates to big datas, and in particular to a kind of system based on big data analysis service.
Background technique
Big data analysis, which refers to, analyzes huge data.Big data may be summarized to be 4 V, and data volume is big
(Volume), speed fast (Velocity), type more (Variety), value (Value).
Vocabulary of the big data as IT industry most burning hot at present, the following data warehouse, data safety, data point
Analysis, data mining etc. are increasingly becoming the profit focus that industry personage falls over each other to pursue around the utilization of the commercial value of big data.
With the arriving of big data era, big data analysis is also come into being.
Enterprise institution is faced with increasing competitive pressure, they not only need to obtain client, it will further recognize that client
Demand, to promote customer experience, the long relationship of concurrent length.Client reduces the privacy grade that data use by sharing data
Not, it is expected that enterprise can understand them, corresponding interaction is formed, and provide seamless experience in all contact points.
Summary of the invention
The present invention preferably realizes the management of big data analysis and its quality of data in order to facilitate enterprise institution, provides one kind
Based on the system of big data analysis service, data classification and deep layer parsing can be realized from the initial stage for collecting data.
The present invention is achieved through the following technical solutions:
A kind of system based on big data analysis service, including MPP database, data-mining module, cloud
Computing platform, in which: MPP database is for collecting, integrating, storing received data, according to different
Data source realizes data acquisition, storage；Data-mining module be used for the data of the MPP database into
Row deep layer is excavated, and does deep layer excavation, the information implied in decomposition data to the information in data；Cloud computing platform is at calculating
Manage the collected data stored of the MPP database.
It further, further include distributed data base, the distributed data base is for collecting at the large-scale parallel
Manage the dispersion data that database is omitted or lost.
Further, the MPP database is large server or memory.
Further, the MPP database uses Teradata AsterData, EMC GreenPlum
As data warehouse.The above Liang Suo company possesses relatively stable and huge data and supports.
Further, the cloud computing platform uses existing cloud computing platform, and operation is mature and stable, guarantees to calculate power, subtract
It is few oneself to build required manpower and material resources.
Preferably, the cloud computing platform calculates service using the big data of existing cloud computing service platform
MaxCompute, the MaxCompute are capable of providing TB/PB grades of data warehouse solutions.
Further, by the cloud computing platform calculate described in allocated resources to by the resource management mode of YARN
MPP database.
Compared with prior art, the present invention having the following advantages and benefits:
1, a kind of system based on big data analysis service of the present invention can be convenient enterprise institution and preferably realize big data
Analysis and its management of the quality of data；
2, a kind of system based on big data analysis service of the present invention realize it is online calculate power distribution, according to different data come
Realize data acquisition, storage in source；
3, a kind of system based on big data analysis service of the present invention has the characteristics that high efficiency, high security.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, below with reference to embodiment, the present invention is made
Further to be described in detail, exemplary embodiment of the invention and its explanation for explaining only the invention, are not intended as to this
The restriction of invention.
Embodiment
A kind of system based on big data analysis service, including MPP database, data-mining module, cloud
Computing platform, in which: MPP database is for collecting, integrating, storing received data, according to different
Data source realizes data acquisition, storage；Data-mining module be used for the data of the MPP database into
Row deep layer is excavated, and does deep layer excavation, the information implied in decomposition data to the information in data；Cloud computing platform is at calculating
Manage the collected data stored of the MPP database.In the actual operation process, collect data, integration and
The movement of storage can be completed at the same time, and the movement of data mining carries out after the completion of storage acts.
It further include distributed data base, the distributed data base is lost for collecting the MPP database
Leakage or the dispersion data lost.
The MPP database is large server or memory.
The MPP database uses Teradata AsterData, and EMC GreenPlum is as data
Warehouse；The above Liang Suo company possesses relatively stable and huge data and supports.
The cloud computing platform uses existing cloud computing platform, and operation is mature and stable, guarantees to calculate power, reduces oneself and build
Required manpower and material resources.
The cloud computing platform calculates service MaxCompute using the big data of existing cloud computing service platform, described
MaxCompute is capable of providing TB/PB grades of data warehouse solutions.
By the cloud computing platform calculate described in by the resource management mode of YARN allocate resources to it is extensive simultaneously
Row processing database.
Above-described specific embodiment has carried out further the purpose of the present invention, technical scheme and beneficial effects
It is described in detail, it should be understood that being not intended to limit the present invention the foregoing is merely a specific embodiment of the invention
Protection scope, all within the spirits and principles of the present invention, any modification, equivalent substitution, improvement and etc. done should all include
Within protection scope of the present invention.
Claims (7)
1. a kind of system based on big data analysis service, which is characterized in that dug including MPP database, data
Dig module, cloud computing platform, in which:
MPP database is for collecting, integrating, storing received data；
Data-mining module is used to carry out deep layer excavation to the data of the MPP database；
Cloud computing platform is for the collected data stored of MPP database described in calculation processing.
2. a kind of system based on big data analysis service according to claim 1, it is characterised in that: further include distribution
Database, the distributed data base are used to collect the dispersion data that the MPP database is omitted or lost.
3. a kind of system based on big data analysis service according to claim 1, it is characterised in that: it is described it is extensive simultaneously
Row processing database is large server or memory.
4. a kind of system based on big data analysis service according to claim 1, it is characterised in that: it is described it is extensive simultaneously
Row processing database uses Teradata AsterData, and EMC GreenPlum is as data warehouse.
5. a kind of system based on big data analysis service according to claim 1, it is characterised in that: the cloud computing is flat
Platform uses existing cloud computing service platform.
6. a kind of system based on big data analysis service according to claim 5, it is characterised in that: the cloud computing is flat
Platform calculates service MaxCompute using the big data of existing cloud computing service platform.
7. a kind of system based on big data analysis service according to claim 5, it is characterised in that: pass through the cloud meter
It calculates the resource management mode by YARN that platform calculates and allocates resources to MPP database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811156380.2A CN109284337A (en) | 2018-09-29 | 2018-09-29 | A kind of system based on big data analysis service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811156380.2A CN109284337A (en) | 2018-09-29 | 2018-09-29 | A kind of system based on big data analysis service |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109284337A true CN109284337A (en) | 2019-01-29 |
Family
ID=65182545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811156380.2A Pending CN109284337A (en) | 2018-09-29 | 2018-09-29 | A kind of system based on big data analysis service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109284337A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103678726A (en) * | 2012-09-05 | 2014-03-26 | 亿阳信通股份有限公司 | Data complementary collection method and system |
| CN106709069A (en) * | 2017-01-25 | 2017-05-24 | 焦点科技股份有限公司 | High-reliability big data logging collection and transmission method |
| CN107423417A (en) * | 2017-07-29 | 2017-12-01 | 合肥千奴信息科技有限公司 | Big data plateform system and its operation method |
| WO2018044058A2 (en) * | 2016-08-31 | 2018-03-08 | 이지세이버 주식회사 | Charging apparatus for electric vehicle and charging system including same |
| CN108335032A (en) * | 2018-01-31 | 2018-07-27 | 合肥和钧正策信息技术有限公司 | A kind of business policy service planning system based on big data |
| CN108427764A (en) * | 2018-03-22 | 2018-08-21 | 安徽八六物联科技有限公司 | The anti-lost system of collection for big data |
-
2018
- 2018-09-29 CN CN201811156380.2A patent/CN109284337A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103678726A (en) * | 2012-09-05 | 2014-03-26 | 亿阳信通股份有限公司 | Data complementary collection method and system |
| WO2018044058A2 (en) * | 2016-08-31 | 2018-03-08 | 이지세이버 주식회사 | Charging apparatus for electric vehicle and charging system including same |
| CN106709069A (en) * | 2017-01-25 | 2017-05-24 | 焦点科技股份有限公司 | High-reliability big data logging collection and transmission method |
| CN107423417A (en) * | 2017-07-29 | 2017-12-01 | 合肥千奴信息科技有限公司 | Big data plateform system and its operation method |
| CN108335032A (en) * | 2018-01-31 | 2018-07-27 | 合肥和钧正策信息技术有限公司 | A kind of business policy service planning system based on big data |
| CN108427764A (en) * | 2018-03-22 | 2018-08-21 | 安徽八六物联科技有限公司 | The anti-lost system of collection for big data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103365929B (en) | The management method of a kind of data base connection and system | |
| Kapil et al. | A study of big data characteristics | |
| CN109582667A (en) | A kind of multiple database mixing storage method and system based on power regulation big data | |
| CN105956015A (en) | Service platform integration method based on big data | |
| CN109544316B (en) | Method and system for urging collection of real-time case division according to proportion | |
| CN104809244B (en) | Data digging method and device under a kind of big data environment | |
| CN104820708B (en) | A kind of big data clustering method and device based on cloud computing platform | |
| CN105631026A (en) | Security data analysis system | |
| US9183598B2 (en) | Identifying event-specific social discussion threads | |
| CN104036139A (en) | Moving object trajectory monitoring method | |
| CN104809246B (en) | The processing method and processing device of charge data | |
| CN103279505B (en) | A kind of based on semantic mass data processing method | |
| CN101902497A (en) | Cloud computing based internet information monitoring system and method | |
| CN103986783A (en) | Cloud computing system | |
| CN104123852A (en) | Vehicle searching method based on cloud computing for parking lot | |
| CN104899278A (en) | Method and apparatus for generating data operation logs of Hbase database | |
| CN106599189A (en) | Dynamic Skyline inquiry device based on cloud computing | |
| Mandal et al. | Architecture of efficient word processing using Hadoop MapReduce for big data applications | |
| CN109284337A (en) | A kind of system based on big data analysis service | |
| CN110826845B (en) | Multidimensional combination cost allocation device and method | |
| CN108288126A (en) | A kind of business policy planning system based on big data | |
| CN204731786U (en) | Adopt the large data analysis system of computing machine verification code technology | |
| MX2018007857A (en) | Methods, systems, and computer readable media for an electronic infrastructure for a rotating savings and credit association. | |
| Bajaj et al. | A comparative analysis of cloud based services platform | |
| Amandeep Singh et al. | Research Challenges on Big Internet of Things Data Analytics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190129 |