CN103106270B - cloud data fusion method and system - Google Patents
cloud data fusion method and system Download PDFInfo
- Publication number
- CN103106270B CN103106270B CN201310043555.XA CN201310043555A CN103106270B CN 103106270 B CN103106270 B CN 103106270B CN 201310043555 A CN201310043555 A CN 201310043555A CN 103106270 B CN103106270 B CN 103106270B
- Authority
- CN
- China
- Prior art keywords
- cloud data
- result
- destructuring
- data
- converted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a kind of cloud data fusion method and system.Described method includes: read destructuring cloud data；Described destructuring cloud data are carried out parallel processing output processing result；Undertaken described result being converted to structuring cloud data, and store.Described system includes: read module, is used for reading destructuring cloud data；Parallel processing module, for carrying out parallel processing output processing result to described destructuring cloud data；Modular converter, for being undertaken being converted to structuring cloud data by described result, and stores.Adopt the present invention can save the time spent by data query.
Description
Technical field
The present invention relates to data processing technique, particularly relate to cloud data fusion method and system.
Background technology
Along with the fast development that large data processes, the application of cloud computing is more and more general, and applies the various cloud data that cloud computing realizes and process and cloud data query also becomes the main task of cloud computing system.Cloud computing mostly adopts unstructured data storehouse, to be prone to divide and merge cloud data, and then meets the distributed variable-frequencypump demand in cloud computing.
But, what adopt due to cloud computing is non-structured database, therefore, is be absent from structurized when data being stored in, and in turn resulting in the cloud data query in cloud computing process needs to expend very many time,
Summary of the invention
Based on this, it is necessary to provide the cloud data fusion method of a kind of time can saved spent by data query.
Additionally, there is a need to provide the cloud data fusion system of a kind of time can saved spent by data query.
A kind of cloud data fusion method, comprises the steps:
Read destructuring cloud data；
Described destructuring cloud data are carried out parallel processing output processing result；
Undertaken described result being converted to structuring cloud data, and store.
Wherein in an embodiment, the step of described reading destructuring cloud data includes:
Unstructured data storehouse is carried out digital independent, obtains destructuring cloud data.
Wherein in an embodiment, described described result being carried out is converted to structuring cloud data, and also includes before the step stored:
Described result is stored in the step in described unstructured data storehouse.
Wherein in an embodiment, described described result being carried out is converted to structuring cloud data, and the step stored includes:
Obtain the data type that the row of the structured database that namely described result is stored into are corresponding；
The data type acquired described in described result is converted to；
By the described result write structure data base being converted to.
Wherein in an embodiment, described described result being carried out is converted to structuring cloud data, and also includes after the step stored:
Obtain the inquiry request of described result, in described structuring cloud data, carry out inquiry according to described inquiry request and obtain result.
A kind of cloud data fusion system, including:
Read module, is used for reading destructuring cloud data；
Parallel processing module, for carrying out parallel processing output processing result to described destructuring cloud data；
Modular converter, for being undertaken being converted to structuring cloud data by described result, and stores.
Wherein in an embodiment, described read module is additionally operable to destructuring cloud data base is carried out digital independent, obtains destructuring cloud data.
Wherein in an embodiment, described system also includes:
Unstructured data storehouse, is used for being stored in result.
Wherein in an embodiment, described modular converter includes:
Type acquiring unit, is used for obtaining the data type corresponding to row of the structured database that namely described result is stored into；
Type conversion unit, is used for the data type acquired described in described result being converted to；
Structured database, is used for the result being converted to described in writing.
Wherein in an embodiment, described system also includes:
Enquiry module, for obtaining the inquiry request of described result, carries out inquiry according to described inquiry request in described structuring cloud data and obtains result.
In above-mentioned cloud data fusion method and system, the destructuring cloud data of reading are carried out parallel processing, obtain result, this result is changed, result is made to be transformed to structuring cloud data by destructuring cloud data and store, owing to result is to store with the form of structuring cloud data, therefore, to make to process structure to can adapt to inquire about frequently, and save the time that inquiry is spent.
Accompanying drawing explanation
Fig. 1 is the flow chart of an embodiment medium cloud data fusion method；
Result is carried out being converted to structuring cloud data the method flow diagram stored by Fig. 2 in Fig. 1；
Fig. 3 is the structural representation of an embodiment medium cloud data fusion system；
Fig. 4 is the structural representation of modular converter in Fig. 3.
Detailed description of the invention
As it is shown in figure 1, in one embodiment, a kind of cloud data fusion method, comprise the steps:
Step S110, reads destructuring cloud data.
In the present embodiment, cloud data are the various data being applied in cloud computing, destructuring cloud data are the data being inconvenient to use two dimension logical table to realize, it may include the office documents of all forms, text, picture, XML, HTML, all kinds of form, image and audio/visual information etc..Prestore the various destructuring cloud data for realizing miscellaneous service logic, used in follow-up processing procedure.
In one embodiment, the detailed process of above-mentioned steps S110 is: unstructured data storehouse is carried out digital independent, obtains destructuring cloud data.
In the present embodiment, pre-set unstructured data storehouse, be used for writing various destructuring cloud data, carry out the storage of destructuring cloud data.Digital independent is carried out, to obtain required destructuring cloud data in unstructured data storehouse.
Destructuring cloud data are carried out parallel processing output processing result by step S130.
In the present embodiment, the destructuring cloud data read are carried out parallel processing, to realize the service logic currently triggered, obtain result and export.The parallel processing of destructuring cloud data is realized by cloud computing, owing to destructuring cloud data are prone to divide and merge, therefore, realizes distributed variable-frequencypump advantageously by cloud computing, to improve treatment effeciency.
Step S150, is undertaken result being converted to structuring cloud data, and stores.
In the present embodiment, result is changed, so that the result as destructuring cloud data is converted to structuring cloud data, and then realize automatically changing of data type, and store the result having become as structuring cloud data.
In one embodiment, also include before above-mentioned steps S150: result is stored in the step in unstructured data storehouse.
In the present embodiment, before the result as destructuring cloud data is changed, this result will be stored in destructuring cloud data, to realize the backup of result, and then ensure data safety.
As in figure 2 it is shown, in one embodiment, above-mentioned steps S150 comprises the steps:
Step S151, obtains the data type that the row of the structured database that namely result is stored into are corresponding.
In the present embodiment, the conversion of destructuring cloud data carries out by column, therefore, need to obtain the row that namely result is stored in structured database, and then obtain the data type corresponding to these row, for instance, data type can be integer type, date type etc..
Step S153, is converted to the data type acquired by result.
In the present embodiment, by column to destructuring cloud data, namely result is changed, and with the structuring cloud data that the data type obtained with acquire is consistent, and structuring cloud data is write in place's row corresponding to structured database.
In another embodiment, also include before above-mentioned steps S151: the result being stored in non-structural data base is judged by column, it is judged that whether the row being currently located exist data, if, then enter step S151, if it is not, then the row corresponding to structured database are set to sky.
In the present embodiment, by column destructuring cloud data are changed and stored, corresponding to ensure structured database right and wrong structured database, and then ensure that the orderly storage of data.
Step S155, the result write structure data base that will be converted to.
In the present embodiment, the row according to the structured database that namely result got is stored into, the result write structure data base that will be converted to.
In another embodiment, also include after above-mentioned steps S150: obtain the inquiry request of result, in structuring cloud data, carry out inquiry according to inquiry request and obtain result.
In the present embodiment, obtain the inquiry request processing structure, the structuring cloud data of storage are inquired about in structured database according to inquiry request, now, the stored data of structured database be carried out parallel processing after obtained result, owing to this result is structurized, therefore can quickly complete the inquiry of data, and then improve search efficiency.
Above-mentioned cloud data fusion method will be applied in cloud computing, and realize cloud data fusion by unstructured data storehouse and structured database, and then the distributed variable-frequencypump realizing in cloud computing under the learning from other's strong points to offset one's weaknesses of destructuring cloud data and structuring cloud data and quick search.
As it is shown on figure 3, in one embodiment, a kind of cloud data fusion system, including read module 110, parallel processing module 130 and modular converter 150.
Read module 110, is used for reading destructuring cloud data.
In the present embodiment, destructuring cloud data are the data being inconvenient to use two dimension logical table to realize, it may include the office documents of all forms, text, picture, XML, HTML, all kinds of form, image and audio/visual information etc..Prestore the various destructuring cloud data for realizing miscellaneous service logic, used in follow-up processing procedure.
In one embodiment, above-mentioned read module 110 is additionally operable to unstructured data storehouse is carried out digital independent, obtains destructuring cloud data.
In the present embodiment, pre-set unstructured data storehouse, be used for writing various destructuring cloud data, carry out the storage of destructuring cloud data.Read module 110 carries out digital independent in unstructured data storehouse, to obtain required destructuring cloud data.
Parallel processing module 130, for carrying out parallel processing output processing result to destructuring cloud data.
In the present embodiment, the destructuring cloud data read are carried out parallel processing by parallel processing module 130, to realize the service logic currently triggered, obtain result and export.The parallel processing of destructuring cloud data is realized by cloud computing, owing to destructuring cloud data are prone to divide and merge, therefore, realizes distributed variable-frequencypump advantageously by cloud computing, to improve treatment effeciency.
Modular converter 150, for being undertaken being converted to structuring cloud data by result, and stores.
In the present embodiment, result is changed by modular converter 150, so that the result as destructuring cloud data is converted to structuring cloud data, and then realizes automatically changing of data type, and stores the result having become as structuring cloud data.
In one embodiment, above-mentioned cloud data fusion system also includes unstructured data storehouse, and this unstructured data storehouse is used for being stored in result.
In the present embodiment, before the result as destructuring cloud data is changed, this result will be stored in destructuring cloud data, to realize the backup of result, and then ensure data safety.
As shown in Figure 4, in one embodiment, above-mentioned modular converter 150 includes type acquiring unit 151, type conversion unit 153 and structured database 155.
Type acquiring unit 151, is used for obtaining the data type corresponding to row of the structured database that namely result is stored into.
In the present embodiment, the conversion of destructuring cloud data carries out by column, therefore, type acquiring unit 151 is needed to obtain the row that namely result is stored in structured database, and then obtain the data type corresponding to these row, for instance, data type can be integer type, date type etc..
Type conversion unit 153, for being converted to the data type acquired by result.
In the present embodiment, type conversion unit 153 is by column to destructuring cloud data, and namely result is changed, and with the structuring cloud data that the data type obtained with acquire is consistent, and structuring cloud data is write in place's row corresponding to structured database.
In another embodiment, above-mentioned modular converter 150 also includes judging unit, this judging unit is for judging by column the result being stored in non-structural data base, judge whether the row being currently located exist data, if, then notification type acquiring unit 151, if it is not, be then set to sky by the row corresponding to structured database.
In the present embodiment, by column destructuring cloud data are changed and stored, corresponding to ensure structured database right and wrong structured database, and then ensure that the orderly storage of data.
Structured database 155, for writing the result being converted to.
In the present embodiment, the row according to the structured database that namely result got is stored into, the result write structure data base that will be converted to.
In another embodiment, above-mentioned cloud data fusion system also includes enquiry module, and this enquiry module, for obtaining the inquiry request of result, carries out inquiry according to inquiry request in structuring cloud data and obtains result.
In the present embodiment, enquiry module obtains the inquiry request processing structure, the structuring cloud data of storage are inquired about in structured database according to inquiry request, now, the stored data of structured database be carried out parallel processing after obtained result, owing to this result is structurized, therefore can quickly complete the inquiry of data, and then improve search efficiency.
Above-mentioned cloud data fusion system will be applied in cloud computing, and realize cloud data fusion by destructuring cloud data base and structured database, and then the distributed variable-frequencypump realizing in cloud computing under the learning from other's strong points to offset one's weaknesses of destructuring cloud data and structuring cloud data and quick search.
In above-mentioned cloud data fusion method and system, the destructuring cloud data of reading are carried out parallel processing, obtain result, this result is changed, result is made to be transformed to structuring cloud data by destructuring cloud data and store, owing to result is to store with the form of structuring cloud data, therefore, to make to process structure to can adapt to inquire about frequently, and save the time that inquiry is spent.
One of ordinary skill in the art will appreciate that all or part of flow process realizing in above-described embodiment method, can be by the hardware that computer program carrys out instruction relevant to complete, described program can be stored in a computer read/write memory medium, this program is upon execution, it may include such as the flow process of the embodiment of above-mentioned each side method.Wherein, described storage medium can be magnetic disc, CD, read-only store-memory body (Read-OnlyMemory, ROM) or random store-memory body (RandomAccessMemory, RAM) etc..
Embodiment described above only have expressed the several embodiments of the present invention, and it describes comparatively concrete and detailed, but therefore can not be interpreted as the restriction to the scope of the claims of the present invention.It should be pointed out that, for the person of ordinary skill of the art, without departing from the inventive concept of the premise, it is also possible to making some deformation and improvement, these broadly fall into protection scope of the present invention.Therefore, the protection domain of patent of the present invention should be as the criterion with claims.
Claims (8)
1. a cloud data fusion method, comprises the steps:
Read destructuring cloud data；
Described destructuring cloud data are carried out parallel processing output processing result；
Undertaken described result being converted to structuring cloud data, and store；
Described described result being carried out is converted to structuring cloud data, and the step stored includes:
Obtain the data type that the row of the structured database that namely described result is stored into are corresponding；
The data type acquired described in described result is converted to；
By the described result write structure data base being converted to.
2. cloud data fusion method according to claim 1, it is characterised in that the step of described reading destructuring cloud data includes:
Unstructured data storehouse is carried out digital independent, obtains destructuring cloud data.
3. cloud data fusion method according to claim 2, it is characterised in that described described result being carried out is converted to structuring cloud data, and also includes before the step stored:
Described result is stored in the step in described unstructured data storehouse.
4. cloud data fusion method according to claim 1, it is characterised in that described described result being carried out is converted to structuring cloud data, and also includes after the step stored:
Obtain the inquiry request of described result, in described structuring cloud data, carry out inquiry according to described inquiry request and obtain result.
5. a cloud data fusion system, it is characterised in that including:
Read module, is used for reading destructuring cloud data；
Parallel processing module, for carrying out parallel processing output processing result to described destructuring cloud data；
Modular converter, for being undertaken being converted to structuring cloud data by described result, and stores；
Described modular converter includes:
Type acquiring unit, is used for obtaining the data type corresponding to row of the structured database that namely described result is stored into；
Type conversion unit, is used for the data type acquired described in described result being converted to；
Structured database, is used for the result being converted to described in writing.
6. cloud data fusion system according to claim 5, it is characterised in that described read module is additionally operable to unstructured data storehouse is carried out digital independent, obtains destructuring cloud data.
7. cloud data fusion system according to claim 6, it is characterised in that described system also includes:
Unstructured data storehouse, for being stored in the result of described parallel processing module output.
8. cloud data fusion system according to claim 5, it is characterised in that described system also includes:
Enquiry module, for obtaining the inquiry request of described result, carries out inquiry according to described inquiry request in described structuring cloud data and obtains result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310043555.XA CN103106270B (en) | 2013-02-02 | 2013-02-02 | cloud data fusion method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310043555.XA CN103106270B (en) | 2013-02-02 | 2013-02-02 | cloud data fusion method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103106270A CN103106270A (en) | 2013-05-15 |
| CN103106270B true CN103106270B (en) | 2016-06-29 |
Family
ID=48314125
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310043555.XA Active CN103106270B (en) | 2013-02-02 | 2013-02-02 | cloud data fusion method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103106270B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104778241B (en) * | 2015-04-08 | 2019-06-04 | 北京京东尚科信息技术有限公司 | A kind of report form generation method and system |
| CN109905423B (en) * | 2017-12-08 | 2022-11-08 | 上海仪电（集团）有限公司中央研究院 | Intelligent management system |
| CN109669965A (en) * | 2018-11-13 | 2019-04-23 | 广州欧赛斯信息科技有限公司 | A kind of acquisition analysis system that supporting unstructured data and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1858735A (en) * | 2005-12-30 | 2006-11-08 | 华为技术有限公司 | Method for processing mass data |
| CN102222090A (en) * | 2011-06-02 | 2011-10-19 | 清华大学 | Mass data resource management frame under cloud environment |
| CN102375860A (en) * | 2010-08-27 | 2012-03-14 | 盛乐信息技术（上海）有限公司 | Cloud-based structured data search method |
| CN102508898A (en) * | 2011-11-04 | 2012-06-20 | 浪潮(北京)电子信息产业有限公司 | Data access method and database system based on cloud computing |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9009664B2 (en) * | 2011-03-31 | 2015-04-14 | Infosys Limited | Structural search of source code |
-
2013
- 2013-02-02 CN CN201310043555.XA patent/CN103106270B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1858735A (en) * | 2005-12-30 | 2006-11-08 | 华为技术有限公司 | Method for processing mass data |
| CN102375860A (en) * | 2010-08-27 | 2012-03-14 | 盛乐信息技术（上海）有限公司 | Cloud-based structured data search method |
| CN102222090A (en) * | 2011-06-02 | 2011-10-19 | 清华大学 | Mass data resource management frame under cloud environment |
| CN102508898A (en) * | 2011-11-04 | 2012-06-20 | 浪潮(北京)电子信息产业有限公司 | Data access method and database system based on cloud computing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103106270A (en) | 2013-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7021228B2 (en) | Blockchain-based data storage and query methods and devices | |
| CN102129425B (en) | The access method of big object set table and device in data warehouse | |
| CN104216961A (en) | Method and device for data processing | |
| CN107622080B (en) | Data processing method and equipment | |
| CN103150359B (en) | Micro-blog information display packing and device | |
| US11200201B2 (en) | Metadata storage method, device and server | |
| CN103106270B (en) | cloud data fusion method and system | |
| CN113112208A (en) | Electric power material storage management method based on three-dimensional visualization model | |
| CN109710224B (en) | Page processing method, device, equipment and storage medium | |
| CN110297955B (en) | Information query method, device, equipment and medium | |
| CN104268179A (en) | Unstructured three-dimensional model data analytical algorithm | |
| CN105354236A (en) | Reconciliation information generation method and system | |
| US20140149854A1 (en) | Server and method for generating object document | |
| CN114359533B (en) | Page number identification method based on page text and computer equipment | |
| CN108241620B (en) | Query script generation method and device | |
| CN103678263A (en) | Graphical interface display method and system for incidence relations among document chapters | |
| US20160342284A1 (en) | Electronic device and note reminder method | |
| CN105426676B (en) | A kind of well data processing method and system | |
| CN106843819B (en) | Object serialization method and device | |
| IN2013CH05996A (en) | ||
| CN105468570A (en) | Document composition display method and device | |
| US10789214B2 (en) | Providing access to resources | |
| CN111125087A (en) | Data storage method and device | |
| CN101826008A (en) | Scene subject changing method and computer system | |
| CN110019631B (en) | Method and device for processing multi-dimensional map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230105 Address after: Room 301, No. 235, Kexue Avenue, Huangpu District, Guangzhou, Guangdong 510000 Patentee after: OURCHEM INFORMATION CONSULTING CO.,LTD. Address before: 1068 No. 518055 Guangdong city in Shenzhen Province, Nanshan District City Xili University School Avenue Patentee before: SHENZHEN INSTITUTES OF ADVANCED TECHNOLOGY Effective date of registration: 20230105 Address after: 510000 room 606-609, compound office complex building, No. 757, Dongfeng East Road, Yuexiu District, Guangzhou City, Guangdong Province (not for plant use) Patentee after: China Southern Power Grid Internet Service Co.,Ltd. Address before: Room 301, No. 235, Kexue Avenue, Huangpu District, Guangzhou, Guangdong 510000 Patentee before: OURCHEM INFORMATION CONSULTING CO.,LTD. |