CN105354251B - Electric power cloud data management indexing means based on Hadoop in electric system - Google Patents
Electric power cloud data management indexing means based on Hadoop in electric system Download PDFInfo
- Publication number
- CN105354251B CN105354251B CN201510680503.2A CN201510680503A CN105354251B CN 105354251 B CN105354251 B CN 105354251B CN 201510680503 A CN201510680503 A CN 201510680503A CN 105354251 B CN105354251 B CN 105354251B
- Authority
- CN
- China
- Prior art keywords
- data
- index
- information
- type
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013523 data management Methods 0.000 title claims abstract description 12
- 238000003860 storage Methods 0.000 claims description 38
- 238000013500 data storage Methods 0.000 claims description 25
- 238000007405 data analysis Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 12
- 238000009825 accumulation Methods 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 claims description 7
- 238000012423 maintenance Methods 0.000 claims description 5
- 238000000034 method Methods 0.000 claims description 5
- 125000006850 spacer group Chemical group 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012546 transfer Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000012544 monitoring process Methods 0.000 description 5
- 230000014509 gene expression Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 239000012634 fragment Substances 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000009428 plumbing Methods 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000009423 ventilation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Human Resources & Organizations (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Public Health (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The data received are divided into mass data type and information data type by the electric power cloud data management indexing means based on Hadoop in electric system provided by the invention, including b.；C. the corresponding index of each type is established according to the type of data；D. solicited message is generated according to the querying condition of user, is scanned for according to solicited message, and search result is fed back into user；The present invention effectively meets take place frequently update, quick multi-dimensional query requirement, and reduces influence of the index creation to system write performance to a certain extent, reduces retrieval threat caused by system stability.
Description
Technical field
The present invention relates to the electric power cloud data managements based on Hadoop in computer realm more particularly to a kind of electric system
Indexing means.
Background technology
The safety and stability of supply of electric power is the primary goal of power department.Constantly enter power industry in computer equipment
Today, with Internet of Things, cloud computing, the fast development of mobile Internet, data in the form of explosion in drastically expand, electricity
The stabilization and safety of Force system data storage, become the essential condition for ensureing power system stability operation.
In recent years, with the Large scale construction of the infrastructure such as intelligent substation, the data volume of power grid enterprises is with several
What stage speed madness increases, and data source has the spy of complexity and diversity (structuring, semi-structured and unstructured)
Point.The storage of various Heterogeneous Informations, such as：Communication power supply, image monitoring, security guard, main transformer fire-fighting, plumbing, fire
The on-line monitorings such as accessory production systems and on off state, equipment state such as alarm, heating and ventilation, gate inhibition, dynamic environment monitoring
Equipment is treated as urgent problem to be solved.
In existing field of power, storage data are broadly divided into two major classes type：Mass data and information data.Its
In, mass data mainly by compositions such as multi-medium data (original sound, video, picture), sensing data and system equipment data,
Big with amount of storage, the low feature of access frequency is mainly used for retrieval playback and backup storage；Information data is mainly grasped with business
Make based on data, access frequency high feature small with amount of storage, is mainly used for report and checks equal business operations.
Currently, existing file system is difficult to meet the actual storage demand of power industry.It is directly used in electric system
It can lead to many problems, such as：(1) disk space usage is low.Due to the concurrently burst of monitoring business, video file amount of storage
Greatly, the features such as small documents are more can cause disk fragments excessive, make from the point of view of safety monitoring business using traditional file system
Space availability ratio is not high；(2) document retrieval is inefficient.The business operations such as document retrieval lookup can be with file in storage system
Drastically increasing for quantity brings huge test (when increasing tens million of above) to system effectiveness.When the quantity of file is more than one
When fixed number amount, or even the problems such as system crash can be caused；(3) fusion of memory technology is inadequate.Existing technological means is mainly examined
What is considered is that existing hardware architecture adds software, with SAN frameworks+volume management software (or parallel file system) for representative.?
Also the shortcomings that volume management software or parallel file system are inherited while inheriting framework disadvantage, such as：On structure and maintenance
Flow is complicated, secondly because the various aspects reason such as data block and network demand, the system based on SAN also are difficult to dilatation, Wu Faman
Sufficient big data storage demand.
Invention content
In view of this, the present invention provides the electric power cloud data management indexing means based on Hadoop in a kind of electric system,
To solve the above problems.
Electric power cloud data management indexing means based on Hadoop in electric system provided by the invention, including：
B. the data received are divided into mass data type and information data type；
C. the corresponding index of each type is established according to the type of data；
D. solicited message is generated according to the querying condition of user, is scanned for according to solicited message, and search result is anti-
Feed user.
Further, further include before step b
A. the data processing architecture of electric system is divided into：
Data access layer, for data are judged according to the business retrieval request of user and classification processing,
Data analysis layer, for receiving the data in electric system and differentiating to the data type in electric system,
Data storage layer, establishment, maintenance and the initial data storage for data directory.
Further, the step b further includes
B1. data analysis layer is transferred data to, data are carried out classification processing and transmitted to accumulation layer by data analysis layer
Information and data type information to be stored.
Further, data storage layer parses the information to be stored and data type information that receive, and root
Data are established into corresponding index according to data type information.
Further, establishing index when data type is information data, in the step c includes
C11. judge whether solicited message is data storage request,
If storage request, then pass to virtual storage system by data analysis information and primary data information (pdi), virtually deposit
Storage system is indexed establishment in trusted servers and forms corresponding reverse indexing table and lexicon file, and is stored according to data
Structure is stored；
If not storage request, then transfer to the accumulation layer of virtual storage system to scan for solicited message, trusted service
Device according to the keyword and syntax tree of the submission of search come calculation document weight, and return user search relevant information data type
Information.
Further, establishing index when data type is mass data, in the step c includes
C21. index cluster and Hbase clusters are established respectively,
C22. when mass data reaches, system is transferred to index cluster and Hbase clusters simultaneously；
C23. index cluster establishes coarseness index, and sends information to HBase clusters；
After c24.Hbase clusters receive the information of the transmission of index cluster, a fine granularity index is established in each memory block,
Obtain demand information.
Further, the fine granularity index in the c24 is local index, and the process of establishing of the local index includes：
C241. n spacer block is divided time into,
C242. in a time interval, data category therein is dynamically divided into corresponding subdivision, it is each
In the data block storage to HBase of a subdivision,
C243. after the time interval currently divided, the data then generated will be repeated from next time interval
Step c241 is stored.
Further, the coarseness index includes time interval index and data category index.
Further, the data store organisation in the step c11 be Store ID, HEAD (Freq, Type, Keys,
TStamp), BODY }, wherein ID indicates that storage label, Head are made of content tab page, respectively：Freq- indicates frequency,
Type- indicates that type, Keys- indicate that keyword and TStamp- indicate timestamp；Body indicates storage information data.
Further, file weight in the step c11, is calculated by following formula
Wherein, Wk,dIndicate weights of the keyword key in storing data, fk,dIndicate keyword key in storing data
Frequency, M indicate this section storage data total size, mk,dIndicate keyword key sizes shared in storing data.
Beneficial effects of the present invention：The present invention classifies data, produced by meeting power industry in practical business
Magnanimity Heterogeneous Information the characteristics of, alleviate that distributed search is inefficient, the practical problem more than low space utilization disk fragments.
Data storage layer classifies to label different in the data of transmission, and respectively different types of data structure generates difference
Index, while respective handling is done to initial data.By generated index, initial data is deposited according to different with treated
Storage strategy is stored, effective to meet take place frequently update, quick multi-dimensional query requirement, and reduces rope to a certain extent
Draw the influence created to system write performance, reduces retrieval threat caused by system stability.
Description of the drawings
The invention will be further described with reference to the accompanying drawings and examples：
Fig. 1 is the system architecture schematic diagram of the present invention.
Fig. 2 is the system flow schematic diagram of the present invention.
Fig. 3 is the information data storing form schematic diagram of the present invention.
Fig. 4 is the mass data storage and retrieval flow schematic diagram of the present invention.
Specific implementation mode
The invention will be further described with reference to the accompanying drawings and examples：Fig. 1 is the system architecture schematic diagram of the present invention,
Fig. 2 is the system flow schematic diagram of the present invention, and Fig. 3 is the information data storing form schematic diagram of the present invention, and Fig. 4 is of the invention
Mass data storage and retrieval flow schematic diagram.
Electric power cloud data management indexing means based on Hadoop in electric system in the present embodiment, including
A. the data processing architecture of electric system is divided into：
Data access layer -- for data are judged according to the business retrieval request of user and classification processing,
Data analysis layer -- for receiving the data in electric system and differentiating to the data type in electric system,
Data storage layer -- establishment, maintenance and the initial data storage for data directory；
B. the data received are divided into mass data type and information data type；
C. the corresponding index of each type is established according to the type of data；
D. solicited message is generated according to the querying condition of user, is scanned for according to solicited message, and search result is anti-
Feed user.
As shown in Figure 1, in the present embodiment, data access layer is responsible for making type point to the business retrieval request that user submits
Class processing, such as report is checked, video is checked, computer room hardware state is checked；Data analysis layer include data interface module and
Data type authentication module is responsible for receiving and differentiating the data type in electric system；Data storage layer is responsible for data rope
Introduce the function that row establishment, maintenance and initial data are stored.Wherein, the storage of electric power system data and index mainly exist
It is completed in data storage layer.
As shown in Fig. 2, the step b further includes
B1. data analysis layer is transferred data to, data are carried out classification processing and transmitted to accumulation layer by data analysis layer
Information and data type information to be stored.The data of generation are passed through data processing by electric system in the virtualization storage of data
After the data-interface of layer is transferred to data type authentication module, the data received are carried out classification by data type authentication module recognizes
Card is handled and (wherein, Data is the original transmitted in data-interface to data storage layer transmission information Message { Data, Type }
Beginning data；Type is the classification of data type, is divided into mass data type and information data class by the characteristics of electric power system data
Type is marked with 0 and 1 respectively)；
Data storage layer parses the information to be stored and data type information that receive, and according to data class
Data are established corresponding index by type information, and data storage layer solves after receiving the message of data analysis layer transmission in the present embodiment
Analyse Message in Type, and according to the type of Type by data according to different storage strategies, establish different Type types pair
The index answered.
In the present embodiment, in data retrieval part, user to system sending service retrieval request Request Type,
Keys, Conditions } (wherein, Type is the mark that system is automatically generated according to the data type of the business retrieval request of user
Note, for distinguishing two different data types：Mass data type and information data type；Keys is mainly used for information data
The retrieval of type is realized for indicating the combination of user's search key by exclusive or；Conditions is user for identification
The condition of requested service data, such as report, time)；Data storage layer is in the system that receives according to customer service retrieval request
After the Request of generation, is retrieved by corresponding index strategy and return to the demand information of user.
In the present embodiment, establishing index when data type is information data, in the step c includes
C11. judge whether solicited message is data storage request,
If storage request, then pass to virtual storage system by data analysis information and primary data information (pdi), virtually deposit
Storage system is indexed establishment in trusted servers and forms corresponding reverse indexing table and lexicon file, and is stored according to data
Structure is stored；
If not storage request, then transfer to the accumulation layer of virtual storage system to scan for solicited message, trusted service
Device according to the keyword and syntax tree of the submission of search come calculation document weight, and return user search relevant information data type
Information.
In the present embodiment, structure reverse indexing mainly uses two Hash functions, is realized respectively to high and low frequency word
Mapping forms inverted file.By zipf law (Zi pf's Law), frequency that a word occurs in extensive text set and
Its ranking in phrase frequency meter is inversely proportional, i.e., the frequency that the highest word of frequency occurs is about deputy 2 times, and second
Position is 3/2 times of third position, it can thus be concluded that going out the probability of word frequency appearance：
Q indicates participle in the substantially ranking of appearance, probability of P (r) expressions in dictionary text.In word frequencies distribution
N ≈ 0.1, m ≈ 1.Therefore obtain formula 2：
In falling to arrange used in file by formula 2, i.e., the low frequency keyword for 80%, inverted index file accounts for 20%, by word
Preceding 20% in allusion quotation library is set as high frequency words, and rear 80% is set as low-frequency word, is mapped using hash functions.
Hash1 functions are mapped using MD5 functions, and using keyword as the input of function, the output of function preserves should
The inverted index filename of keyword.
For Hash2 using the frequency ranking of low-frequency word as the output of function, output also preserves the filename of the word inverted index.
The expression of Hash2 is as shown in formula 3：
Hash2=MD5 (h (p)) (3)
Wherein MD5 is MD5 functions, and p is ranking of the keyword in dictionary, and formula 4 is shown in the definition of h (p) formula：
N is threshold values in h (p), belongs to low-frequency word more than N in dictionary, what it is less than or equal to N is high frequency words.In Hbase
M values 1.
Hash2 by the inverted index of several keywords when being mapped to same file name, the smaller keyword of ranking,
It is smaller with mapping inverted file.
In the present embodiment, using based on the virtualization storage platform under Hadoop system, Hadoop be one by
The distributed system architecture of Apache funds club exploitation.Whether data analysis layer has data to electric power system data interface
Storage is needed to be judged that if desired store, then data storage layer terminal is after receiving the data according to storage demand to data
It is analyzed, and certification label information；If need not store, wait for, when data storage layer receives system transmission
When Request is asked, the querying condition information that user submits in access layer transfers to the accumulation layer of virtual storage system to be searched
Rope, trusted servers are according to the keyword and syntax tree of the submission of search come calculation document weight
Wherein, Wk,dIndicate weights of the keyword key in storing data；fk,dIndicate frequencies of the keyword key in storing data；M
Indicate the total size of this section storage data；mk,dIt indicates keyword key sizes shared in storing data, and returns to user and look into
Look for relevant information data type information, trusted servers press the variation of frequency, and system at regular intervals is by information update index file information.
As shown in figure 3, in the present embodiment, when data type is mass data, foundation index includes in the step c
C21. index cluster and Hbase clusters are established respectively,
C22. when mass data reaches, system is transferred to index cluster and Hbase clusters simultaneously；
C23. index cluster establishes coarseness index, and sends information to HBase clusters；
After c24.Hbase clusters receive the information of the transmission of index cluster, a fine granularity index is established in each memory block,
Obtain demand information.
In the present embodiment, Hbase (Hadoop Database) be a high reliability, high-performance, towards row, can stretch
The distributed memory system of contracting, index cluster are mainly responsible for the retrieval for insertion and user's Query Information to data；HBase clusters
It is mainly responsible for storage data and index each stores historical data information in the block, when mass data reaches, system is connecing
Mass data can be sent to index cluster and Hbase clusters after receiving mass data type simultaneously；Index cluster can establish this rugosity
Index, and to HBase clusters send information；Hbase clusters establish one after the request for receiving index cluster in each memory block
Local index；In the query processing request for receiving user, virtual storage system is obtained using index cluster according to querying condition
Associated memory block is taken, then corresponding demand information is obtained from corresponding memory block by HBase clusters.
In the present embodiment, the fine granularity index in the c24 is local index, and the local index establishes process packet
It includes：
C241. n spacer block is divided time into,
C242. in a time interval, data category therein is dynamically divided into corresponding subdivision, it is each
In the data block storage to HBase of a subdivision,
C243. after the time interval currently divided, the data then generated will be repeated from next time interval
Step c241 is stored.
Mass data storage process mainly indexes frame by foundation and realizes that the storage of data is handled, and there are three types of different classes of
Index, be respectively time interval index, data category index and local index.Wherein, time interval index and data category
Index belongs to the index of coarse grain level, is examined to current data for the time of the responsible storage according to data and classification
Rope；Local index is that fine granularity level index can be used for retrieval to historical data.N is divided time into using time interval
A spacer block, B+- tree indexes can be used for retrieving these time intervals；It, will be therein in a specific time interval
Data category is dynamically divided into corresponding subdivision.The data block of each subdivision is stored in the data block of HBase
In；After the time interval currently divided, when the data then generated will repeat above-mentioned division from next time interval
Between the method that is spaced stored；Original historical data will be no longer changed after being stored in HBase, for these history
Data can establish corresponding local rope by R-tree (R trees are another forms that B-tree develops to hyperspace) in batches
Draw.By this index strategy can when retrieving current data retrieval time interval and corresponding subspace, without
Retrieve current data itself；During data are inserted into, the renewal time of index can also effectively be lowered, to adapt to high frequency
The requirement of the data update of rate.
As shown in figure 3, in the present embodiment, the data store organisation in the step c11 is Store { ID, HEAD
(Freq, Type, Keys, TStamp), BODY }, wherein ID indicates that storage label, Head are made of content tab page, respectively
For：Freq- indicates that frequency, Type- indicate that type, Keys- indicate that keyword and TStamp- indicate timestamp；Body expressions are deposited
Store up information data.Wherein, frequency and timestamp are used for indicating the frequency of file being retrieved, and type and keyword are used to indicate that
The basic information content of stored data, convenient for retrieval.Specific data content of the parts Body for storing information data.
In the present embodiment, file weight in the step c11, is calculated by following formula
Wherein, Wk,dIndicate weights of the keyword key in storing data, fk,dIndicate keyword key in storing data
Frequency, M indicate this section storage data total size, mk,dIndicate keyword key sizes shared in storing data.
Finally illustrate, the above examples are only used to illustrate the technical scheme of the present invention and are not limiting, although with reference to compared with
Good embodiment describes the invention in detail, it will be understood by those of ordinary skill in the art that, it can be to the skill of the present invention
Art scheme is modified or replaced equivalently, and without departing from the objective and range of technical solution of the present invention, should all be covered at this
In the right of invention.
Claims (5)
1. the electric power cloud data management indexing means based on Hadoop in a kind of electric system, it is characterised in that：Including：
A. the data processing architecture of electric system is divided into：
Data access layer, for data are judged according to the business retrieval request of user and classification processing,
Data analysis layer, for receiving the data in electric system and differentiating to the data type in electric system,
Data storage layer, establishment, maintenance and the initial data storage for data directory；
B. the data received are divided into mass data type and information data type；And data analysis layer is transferred data to,
Data are carried out classification processing and transmit information and data type information to be stored to accumulation layer by data analysis layer；
C. the corresponding index of each type is established according to the type of data：Data storage layer is to the information to be stored that receives
It is parsed with data type information, and data is established by corresponding index according to data type information, specifically：
Establishing index when data type is information data, in the step c includes
C11. judge whether solicited message is data storage request,
If storage request, then pass to virtual storage system, virtual memory system by data analysis information and primary data information (pdi)
System is indexed establishment in trusted servers and forms corresponding reverse indexing table and lexicon file, and according to data store organisation
It is stored；
If not storage request, then transfer to the accumulation layer of virtual storage system to scan for solicited message, trusted servers root
Carry out calculation document weight according to the keyword and syntax tree of the submission of search, and returns to user and search relevant information data type letter
Breath；Establishing index when data type is mass data, in the step c includes
C21. index cluster and Hbase clusters are established respectively,
C22. when mass data reaches, system is transferred to index cluster and Hbase clusters simultaneously；
C23. index cluster establishes coarseness index, and sends information to HBase clusters；
After c24.Hbase clusters receive the information of the transmission of index cluster, a fine granularity index is established in each memory block, is obtained
Demand information；
D. business retrieval request information is generated according to the querying condition of user, is scanned for according to business retrieval request information, and
Search result is fed back into user.
2. the electric power cloud data management indexing means based on Hadoop, feature in electric system according to claim 1
It is：Fine granularity index in the c24 is local index, and the process of establishing of the local index includes：
C241. n spacer block is divided time into,
C242. in the time interval of a spacer block, data category therein is dynamically divided into corresponding subdivision,
In the data block storage to HBase of each subdivision,
C243. after the time interval currently divided, the data then generated will repeat step from next time interval
C241 is stored.
3. the electric power cloud data management indexing means based on Hadoop, feature in electric system according to claim 1
It is：The coarseness index includes time interval index and data category index.
4. the electric power cloud data management indexing means based on Hadoop, feature in electric system according to claim 1
It is：Data store organisation in the step c11 is Store { ID, HEAD (Freq, Type, Keys, TStamp), BODY },
Wherein ID indicates that storage label, HEAD are made of content tab page, respectively：Freq- indicates that frequency, Type- indicate type,
Keys- indicates that keyword and TStamp- indicate timestamp；BODY indicates storage information data.
5. the electric power cloud data management indexing means based on Hadoop, feature in electric system according to claim 1
It is：File weight in the step c11, is calculated by following formula
Wherein, Wk,dIndicate weights of the keyword key in storing data, fk,dIndicate frequencies of the keyword key in storing data
Rate, M indicate the total size of storage data, mk,dIndicate keyword key sizes shared in storing data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510680503.2A CN105354251B (en) | 2015-10-19 | 2015-10-19 | Electric power cloud data management indexing means based on Hadoop in electric system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510680503.2A CN105354251B (en) | 2015-10-19 | 2015-10-19 | Electric power cloud data management indexing means based on Hadoop in electric system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105354251A CN105354251A (en) | 2016-02-24 |
| CN105354251B true CN105354251B (en) | 2018-10-30 |
Family
ID=55330224
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510680503.2A Active CN105354251B (en) | 2015-10-19 | 2015-10-19 | Electric power cloud data management indexing means based on Hadoop in electric system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105354251B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106060125A (en) * | 2016-05-24 | 2016-10-26 | 南京国电南自美卓控制***有限公司 | Distributed real-time data transmission method based on data tags |
| CN106326464B (en) * | 2016-08-31 | 2019-09-10 | 成都科来软件有限公司 | A kind of network session packet indexing means based on retrieval information projection |
| CN107341198B (en) * | 2017-06-16 | 2020-05-12 | 云南电网有限责任公司信息中心 | Electric power mass data storage and query method based on theme instance |
| CN109299036B (en) * | 2017-07-25 | 2021-01-05 | 北京嘀嘀无限科技发展有限公司 | Label generation method, device, server and computer readable storage medium |
| CN107766452B (en) * | 2017-09-26 | 2021-07-06 | 广西电网有限责任公司电力科学研究院 | Indexing system suitable for high-speed access of power dispatching data and indexing method thereof |
| CN107832464B (en) * | 2017-11-28 | 2021-11-23 | 中国银行股份有限公司 | Data bleaching method and device |
| CN109614685A (en) * | 2018-12-04 | 2019-04-12 | 上海宝冶集团有限公司 | Construction based on BIM model is told somebody what one's real intentions are method, apparatus, equipment and mobile terminal |
| CN111597198A (en) * | 2020-04-01 | 2020-08-28 | 北京邮电大学 | Internet of things data query method for heterogeneous resource access and related equipment |
| CN112817958A (en) * | 2021-02-25 | 2021-05-18 | 广东电网有限责任公司 | Electric power planning data acquisition method and device and intelligent terminal |
| CN112860769B (en) * | 2021-03-10 | 2023-01-20 | 广东电网有限责任公司 | Energy planning data management system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101593191A (en) * | 2008-05-29 | 2009-12-02 | 国络多媒体有限公司 | The index establishing method of database, database search method and system |
| CN103366015B (en) * | 2013-07-31 | 2016-04-27 | 东南大学 | A kind of OLAP data based on Hadoop stores and querying method |
| IN2013MU03472A (en) * | 2013-10-31 | 2015-07-24 | Tata Consultancy Services Ltd | |
| CN104714983B (en) * | 2013-12-17 | 2019-02-19 | 中兴通讯股份有限公司 | The generation method and device of distributed index |
| CN104731945B (en) * | 2015-03-31 | 2018-04-06 | 浪潮集团有限公司 | A kind of text searching method and device based on HBase |
-
2015
- 2015-10-19 CN CN201510680503.2A patent/CN105354251B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN105354251A (en) | 2016-02-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105354251B (en) | Electric power cloud data management indexing means based on Hadoop in electric system | |
| CN112860695B (en) | Monitoring data query method, device, equipment, storage medium and program product | |
| US9460117B2 (en) | Image searching | |
| EP4099170B1 (en) | Method and apparatus of auditing log, electronic device, and medium | |
| US8543576B1 (en) | Classification of clustered documents based on similarity scores | |
| US20140279774A1 (en) | Classifying Resources Using a Deep Network | |
| US11977567B2 (en) | Method of retrieving query, electronic device and medium | |
| WO2022105119A1 (en) | Training corpus generation method for intention recognition model, and related device thereof | |
| CN107085568A (en) | A kind of text similarity method of discrimination and device | |
| WO2019041528A1 (en) | Method, electronic apparatus, and computer readable storage medium for determining polarity of news sentiment | |
| CN111586695A (en) | Short message identification method and related equipment | |
| US9424338B2 (en) | Clustering queries for image search | |
| WO2019075070A1 (en) | Automatic database analysis | |
| US20230153357A1 (en) | Method of processing an observation information, electronic device and storage medium | |
| US10601749B1 (en) | Trends in a messaging platform | |
| CN110764907A (en) | Cloud computing resource map construction method | |
| CN110852078A (en) | Method and device for generating title | |
| CN111143328A (en) | Agile business intelligent data construction method, system, equipment and storage medium | |
| US20230177269A1 (en) | Conversation topic extraction | |
| CN114461661B (en) | Data query method and system based on big data | |
| US10915561B2 (en) | Implementing unstructured content utilization from structured sources in system for answering questions | |
| CN116723002B (en) | Intelligent encryption method for power report based on situation awareness | |
| US11500908B1 (en) | Trends in a messaging platform | |
| WO2024103765A1 (en) | Sensitive data recognition model generation method and apparatus, and device and storage medium | |
| CN114650252B (en) | Routing method and device based on enterprise service bus and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |