CN107085595A - A kind of unstructured metadata association method and system of power industry - Google Patents

A kind of unstructured metadata association method and system of power industry Download PDF

Info

Publication number
CN107085595A
CN107085595A CN201710180451.1A CN201710180451A CN107085595A CN 107085595 A CN107085595 A CN 107085595A CN 201710180451 A CN201710180451 A CN 201710180451A CN 107085595 A CN107085595 A CN 107085595A
Authority
CN
China
Prior art keywords
document
data
unstructured
metadata
coding
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710180451.1A
Other languages
Chinese (zh)
Other versions
CN107085595B (en
Inventor
张彩友
江樱
王志强
陈建
戴波
王红凯
王剑
褚大可
梅峰
姚杨
姚一杨
边伟亮
卢文达
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Zhejiang Electric Power Co Ltd
Information and Telecommunication Branch of State Grid Zhejiang Electric Power Co Ltd
Original Assignee
State Grid Zhejiang Electric Power Co Ltd
Information and Telecommunication Branch of State Grid Zhejiang Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Zhejiang Electric Power Co Ltd, Information and Telecommunication Branch of State Grid Zhejiang Electric Power Co Ltd filed Critical State Grid Zhejiang Electric Power Co Ltd
Priority to CN201710180451.1A priority Critical patent/CN107085595B/en
Publication of CN107085595A publication Critical patent/CN107085595A/en
Application granted granted Critical
Publication of CN107085595B publication Critical patent/CN107085595B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2291User-Defined Types; Storage management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04SSYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Document Processing Apparatus (AREA)

Abstract

The present invention is intended to provide a kind of unstructured metadata association method and system of power industry, to solve that in prior art merging for unstructured data and structural data can not be realized.Comprise the following steps:Structural data table is set up at structural data center, for storage equipment information and corresponding document coding;Unstructured data management platform sets up unstructured data table, document information and corresponding document coding for storage equipment;When unstructured data management platform has document upload, the metadata of document is obtained, a unique document coding is distributed to the document of each upload, and document coding and corresponding metadata are stored in unstructured data metadata table;The document coding is added in structural data table simultaneously, to by document coding by the data correlation in unstructured data into structural data table, realize associating for unstructured data table and structural data table.

Description

A kind of unstructured metadata association method and system of power industry
Technical field
The present invention relates in power domain, specifically a kind of unstructured metadata association method and system of power industry.
Background technology
As socio-economic development enters " internet+" epoch, data are known as enterprise's most valuable assets by industry One of, how to run data well, mining data assets value turned into and accelerated business innovation when predecessor company, improve fine-grained management and One of important foundation work of science decision level.Business data type includes structural data, unstructured data, magnanimity number According to, GIS data etc., substantial amounts of enterprise information assets are analysed in depth, advantageous information is excavated, is intelligent decision service, beats Broken stream compression and shared barrier, realize that all types of data of enterprise are effectively integrated, and need to carry out the design of four class data storages, go forward side by side Row correlation model is designed.Wherein there is larger difference in unstructured data with structural data in terms of data type, especially In power industry, unstructured data existence range is wide, existence form is diversified, data resource lacks unified management, it is impossible to real Existing unstructured data is merged with the height of structural data, therefore this patent proposes a kind of unstructured first number of power industry According to correlating method and system.
The content of the invention
The present invention is intended to provide a kind of unstructured metadata association method and system of power industry, to solve prior art Merging for unstructured data and structural data can not be realized in scheme.
According to an aspect of the present invention, the invention provides a kind of unstructured metadata association method of power industry, bag Include following steps:
Structural data table is set up at structural data center, for storage equipment information and corresponding document coding;
Unstructured data management platform sets up unstructured data table, for the document information of storage equipment and corresponding Document coding;
When unstructured data management platform has document upload, the metadata of document is obtained, to the text of each upload Shelves all distribute a unique document coding, and document coding and corresponding metadata are stored in into unstructured data metadata In table;Simultaneously by the document coding be added in structural data table, to by document coding by unstructured data Data correlation is into structural data table.
Further, the metadata includes document path and document title.
Further, the document path is by setting the network site that IP modes are defined.
Further, the metadata also includes document data size, Doctype, document modification time, document data The circulation way of storage mode, the producer of document data and document data.
Further, when document is copied to multiple positions, multiple documents are stored in unstructured data metadata table Path.
According to another aspect of the present invention, the invention provides a kind of unstructured metadata association system of power industry, Including:
Structural data module, for storage equipment information and corresponding document coding;
Unstructured data module, document information and corresponding document coding for storage equipment;
Relating module, for when unstructured data management platform has document upload, storing the document uploaded and acquisition The metadata of document, a unique document coding is distributed to document of each upload, and by document coding and corresponding Metadata is stored in unstructured data meta data block;The document coding is added in structural data module, is used to By document coding by the data correlation in unstructured data into structural data module.
Further, the metadata includes document path and document title.
Further, the document path is by setting the network site that IP modes are defined.
Further, the metadata also includes document data size, Doctype, document modification time, document data The circulation way of storage mode, the producer of document data and document data.
Further, when document is copied to multiple positions, multiple documents are stored in unstructured data metadata table Path.
Following advantageous effects can be obtained by implementing the present invention:When unstructured data management platform has on document During biography, the metadata of document is obtained, unique document coding is distributed, and document coding and corresponding metadata are stored in non-knot In structure data metadata table, while the document coding is added in structural data table, non-knot is realized by document coding Structure tables of data is associated with structural data table.
Brief description of the drawings
Fig. 1 is a kind of flow chart of the unstructured metadata association method of power industry of the invention;
Fig. 2 is a kind of instance graph of the unstructured metadata association method of power industry of the invention;
Fig. 3 is a kind of connection figure of the unstructured metadata association system of power industry of the invention.
Embodiment
For the ease of the understanding of those skilled in the art, the present invention is made further with reference to specific embodiment It is bright:
Embodiment 1:
As shown in figure 1, a kind of unstructured metadata association method of power industry of the invention, comprises the following steps:
S1:Structural data table is set up at structural data center, for storage equipment information and corresponding document coding;
S2:Unstructured data management platform sets up unstructured data table, for the document information of storage equipment and right The document coding answered;
S3:When unstructured data management platform has document upload, store the document uploaded and obtain first number of document According to, distribute a unique document coding to document of each upload, and by document coding and the storage of corresponding metadata Into unstructured data metadata table;The document coding is added in structural data table simultaneously, to be compiled by document Code is by the data correlation in unstructured data into structural data table.
The present invention realizes information and information in unstructured data metadata table in structural data table by unique encodings Association, realize merging for unstructured data and structural data.
Unstructured data in the form of metadata (Metadata) respectively with entity documents with existing, and metadata includes version This number, each document can have multiple documentation releases, and each documentation release has independent man-to-man access right data and public affairs Common metadata;Metadata includes document path, when document is copied to multiple positions, is deposited in unstructured data metadata table Multiple document paths are stored up, when client accesses document, document is accessed nearby.The net that document path can define for setting IP modes Network position;Network site is defined by way of IP address+specific path.
Unstructured data management platform can be realized by the following method accesses document nearby, when client accesses document When, unstructured data management platform obtains the IP address in the corresponding all document paths of the document and the client, obtains institute Have the corresponding IP address in document path, by the corresponding IP address in document path obtain the document path all physical locations and Corresponding Network Provider (movement, UNICOM etc.), the physical location and network for obtaining client by the IP address of client is supplied Answer business;Calculate the distance between each physical location in document path and the physical location of client Di, DiRepresent client and text Physical distance between shelves path i, document path i priority P passes through P=Pi=Di÷ (S × M) is calculated, and S is distance parameter, 5KM may be selected, or is set according to actual conditions, M is Network Provider parameters, if the Network Provider of client and document road The corresponding Network Provider in footpath is identical, then M=2;If the Network Provider of client network provisionings corresponding with document path Business is different, then M=1, and descending sort is carried out to P, obtains the visit that P values highest document path accesses the document as the client Ask the way footpath, and then realize and access document nearby, improve access speed.When client accesses document path, client pair is obtained The feedback of access speed, the feedback can be fed back including fast, very fast, relatively slow, slow by manual feedback;If feedback is fast, The priority P in the document path is carried out Jia one to handle, i.e. Pi=Pi+ 1, if feedback is very fast, not to the excellent of the document path First level P is processed, if feedback is slower, carries out subtracting one processing, i.e. P to the priority P in the document pathi=Pi+ 1, if instead Feedback is very fast, then the priority P to the document path carries out zero setting processing, i.e. Pi=0, pass through the document path priority after processing During as client acquisition next time this document, the priority in the document path.
This method need not read other document paths, be downloaded and test the speed compared to each path, and selection download is tested the speed speed Spending most fast document path will facilitate as read path, because selection downloads the document path for testing the speed fastest as reading Path needs to test the speed of download in each document path, and when wherein some document path can not be communicated with client, The required time downloaded and tested the speed can be increased considerably.And the method for using the present invention, without being downloaded velocity test, significantly Improve access speed.
Metadata is also included:The owner of document content, version number, be built into the date, size of data, type, modification time, Storage location, data storage method, the producer of data, circulation way of data etc..
In order to which a kind of more preferable unstructured metadata association method of power industry is described further, one is lifted simply Example:As shown in Fig. 2 structural data table includes the information of the information, the information of breaker and wire of switch;Wherein switch Information include:Switch ID, handbook document id etc.;The information of breaker includes:Breaker ID, treaty documents ID etc.;Wire bag Wire ID, treaty documents ID etc. are included, wherein handbook document id is the unique encodings for switching handbook, and treaty documents ID is treaty documents Unique encodings;When service application sends the information and treaty documents of requesting query breaker to structural data center, knot The information of breaker and treaty documents ID are sent back to service application by Gou Hua data centers, service application based on contract document id to Unstructured data management platform sends request, and the related treaty documents information of inquiry breaker finds the FTP roads of treaty documents Footpath (i.e. document path), document is obtained according to FTP paths.Unstructured data management platform is led to according to the request of service application Cross treaty documents ID and obtain relevant documentation, and return to service application.
To realize the full-text search to document, searching for the full-text search to document content, document metadata etc. is supported in foundation Index is held up, to support the search of various files, file format can include word, excel, ppt, wps, pdf, html/xml, The Common File Formats such as tif, text, can also include the metadata for realizing Voice & Video file.
Unstructured data is designed according to IEC CIM standards, is stored with specification unstructured data.
Embodiment 2:
As shown in figure 3, a kind of unstructured metadata association system of power industry, including:
Structural data module 1, for storage equipment information and corresponding document coding;
Unstructured data module 2, document information and corresponding document coding for storage equipment;
Relating module 3, for when unstructured data management platform has document upload, storing the document uploaded and acquisition The metadata of document, a unique document coding is distributed to document of each upload, and by document coding and corresponding Metadata is stored in unstructured data meta data block;The document coding is added in structural data module, is used to By document coding by the data correlation in unstructured data into structural data module.
Unstructured data in the form of metadata (Metadata) respectively with entity documents with existing, and metadata includes version This number, each document can have multiple documentation releases, and each documentation release has independent man-to-man access right data and public affairs Common metadata;Metadata includes document path and document title, when document is copied to multiple positions, unstructured data member Multiple document paths are stored in tables of data, can be by judging which document the weighted information in document path judges from client most Closely, realize and access nearby.Document path is by that can set the network site that IP modes are defined.
Metadata is also included:The owner of document content, version number, be built into the date, size of data, type, modification time, Data storage method, the producer of data, circulation way of data etc..
The specific embodiment of the present invention is the foregoing is only, but the technical characteristic of the present invention is not limited thereto, Ren Heben The technical staff in field is in the field of the invention, and the change or modification made all are covered among the scope of the claims of the present invention.

Claims (10)

1. a kind of unstructured metadata association method of power industry, it is characterised in that comprise the following steps:
Structural data table is set up at structural data center, for storage equipment information and corresponding document coding;
Unstructured data management platform sets up unstructured data table, document information and corresponding document for storage equipment Coding;
When unstructured data management platform has document upload, the metadata of document is obtained, to the document of each upload A unique document coding is distributed, and document coding and corresponding metadata are stored in unstructured data metadata table In;The document coding is added in structural data table simultaneously, to by document coding by the number in unstructured data According to being associated with structural data table.
2. a kind of unstructured metadata association method of power industry as claimed in claim 1, it is characterised in that:First number According to including document path and document title.
3. a kind of unstructured metadata association method of power industry as claimed in claim 2, it is characterised in that:The document Path is by setting the network site that IP modes are defined.
4. a kind of unstructured metadata association method of power industry as claimed in claim 2, it is characterised in that:First number According to also include document data size, Doctype, document modification time, document datastore mode, document data the producer With the circulation way of document data.
5. a kind of unstructured metadata association method of power industry as claimed in claim 1, it is characterised in that:When document quilt When copying to multiple positions, multiple document paths are stored in unstructured data metadata table.
6. a kind of unstructured metadata association system of power industry, it is characterised in that including:
Structural data module, for storage equipment information and corresponding document coding;
Unstructured data module, document information and corresponding document coding for storage equipment;
Relating module, for when unstructured data management platform has document upload, storing the document uploaded and obtaining document Metadata, distribute a unique document coding to document of each upload, and by document coding and corresponding first number According to being stored in unstructured data meta data block;The document coding is added in structural data module, to pass through Document coding is by the data correlation in unstructured data into structural data module.
7. a kind of unstructured metadata association system of power industry as claimed in claim 6, it is characterised in that:First number According to including document path and document title.
8. a kind of unstructured metadata association system of power industry as claimed in claim 7, it is characterised in that:The document Path is by setting the network site that IP modes are defined.
9. a kind of unstructured metadata association system of power industry as claimed in claim 7, it is characterised in that:First number According to also include document data size, Doctype, document modification time, document datastore mode, document data the producer With the circulation way of document data.
10. a kind of unstructured metadata association system of power industry as claimed in claim 6, it is characterised in that:Work as document When being copied to multiple positions, multiple document paths are stored in unstructured data metadata table.
CN201710180451.1A 2017-03-23 2017-03-23 Unstructured metadata association method and system for power industry Active CN107085595B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710180451.1A CN107085595B (en) 2017-03-23 2017-03-23 Unstructured metadata association method and system for power industry

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710180451.1A CN107085595B (en) 2017-03-23 2017-03-23 Unstructured metadata association method and system for power industry

Publications (2)

Publication Number Publication Date
CN107085595A true CN107085595A (en) 2017-08-22
CN107085595B CN107085595B (en) 2023-07-14

Family

ID=59615226

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710180451.1A Active CN107085595B (en) 2017-03-23 2017-03-23 Unstructured metadata association method and system for power industry

Country Status (1)

Country Link
CN (1) CN107085595B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110765198A (en) * 2019-11-01 2020-02-07 云南电网有限责任公司信息中心 Data resource pool establishing method and device
CN110851449A (en) * 2019-11-11 2020-02-28 云南电网有限责任公司电力科学研究院 Unstructured document management method and device based on block chain technology

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050055355A1 (en) * 2003-09-05 2005-03-10 Oracle International Corporation Method and mechanism for efficient storage and query of XML documents based on paths
US6968338B1 (en) * 2002-08-29 2005-11-22 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Extensible database framework for management of unstructured and semi-structured documents
CN101010674A (en) * 2004-06-16 2007-08-01 甲骨文国际公司 Efficient extraction of XML content stored in a LOB
CN101178730A (en) * 2007-12-14 2008-05-14 清华大学 Document management method facing to integration business model
US20080114724A1 (en) * 2006-11-13 2008-05-15 Exegy Incorporated Method and System for High Performance Integration, Processing and Searching of Structured and Unstructured Data Using Coprocessors
CN101477568A (en) * 2009-02-12 2009-07-08 清华大学 Integrated retrieval method for structured data and non-structured data
CN101807211A (en) * 2010-04-30 2010-08-18 南开大学 XML-based retrieval method oriented to constraint on integrated paths of large amount of small-size XML documents
US20120117116A1 (en) * 2010-11-05 2012-05-10 Apple Inc. Extended Database Search
US20120117120A1 (en) * 2010-11-05 2012-05-10 Apple Inc. Integrated Repository of Structured and Unstructured Data
CN103262106A (en) * 2010-10-19 2013-08-21 惠普发展公司,有限责任合伙企业 Managing content from structured and unstructured data sources
CN103631909A (en) * 2013-11-26 2014-03-12 烽火通信科技股份有限公司 System and method for combined processing of large-scale structured and unstructured data
CN104050102A (en) * 2014-06-26 2014-09-17 北京思特奇信息技术股份有限公司 Object storing method and device in telecommunication system
CN104992124A (en) * 2015-08-03 2015-10-21 电子科技大学 Document safety access method for cloud storage environment
CN105653732A (en) * 2016-02-02 2016-06-08 优品财富管理有限公司 Automatic table establishment method and system based on metadata

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6968338B1 (en) * 2002-08-29 2005-11-22 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Extensible database framework for management of unstructured and semi-structured documents
US20050055355A1 (en) * 2003-09-05 2005-03-10 Oracle International Corporation Method and mechanism for efficient storage and query of XML documents based on paths
CN101010674A (en) * 2004-06-16 2007-08-01 甲骨文国际公司 Efficient extraction of XML content stored in a LOB
US20080114724A1 (en) * 2006-11-13 2008-05-15 Exegy Incorporated Method and System for High Performance Integration, Processing and Searching of Structured and Unstructured Data Using Coprocessors
CN101178730A (en) * 2007-12-14 2008-05-14 清华大学 Document management method facing to integration business model
CN101477568A (en) * 2009-02-12 2009-07-08 清华大学 Integrated retrieval method for structured data and non-structured data
CN101807211A (en) * 2010-04-30 2010-08-18 南开大学 XML-based retrieval method oriented to constraint on integrated paths of large amount of small-size XML documents
CN103262106A (en) * 2010-10-19 2013-08-21 惠普发展公司,有限责任合伙企业 Managing content from structured and unstructured data sources
US20120117116A1 (en) * 2010-11-05 2012-05-10 Apple Inc. Extended Database Search
US20120117120A1 (en) * 2010-11-05 2012-05-10 Apple Inc. Integrated Repository of Structured and Unstructured Data
CN103631909A (en) * 2013-11-26 2014-03-12 烽火通信科技股份有限公司 System and method for combined processing of large-scale structured and unstructured data
CN104050102A (en) * 2014-06-26 2014-09-17 北京思特奇信息技术股份有限公司 Object storing method and device in telecommunication system
CN104992124A (en) * 2015-08-03 2015-10-21 电子科技大学 Document safety access method for cloud storage environment
CN105653732A (en) * 2016-02-02 2016-06-08 优品财富管理有限公司 Automatic table establishment method and system based on metadata

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
邹盟军;杨莹;王建红;: "分布式非结构数据管理平台的应用研究", 电力信息与通信技术, no. 09 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110765198A (en) * 2019-11-01 2020-02-07 云南电网有限责任公司信息中心 Data resource pool establishing method and device
CN110851449A (en) * 2019-11-11 2020-02-28 云南电网有限责任公司电力科学研究院 Unstructured document management method and device based on block chain technology

Also Published As

Publication number Publication date
CN107085595B (en) 2023-07-14

Similar Documents

Publication Publication Date Title
US9576046B2 (en) Methods for integrating semantic search, query, and analysis across heterogeneous data types and devices thereof
Steege et al. A model of botanical collectors' behavior in the field: never the same species twice
Kiehle et al. Requirements for next generation spatial data infrastructures‐standardized web based geoprocessing and web service orchestration
CN102479252A (en) Query expression conversion apparatus and query expression conversion method
Wu Service Computing: Concept, Method and Technology
CN102567436A (en) Multi-Tenant system
Foerster et al. Establishing an OGC Web Processing Service for generalization processes
Hansen et al. Integrating the web and the world: contextual trails on the move
Marcer et al. Quality issues in georeferencing: From physical collections to digital data repositories for ecological research
CN109669400A (en) System and method for selecting Agent Computer
CN106021583A (en) Statistical method and system for page flow data
CN103995905A (en) Electronic commerce content multi-dimensional classification, navigation and skipping method
Doudali et al. Spaten: A spatio-temporal and textual big data generator
CN107085595A (en) A kind of unstructured metadata association method and system of power industry
Dong Exploration on web usage mining and its application
CN105893396A (en) Interpreting user queries based on nearby locations
CN109409959A (en) A kind of user information analysis method, device, equipment and medium
CN101719059A (en) Method for manipulating objects in a soa registry
Mohammed Free and Open Source GIS: an overview on the recent evolution of projects, standards and communities
CN106649462B (en) A kind of implementation method for mass data full-text search scene
Gordon et al. Designing agent based travel support system
CN103077218A (en) Method and equipment for determining demand information of query sequence in query request
Breen et al. Street trees as stormwater treatment measures
CN101887572A (en) Internet-based virtual experimental teaching resource management method
Shujun et al. Research on geographical information service based on SOA

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant