CN109739878A - Big data querying method, device, server and storage medium - Google Patents
Big data querying method, device, server and storage medium Download PDFInfo
- Publication number
- CN109739878A CN109739878A CN201811526906.1A CN201811526906A CN109739878A CN 109739878 A CN109739878 A CN 109739878A CN 201811526906 A CN201811526906 A CN 201811526906A CN 109739878 A CN109739878 A CN 109739878A
- Authority
- CN
- China
- Prior art keywords
- data
- computations
- business
- tree
- calculating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure is directed to a kind of big data querying method, device, server and storage mediums, this method comprises: receiving the data inquiry request of user terminal natural language mode；According to the data inquiry request, the data computations tree including data computations set is generated；According to the data computations tree, the data computations in the data computations tree are distributed into corresponding calculating task node and are executed；The implementing result of the calculating task node is obtained, and according to the data computations tree, the implementing result is carried out to summarize calculating, obtains data query result；The data query result is returned into the user terminal.The disclosure can carry out effective query analysis to the data for being stored in different location, it realizes and data interpretation and integration is carried out with the angle of global business, it effectively can be analyzed to enterprise's domestic demand otherwise with the scene of division data, realize the data cross across business scope in business data statistics and be associated with.
Description
Technical field
This disclosure relates to data query technique field more particularly to a kind of big data querying method, device, server and deposit
Storage media.
Background technique
Into big data era, the achievement that enterprise needs that big data analysis is made full use of to excavate realizes the visitor of data-driven
Family analysis, market sale, products innovation and managed operation etc..
Currently based on the commercial statistics analytical statement system of traditional BI (Business Intelligence, business intelligence),
The acquisition related data that business department can be allowed sufficient in time is reported, to hold the information in project, user or market etc..But tradition
Business data statistics has apparent limitation and deficiency in the data cross association across business scope, is often confined to logarithm
According to the data query operation in library, the operation and utilization of data are confined in local data, cannot be integrated with the angle of global business
And unscrambling data.In addition, business personnel is limited to the factors such as business isolation and the isolation of system of data, it is difficult in existing report system
From business demand on system, freely operating and interpreting for global data is carried out.
Summary of the invention
To overcome the problems in correlation technique, the disclosure provide a kind of big data querying method, device, server and
Storage medium.
According to the first aspect of the embodiments of the present disclosure, a kind of big data querying method is provided, comprising:
Receive the data inquiry request of user terminal natural language mode；
According to the data inquiry request, the data computations tree including data computations set is generated；
According to the data computations tree, the data computations in the data computations tree are distributed into correspondence
Calculating task node executed；
The implementing result of the calculating task node is obtained, and according to the data computations tree, executes knot to described
Fruit carries out summarizing calculating, obtains data query result；
The data query result is returned into the user terminal.
Optionally, described according to the data inquiry request, it generates the data calculating including data computations set and refers to
Enable tree, comprising:
The data inquiry request is converted into structuralized query data representation sentence；
According to the structuralized query data representation sentence, the data computations including data computations set are generated
Tree, the data computations tree include data dependence relation and calculating process dependence.
It is optionally, described that the data inquiry request is converted into structuralized query data representation sentence, comprising:
Text participle and business semantics mark are carried out to the data inquiry request, segmented and semantic annotation result；
Contextual analysis is carried out to the data inquiry request, is patrolled with carrying out business with semantic annotation result to the participle
Polishing is collected, service logic polishing result is obtained；
According to the participle with semantic annotation result and the service logic polishing as a result, generating structuralized query tables of data
Up to sentence.
Optionally, described according to the structuralized query data representation sentence, generating includes data computations set
Data computations tree, comprising:
According to the structuralized query data representation sentence and business data knowledge mapping, data place to be checked is determined
Calculating task node, and determine that data dependence relation and calculating process dependence, the business data knowledge mapping include
The metadata information of business data；
Calculating task node, data dependence relation and calculating process where the data to be checked, which rely on, to close
System determines data computations set, and generates the data computations tree including the data computations set.
Optionally, the business data knowledge mapping includes Business Logic, analysis system layer and data indicator layer；
The Business Logic includes the relationship of service groups belonging to data and data service metadata；
The analysis system layer includes analysis method and analysis architectural definition, the text description and business solution of analyzing system
It releases, and the service correlation information of analysis system；
The data target layer includes the definition of data target, the text description of data target, the calculating of data target rule
The store path of model and data target calculates time and history span.
Optionally, further includes:
The metadata information of business data is acquired, and according to the business data knowledge mapping to the metadata information of acquisition
It is arranged, the metadata information after arrangement is saved in the business data knowledge mapping.
Optionally, the metadata information of the acquisition business data, and according to the business data knowledge mapping to acquisition
Metadata information arranged, the metadata information after arrangement is saved in the business data knowledge mapping, comprising:
The metadata information of monitoring and acquisition business data；
According to the metadata information standard of enterprise, cleaning alignment is carried out to the metadata information；
According to the metadata information after cleaning alignment, the business in metadata information after extracting the cleaning alignment is patrolled
Volume, and the service logic is saved in the Business Logic of the business data knowledge mapping；
According to the analysis architectural definition, determine that analysis system in the service logic and the analysis system are corresponding
Classification, and the analysis body that the corresponding classification of the analysis system and the analysis system is saved in business data knowledge mapping
It is layer；
Extraction alignment is carried out to the data target under the corresponding classification of the analysis system, with the unified data target
Title, text description and calculating specification, and the title of the data target, text are described and calculated specification and is saved in enterprise's number
According in the data target layer of knowledge mapping.
Optionally, according to the data computations tree, by the data computations in the data computations tree point
After the corresponding calculating task node of dispensing is executed, further includes:
It is obtained by the local data's management of computing module being deployed in calculating task node according to the data computations
Take corresponding data；Or
It is right by local data's management of computing module for being deployed in calculating task node according to the data computations
The data that other calculating task nodes obtain carry out summarizing calculating.
Optionally, it is described by local data's management of computing module for being deployed in calculating task node according to the data
Computations, the data obtained to other calculating task nodes carry out summarizing calculating, comprising:
Other calculating task nodes hair is received by the local data's management of computing module being deployed in calculating task node
Data to be summarized sent, and according to the data computations, to the data to be summarized carry out data missing value polishing and
Dimensional normalization processing, and data carry out summarizing calculating to treated.
Optionally, described to include: to the data progress data missing value polishing to be summarized
If data to be summarized are the data of different densities, using mean value interpolation polishing, sparse alignment or distribution interpolation
Polishing carries out missing value polishing to the data to be summarized；
If data to be summarized are the data that different methods of summary obtain, the data to be summarized are carried out heuristic
Calculate polishing.
Optionally, the data query result is returned into the user terminal, comprising:
According to customer analysis object, data analysis operator and the result data tissue pattern in the data computations tree
Data computations, determine the data exhibiting template of the data query result；
According to the data exhibiting template, the data query result tissue is showed into sample for the data exhibiting template
Formula obtains the data exhibiting result of the data query result；
The data exhibiting result is sent to the user terminal.
Optionally, the calculating task node includes data center and database.
According to the second aspect of an embodiment of the present disclosure, a kind of big data inquiry unit is provided, comprising:
Inquiry request receiving module is configured as receiving the data inquiry request of user terminal natural language mode；
Instruction tree generation module is configured as according to the data inquiry request, and generating includes data computations set
Data computations tree；
Global data management of computing module is configured as being referred to data calculating according to the data computations tree
It enables the data computations in tree distribute to corresponding calculating task node to be executed；
As a result summarizing module is configured as obtaining the implementing result of the calculating task node, and according to the data meter
Instruction tree is calculated, the implementing result is carried out to summarize calculating, obtains data query result；
Data result display module is configured as the data query result returning to the user terminal.
Optionally, described instruction tree generation module includes:
Query analysis unit is configured as being converted to the data inquiry request into structuralized query data representation sentence；
Calculating process analysis engine is configured as according to the structuralized query data representation sentence, and generating includes data
The data computations tree of computations set, the data computations tree include that data dependence relation and calculating process rely on
Relationship.
Optionally, the query analysis unit is specifically used for:
Text participle and business semantics mark are carried out to the data inquiry request, segmented and semantic annotation result；
Contextual analysis is carried out to the data inquiry request, is patrolled with carrying out business with semantic annotation result to the participle
Polishing is collected, service logic polishing result is obtained；
According to the participle with semantic annotation result and the service logic polishing as a result, generating structuralized query tables of data
Up to sentence.
Optionally, the calculating process analysis engine is specifically used for:
According to the structuralized query data representation sentence and business data knowledge mapping, data place to be checked is determined
Calculating task node, and determine that data dependence relation and calculating process dependence, the business data knowledge mapping include
The metadata information of business data；
Calculating task node, data dependence relation and calculating process where the data to be checked, which rely on, to close
System determines data computations set, and generates the data computations tree including the data computations set.
Optionally, the business data knowledge mapping includes Business Logic, analysis system layer and data indicator layer；
The Business Logic includes the relationship of service groups belonging to data and data service metadata；
The analysis system layer includes analysis method and analysis architectural definition, the text description and business solution of analyzing system
It releases, and the service correlation information of analysis system；
The data target layer includes the definition of data target, the text description of data target, the calculating of data target rule
The store path of model and data target calculates time and history span.
Optionally, the device further includes
Business data knowledge mapping module is configured as the metadata information of acquisition business data, and according to the enterprise
Data knowledge map arranges the metadata information of acquisition, and the metadata information after arrangement is saved in the business data
In knowledge mapping.
Optionally, the business data knowledge mapping module includes:
Metadata acquisition unit is configured as monitoring and acquiring the metadata information of business data；
Alignment unit is cleaned, the metadata information standard according to enterprise is configured as, the metadata information is carried out clear
Wash alignment；
Service logic extracting unit is configured as extracting the cleaning alignment according to the metadata information after cleaning alignment
The service logic in metadata information afterwards, and the business that the service logic is saved in the business data knowledge mapping is patrolled
It collects in layer；
Analysis system determination unit is configured as determining point in the service logic according to the analysis architectural definition
Analysis system and the corresponding classification of the analysis system, and the corresponding classification of the analysis system and the analysis system is saved in
Analysis system layer in business data knowledge mapping；
Data target extracting unit is configured as extracting the data target under the corresponding classification of the analysis system
Alignment with the title of the unified data target, text description and calculates specification, and by the title of the data target, text
Description and calculating specification are saved in the data target layer of business data knowledge mapping.
Optionally, the device further include:
Local data's management of computing module, is deployed in calculating task node, is configured as being referred to according to data calculating
It enables and obtains corresponding data；Alternatively, the data obtained to other calculating task nodes are converged according to the data computations
It is total to calculate.
Optionally, local data's management of computing module includes:
Summarize computing unit, is configured as receiving the data to be summarized of other calculating task nodes transmission, and according to institute
State data computations, data missing value polishing carried out to the data to be summarized and dimensional normalization is handled, and to processing after
Data carry out summarizing calculating.
Optionally, the computing unit that summarizes includes:
Missing value polishing subelement uses mean value interpolation if being configured as the data that data to be summarized are different densities
Polishing, sparse alignment or distribution interpolation polishing carry out missing value polishing to the data to be summarized；If data to be summarized are not
With the data that method of summary obtains, then heuristic calculating polishing is carried out to the data to be summarized.
Optionally, the data result display module is specifically used for:
According to customer analysis object, data analysis operator and the result data tissue pattern in the data computations tree
Data computations, determine the data exhibiting template of the data query result；
According to the data exhibiting template, the data query result tissue is showed into sample for the data exhibiting template
Formula obtains the data exhibiting result of the data query result；
The data exhibiting result is sent to the user terminal.
Optionally, the calculating task node includes data center and database.
According to the third aspect of an embodiment of the present disclosure, a kind of server is provided, comprising:
Processor；
Memory for storage processor executable instruction；
Wherein, the processor is configured to executing big data querying method as described in relation to the first aspect.
According to a fourth aspect of embodiments of the present disclosure, a kind of non-transitorycomputer readable storage medium is provided, when described
When instruction in storage medium is executed by the processor of mobile terminal, so that mobile terminal is able to carry out as described in relation to the first aspect
A kind of big data querying method.
According to a fifth aspect of the embodiments of the present disclosure, a kind of computer program is provided, the method for the computer program includes
The step of one of first aspect big data querying method.
The technical scheme provided by this disclosed embodiment can include the following benefits: by receiving user terminal nature language
The data inquiry request of language model generates the data computations including data computations set according to data inquiry request
Data computations in data computations tree according to data computations tree, are distributed to corresponding calculating task section by tree
Point is executed, and the implementing result of calculating task node is obtained, and according to data computations tree, to each calculating task node
Implementing result carry out summarizing calculating, obtain data query result, and data query result is returned into user terminal, can be to depositing
The data for being stored in different location carry out effective query analysis, realize and carry out data interpretation and integration with the angle of global business,
It effectively can be analyzed to enterprise's domestic demand otherwise with the scene of division data, realize the trans-sectoral business neck in business data statistics
The data cross in domain is associated with, and eliminates user during data query, suffered data organization isolation, physical isolation and industry
The constraints such as business isolation can carry out Operations Analyst to the data of any dimension.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not
The disclosure can be limited.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows and meets implementation of the invention
Example, and be used to explain the principle of the present invention together with specification.
Fig. 1 is a kind of flow chart of big data querying method shown according to an exemplary embodiment；
Fig. 2 is a kind of flow chart of big data querying method shown according to an exemplary embodiment；
Fig. 3 is the stream that data inquiry request is converted to structuralized query data representation sentence in an exemplary embodiment
Cheng Tu；
Fig. 4 is to generate data computations tree according to structuralized query data representation sentence in an exemplary embodiment
Flow chart；
Fig. 5 is the acquisition in an exemplary embodiment and arranges the metadata information of business data to Company Knowledge map
Flow chart；
Fig. 6 is a kind of flow chart of big data querying method shown according to an exemplary embodiment；
Fig. 7 is a kind of structural block diagram of big data inquiry unit shown according to an exemplary embodiment；
Fig. 8 is a kind of structural block diagram of server shown according to an exemplary embodiment.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistented with the present invention.On the contrary, they be only with it is such as appended
The example of device and method being described in detail in claims, some aspects of the invention are consistent.
Fig. 1 is a kind of flow chart of big data querying method shown according to an exemplary embodiment, big data inquiry
Method is used in server, the application scenarios of this method are as follows: difference department, enterprise carries different business responsibilities, is based on industry
Business demand can obtain the support for statistical analysis of the independent data of relevant departments.The data analysis result of each service line is general
It is stored in business isolation, department's isolation or even the persistence medium (such as relevant database, file) across data center, one
As business personnel lack the integration that technical capability carries out these data, and conclude and summarize, but the depth data of single business
Analysis, often further relates to other relevant data results, such as financial analyst needs the sales volume to product, acceptance of the users, confession
Chain situation etc. is answered to carry out comprehensive understanding.
As shown in Figure 1, this approach includes the following steps.
In step s 11, the data inquiry request of user terminal natural language mode is received.
User can input the sentence of natural language mode by user terminal, as data inquiry request.For example, user is defeated
The data inquiry request entered can be with are as follows: I will 10 points of a whole morning all liveness mails letters for subscribing to the mobile App users of groups weekly
Report.
By the data inquiry request of natural language mode so that user can be used oneself understanding business type language or
Natural language inquires the data analysis result of needs.
In step s 12, it according to the data inquiry request, generates the data calculating including data computations set and refers to
Enable tree.
Natural language processing is carried out to the data inquiry request, participle and semanteme such as are carried out to the data inquiry request
Mark, to decompose to the data inquiry request, determines the storage location of data to be checked, generates each storage location
Data computations obtain data computations set, and according to data dependence relation and calculating process dependence, generate packet
Include the data computations tree of data computations set.The storage location of data to be checked is the meter in data computations tree
Calculate task node.
In step s 13, according to the data computations tree, the data calculating in the data computations tree is referred to
Order is distributed to corresponding calculating task node and is executed.
It wherein, include calculating task node and corresponding data computations in data computations tree.Calculating task section
Point includes data center and database.
Data computations tree, bottom-up mode, starting and management data integration summarize calculating
It can be started by global data management of computing module according to data computations tree using bottom-up mode
Summarize calculating with management data integration, data computations are distributed into calculating task section corresponding with the data computations
Point executes data computations by calculating task node, obtains corresponding data.One calculating task node can be obtained from local
Data to be checked are taken, the data that other calculating task nodes are got can also be carried out to summarize calculating, or obtain to local
The data that the data and other calculating task nodes taken are got carry out summarizing calculating.It is calculated by data by instruction tree, it can be with
The distributed query and calculating, the available data to different storage locations of data are carried out, and improves processing speed.
In step S14, the implementing result of the calculating task node is obtained, and according to the data computations tree,
The implementing result is carried out to summarize calculating, obtains data query result.
Server can be the root node in the data computations tree, to the implementing result of each calculating task node
It carries out summarizing calculating, obtains data query result corresponding with the data inquiry request of user.
In step S15, the data query result is returned into the user terminal.
Data query result can be directly returned to user terminal, data query result can also be matched to what user required
Display form, and return to user terminal.
Optionally, the data query result is returned into the user terminal, comprising:
According to customer analysis object, data analysis operator and the result data tissue pattern in the data computations tree
Data computations, determine the data exhibiting template of the data query result；
According to the data exhibiting template, the data query result tissue is showed into sample for the data exhibiting template
Formula obtains the data exhibiting result of the data query result；
The data exhibiting result is sent to the user terminal.
Template can be showed by data result display module come matched data, can store in data result display module
Data analyze the corresponding relationship of operator, customer analysis object, result data tissue pattern and data exhibiting template, determine that data are looked into
Ask the data exhibiting template of result.For example, according to customer analysis object be customer analysis, data analysis operator be Macro or mass analysis and
Result data tissue pattern is bulletin, and available data exhibiting template summarizes for user and shows template 1.In data exhibiting template
In, there are corresponding legend, table, file format to each index of analysis, it, can be according to rule according to the inquiry and scene of user
Template is then automatically extracted, generates and shows pattern desired by user.Such as " user activity week compares report " can be automatically according to mould
Plate generates Visual Report Forms, wherein " user logs in pv/uv " index can be shown automatically with hour/day granularity line chart, and attached
Add the sliding option of time window.
Data exhibiting result is sent to user terminal, is somebody's turn to do so that user terminal can be shown in the form of data exhibiting desired by user
Data exhibiting result.
The big data querying method that the present exemplary embodiment provides, the data by receiving user terminal natural language mode are looked into
Request is ask, according to data inquiry request, the data computations tree including data computations set is generated, is calculated according to data
Data computations in data computations tree are distributed to corresponding calculating task node and executed by instruction tree, obtain
The implementing result of calculating task node, and according to data computations tree, the implementing result of each calculating task node is carried out
Summarize calculating, obtains data query result, and data query result is returned into user terminal, it can be to being stored in different location
Data carry out effective query analysis, realize and carry out data interpretation and integration with the angle of global business, can be to enterprise's domestic demand
Otherwise it is effectively analyzed with the scene of division data, realizes the data cross across business scope in business data statistics and close
Connection, eliminates user during data query, the constraint such as suffered data organization isolation, physical isolation and business isolation,
Operations Analyst can be carried out to the data of any dimension.
Fig. 2 is a kind of flow chart of big data querying method shown according to an exemplary embodiment, as shown in Fig. 2, should
Method includes the following steps.
In the step s 21, the data inquiry request of user terminal natural language mode is received.
The particular content of this step is identical as the particular content of step S11 in the above exemplary embodiments, here no longer
It repeats.
In step S22, the data inquiry request is converted into structuralized query data representation sentence.
Wherein, structuralized query data representation sentence description information include: 1) identity of inquiry, owning user group and
Current contextual information；2) user query operation calculating arrangement, as single starting or start by set date, the entry-into-force time and/or
Expired time etc.；3) user wishes to inquire the scope of business of data, and the scope of business includes enterprise belonging to the data to be inquired
Industry, department, product line and specific data owner's list；4) description object of user accesses data, such as user data,
Product data, financial data or supply chain data etc.；5) the corresponding data of user's request data analyze operator, available data point
Analysing operator includes data comparison (Compare), data summarization (Aggregation), key factor (Factor) etc.；6) user's phase
Data exhibiting form, including mail, report, file or APIs of prestige etc.；7) data organization form desired by user, including letter
Report, common or detail.
Fig. 3 is the stream that data inquiry request is converted to structuralized query data representation sentence in an exemplary embodiment
Cheng Tu, as shown in figure 3, described be converted to structuralized query data representation sentence for the data inquiry request, may include with
Lower step:
In step S221, text participle and business semantics are carried out to the data inquiry request and are marked, obtain participle with
Semantic annotation result.
Text participle is carried out to the data inquiry request of natural language mode, word segmentation result is obtained, to each word segmentation result
Business semantics mark is carried out, is segmented and semantic annotation result.For example, the data inquiry request of natural language mode are as follows: " I
Will 10 points of a whole morning all liveness mail bulletins for subscribing to the mobile App user of groups weekly ", pass through form shown in table 1 and converts
For standardized input semantic component, segmented and semantic annotation result: I am (Requester), (V), weekly a whole morning
It 10 points (Task Scheduling), subscribes to (V), group (Enterprise), mobile App (Business Group), user
(Analysis Object), all (Data Time Span), liveness (Analysis Method), mail (Result
), Format bulletin (Result Schema).
The participle of table 1 is marked with business semantics
In step S222, contextual analysis is carried out to the data inquiry request, to the participle and semantic tagger
As a result service logic polishing is carried out, service logic polishing result is obtained.
Still by taking above-mentioned data inquiry request as an example, contextual analysis is carried out to data inquiry request, determines collection belonging to user
Group is ECS, department E2E；User browses KPI report data by Web client.It obtains contextual analysis result: using
Family group (User Group)-Lenovo:ECS:E2E:Analyist, user's scene (Query Scenario)-Web Client+
KPI Report.When carrying out service logic polishing with semantic annotation result to the participle, according to affiliated group of user, polishing is inquired
The affiliated service groups of data are whole department, ECS group；According to active user's scene, data result is generated into the end Web report.It obtains
Service logic polishing result: the affiliated service groups of data (Data Group)-Lenovo:ECS:All_Dep:App, data result sample
Formula (Result Schema)-Web Report+Email.
In step S223, tied with semantic annotation result and the service logic polishing as a result, generating according to the participle
Structureization inquires data representation sentence.
Still by taking above-mentioned data inquiry request as an example, obtained partial structured inquiry data representation sentence is as follows:
Enterprise_ID:002,035
Department_ID:0021,0358
Product_ID:156,386
Analysis_Obj:user
Analysis_Method:Aggregation
Result Format:Email+Web Report
Result Schema:Briefing
Above structureization inquiry data representation sentence is the partial structured inquiry number of above-mentioned data inquiry request conversion
According to expression sentence, further includes many relevant to data query structuralized query expression sentences, no longer list one by one here.It utilizes
The technologies such as natural language processing and professional knowledge ontology library realize the natural language querying language by user based on operational angle
Sentence, is converted to structuralized query data representation sentence, can be in order to business personnel's understanding and operation data.
In step S23, according to the structuralized query data representation sentence, generating includes data computations set
Data computations tree, the data computations tree include data dependence relation and calculating process dependence.
According to structuralized query data representation sentence, generate data computations set, and according to data dependence relation and
The data computations set expression is tree-shaped execution process by calculating process dependence, is obtained the data calculating and is referred to
Enable tree.
Fig. 4 is to generate data computations tree according to structuralized query data representation sentence in an exemplary embodiment
Flow chart, as shown in figure 4, described according to the structuralized query data representation sentence, generating includes data computations set
Data computations tree, comprising:
In step S231, according to the structuralized query data representation sentence and business data knowledge mapping, determine to
Calculating task node where the data of inquiry, and determine data dependence relation and calculating process dependence, enterprise's number
It include the metadata information of business data according to knowledge mapping.
Wherein, business data knowledge mapping is metadata (Metadata) information of the data in full enterprise-wide.First number
The information of data attribute is mainly described, for supporting such as according to also known as broker data, relaying data for the data for describing data
Indicate the functions such as storage location, historical data, resource lookup, file record.Metadata is a kind of electronic type catalogue, in order to reach
The purpose of scheduling, it is necessary to describe and collect the interior perhaps characteristic of data, and then reach the purpose for assisting data retrieval.
The metadata information of enterprise's big data is stored and is reported using predefined standardized data structures, unified by enterprise
It formulates data structure standard and designs and manage the data of respective business by each department and data team in accordance with business data standard
Structure and metadata information.
As shown in table 2, it includes Business Logic, analysis system layer and data target that the business data knowledge mapping is optional
Layer；The Business Logic includes the relationship of service groups belonging to data and data service metadata；The analysis system layer includes point
Analysis method and analysis architectural definition, the text description and business explanation of analyzing system, and the service correlation information of analysis system；
The data target layer include the definition of data target, data target text description, data target calculating specification, and number
According to the store path of index, calculate time and history span.Business data knowledge mapping can provide query interface, for for clothes
Business device query metadata information is closed come the data dependence for determining calculating task node where data to be checked and between data
System and calculating process dependence.
2 business data knowledge mapping structure of table
Calculating task node, data dependence relation and meter in step S232, where the data to be checked
Calculation process dependence determines data computations set, and generates the data including the data computations set and calculate
Instruction tree.
Calculating needed for user query is decomposed as calculating process analysis engine, and according to data dependence relation and was calculated
Data computations set is organized into tree-shaped execution process by journey dependence, and each non-leaf nodes represents one and summarizes place
Reason, each leaf node represent the reading process of an initial data, the final data analysis result of root nodes stand.For example,
Root node in data computations tree is server, and the calculating task node of server next stage is data center, in data
The calculating task node of heart next stage is database, and the calculating task node of database next stage is tables of data, thus server
According to the data center where data to be checked, so that the data with inquiry are assigned to corresponding data center, such as
Data to be checked are located at different databases in one data center of fruit, then corresponding data computations are assigned to correspondence
Database, if data to be checked in a database are located at different tables of data, further according to data computations obtain
Data in different data table.
The data inquiry request of user and the physical mappings relationship of business data, can be by inquiring business data knowledge graph
Spectrum obtains, for example, which product line " mobile App " specifically covers, " user activity " specifically corresponds to which statistical indicator, target
The storage of data specifically corresponds to which data center, which database and tables of data etc., generates data computations to parse
Set, so that data computations set is executed and be managed with management of computing module by global data, and generates data
Computations tree, control calculating task node are gradually executed according to data computations tree is bottom-up.
By realizing structuralized query data representation language using technologies such as computer language compiling and distributed computings
Sentence, which is converted into, the distributed data query executed and to summarize integrated data computations set, according to priority and step
Dependent status optimizes tissue to data computations set.
In step s 24, according to the data computations tree, the data calculating in the data computations tree is referred to
Order is distributed to corresponding calculating task node and is executed.
The particular content of this step is identical as the particular content of step S13 in the above exemplary embodiments, here no longer
It repeats.
In step s 25, the implementing result of the calculating task node is obtained, and according to the data computations tree,
The implementing result is carried out to summarize calculating, obtains data query result.
The particular content of this step is identical as the particular content of step S14 in the above exemplary embodiments, here no longer
It repeats.
In step S26, the data query result is returned into the user terminal.
The particular content of this step is identical as the particular content of step S15 in the above exemplary embodiments, here no longer
It repeats.
The big data querying method that the present exemplary embodiment provides, by converting structuralized query for data inquiry request
Data representation sentence generates the data calculating including data computations set and refers to according to structuralized query data representation sentence
Enable tree, instruction tree in include data dependence relation and calculating process dependence, so as to according to data computations tree by
Each calculating task Node distribution formula obtains corresponding data, so as to get the accurate data in enterprise's global scope.
Based on the above technical solution, also optional to include:
The metadata information of business data is acquired, and according to the business data knowledge mapping to the metadata information of acquisition
It is arranged, the metadata information after arrangement is saved in the business data knowledge mapping.
It can be by the metadata information of business data knowledge mapping module monitors and acquisition business data, and to acquisition
Metadata information is arranged, and is saved in business data knowledge mapping.
Fig. 5 is the acquisition in an exemplary embodiment and arranges the metadata information of business data to Company Knowledge map
Flow chart, as shown in figure 5, the metadata information of the acquisition business data, and according to the business data knowledge mapping to adopting
The metadata information of collection is arranged, and the metadata information after arrangement is saved in the business data knowledge mapping, comprising:
In step S501, the metadata information of monitoring and acquisition business data.
In step S502, according to the metadata information standard of enterprise, cleaning alignment is carried out to the metadata information.
According to the metadata information standard of enterprise, and text analysis technique is combined, to the member of enterprise's global data of collection
Data information carries out cleaning alignment, for example, will the synonymous index term such as " user activity, user's activity and activation user " into
" mobile division department, MBD, Mobile Business Department " etc. is referred to same service groups by row normalized
Term merges processing.
Metadata letter in step S503, according to the metadata information after cleaning alignment, after extracting the cleaning alignment
Service logic in breath, and the service logic is saved in the Business Logic of the business data knowledge mapping.
To the metadata information after cleaning alignment, service groups and business belonging to the data in automatic extracting metadata information
The service logics such as the relationship of metadata, and automatically according to hierarchical relationship induction-arrangement, such as " KPI sells in mobile division department, group
It can extract in index " level business structure " XX group → movement division department → selling operation " etc..
In step S504, according to the analysis architectural definition, analysis system in the service logic and described is determined
The corresponding classification of analysis system, and the corresponding classification of the analysis system and the analysis system is saved in business data knowledge
Analysis system layer in map.
The definition (such as sale KPI, user activity, advertisement marketing effect analysis) of system is analyzed according to data, first
The method of trial machine learning automatically concludes newfound analysis system in the analysis system for having classification, for can not
Automatically the data classified take the mode of similarity cluster (such as index similarity or text description similarity), first will classification
System cluster is concluded carrying out artificial mark, and that takes in business data knowledge mapping has point that in classification or creation is new
Class branch.
In step S505, extraction alignment is carried out to the data target under the corresponding classification of the analysis system, with unified
Title, text description and the calculating specification of the data target, and the title of the data target, text are described and calculated to advise
Model is saved in the data target layer of business data knowledge mapping.
By monitoring and acquiring the metadata information of business data, and arranges and be saved in business data knowledge mapping, and
Query interface is provided, is convenient for inquiry to position data to be checked and determine that data dependence relation and calculating process rely on
Relationship.
Fig. 6 is a kind of flow chart of big data querying method shown according to an exemplary embodiment, as shown in fig. 6, should
Method may include:
In step S61, the data inquiry request of user terminal natural language mode is received.
The particular content of this step is identical as the particular content of step S11 in the above exemplary embodiments, here no longer
It repeats.
In step S62, according to the data inquiry request, generates the data calculating including data computations set and refer to
Enable tree.
The particular content of this step is identical as the particular content of step S12 in the above exemplary embodiments, here no longer
It repeats.
In step S63, according to the data computations tree, the data calculating in the data computations tree is referred to
Order is distributed to corresponding calculating task node and is executed.
According to data needed for each calculating task node be subordinate to and storage and distribution, in conjunction with the utilization of global data computing resource
Data analytical calculation task is distributed to the calculating tasks sections such as data center, the enterprise/division data platform where data by situation
Point is calculated by the data for disposing local data thereon and management of computing module starts and management is respectively administered, and is notified in real time
Calculating state.After the completion of each calculating task calculates, is summarized if you need to the result with other calculate nodes, then take data just
Closely summarize principle and operation storage capacity dominance principle, successively choose data specific gravity maximum and calculates the most sufficient number of storage resource
Data summarization calculating is carried out according to the calculating tasks node such as library, data platform or data center.
In step S64, by local data's management of computing module for being deployed in calculating task node according to the number
Corresponding data are obtained according to computations；Alternatively, passing through the local data's management of computing module being deployed in calculating task node
According to the data computations, the data obtained to other calculating task nodes carry out summarizing calculating.
The local data's management of computing module being deployed in calculating task node is responsible for managing the number of the calculating task node
According to and calculate, support data management across data platform and management of computing inside data center.Calculating task node can be one
A independent data center, business subregion or department's subregion.In a complete calculation process, local data and management of computing
Module may be assigned a computations subtree.
Further minute inspection verifies the completeness for calculating required data and can automatically for local data and management of computing module
With property, and the task status in calculating process is monitored, the global data into server and management of computing module update in real time, such as
Fruit calculates failure, it tries starting retries strategy.
Optionally, it is described by local data's management of computing module for being deployed in calculating task node according to the data
Computations, the data obtained to other calculating task nodes carry out summarizing calculating, comprising:
Other calculating task nodes hair is received by the local data's management of computing module being deployed in calculating task node
Data to be summarized sent, and according to the data computations, to the data to be summarized carry out data missing value polishing and
Dimensional normalization processing, and data carry out summarizing calculating to treated.
Local data and management of computing module may undertake one or more in calculation process and summarize calculating task, receive
Local or other local datas and management of computing module calculate the data completed, and execute the integrated of higher level and summarize, and are responsible for
Result is sent to upper one layer of calculating task node, until task root node.If data to be summarized are different dimension data,
Use the methods of standard normalizing (mean value 0, variance 1) or unidirectional normalizing (using one of data as data zooming standard) place
Reason, so that the dimension normalizing of data to be summarized.
Optionally, described to include: to the data progress data missing value polishing to be summarized
If data to be summarized are the data of different densities, using mean value interpolation polishing, sparse alignment or distribution interpolation
Polishing carries out missing value polishing to the data to be summarized；
If data to be summarized are the data that different methods of summary obtain, the data to be summarized are carried out heuristic
Calculate polishing.
For the data of different densities, use mean value interpolation polishing (filling data by mean value), sparse alignment (with sparse number
Subject to), data processing is aligned by distribution the methods of interpolation polishing (estimate by data distribution and fill data).For example, data A
It is compared with data B, but A is hour granularity data, B is day granularity data, then the above method can be used by A and B long
Degree alignment.It for the data of same source difference method of summary, is then aligned using heuristic calculating, such as A is mean value, B is total
Meter, then obtain C=A*N, then compare C and B again.
In step S65, the implementing result of the calculating task node is obtained, and according to the data computations tree,
The implementing result is carried out to summarize calculating, obtains data query result.
The particular content of this step is identical as the particular content of step S14 in the above exemplary embodiments, here no longer
It repeats.
In step S66, the data query result is returned into the user terminal.
The big data querying method that the present exemplary embodiment provides, passes through the local data being deployed in calculating task node
Management of computing module is responsible for acquisition and the management of computing of the data in the calculating task node, or to other calculating task nodes
Data carry out summarizing calculating, to realize the distributed computing of each calculating task node, realize in enterprise's overall situation
Carry out the acquisition of business data.
Fig. 7 is a kind of structural block diagram of big data inquiry unit shown according to an exemplary embodiment.It, should referring to Fig. 7
Device includes inquiry request receiving module 71, tree generation module 72, global data management of computing module 73, result is instructed to summarize mould
Block 74 and data result presentation module 73.
The inquiry request receiving module 71 is configured as receiving the data inquiry request of user terminal natural language mode；
The instruction tree generation module 72 is configured as according to the data inquiry request, and generating includes data computations collection
The data computations tree of conjunction；
The global data management of computing module 73 is configured as being calculated the data according to the data computations tree
Data computations in instruction tree are distributed to corresponding calculating task node and are executed；
The result summarizing module 74 is configured as obtaining the implementing result of the calculating task node, and according to the data
Computations tree carries out the implementing result to summarize calculating, obtains data query result；
The data result display module 75 is configured as the data query result returning to the user terminal.
Optionally, described instruction tree generation module includes:
Query analysis unit is configured as being converted to the data inquiry request into structuralized query data representation sentence；
Calculating process analysis engine is configured as according to the structuralized query data representation sentence, and generating includes data
The data computations tree of computations set, the data computations tree include that data dependence relation and calculating process rely on
Relationship.
Optionally, the query analysis unit is specifically used for:
Text participle and business semantics mark are carried out to the data inquiry request, segmented and semantic annotation result；
Contextual analysis is carried out to the data inquiry request, is patrolled with carrying out business with semantic annotation result to the participle
Polishing is collected, service logic polishing result is obtained；
According to the participle with semantic annotation result and the service logic polishing as a result, generating structuralized query tables of data
Up to sentence.
Optionally, the calculating process analysis engine is specifically used for:
According to the structuralized query data representation sentence and business data knowledge mapping, data place to be checked is determined
Calculating task node, and determine that data dependence relation and calculating process dependence, the business data knowledge mapping include
The metadata information of business data；
Calculating task node, data dependence relation and calculating process where the data to be checked, which rely on, to close
System determines data computations set, and generates the data computations tree including the data computations set.
Optionally, the business data knowledge mapping includes Business Logic, analysis system layer and data indicator layer；
The Business Logic includes the relationship of service groups belonging to data and data service metadata；
The analysis system layer includes analysis method and analysis architectural definition, the text description and business solution of analyzing system
It releases, and the service correlation information of analysis system；
The data target layer includes the definition of data target, the text description of data target, the calculating of data target rule
The store path of model and data target calculates time and history span.
Optionally, the device further includes
Business data knowledge mapping module is configured as the metadata information of acquisition business data, and according to the enterprise
Data knowledge map arranges the metadata information of acquisition, and the metadata information after arrangement is saved in the business data
In knowledge mapping.
Optionally, the business data knowledge mapping module includes:
Metadata acquisition unit is configured as monitoring and acquiring the metadata information of business data；
Alignment unit is cleaned, the metadata information standard according to enterprise is configured as, the metadata information is carried out clear
Wash alignment；
Service logic extracting unit is configured as extracting the cleaning alignment according to the metadata information after cleaning alignment
The service logic in metadata information afterwards, and the business that the service logic is saved in the business data knowledge mapping is patrolled
It collects in layer；
Analysis system determination unit is configured as determining point in the service logic according to the analysis architectural definition
Analysis system and the corresponding classification of the analysis system, and the corresponding classification of the analysis system and the analysis system is saved in
Analysis system layer in business data knowledge mapping；
Data target extracting unit is configured as extracting the data target under the corresponding classification of the analysis system
Alignment with the title of the unified data target, text description and calculates specification, and by the title of the data target, text
Description and calculating specification are saved in the data target layer of business data knowledge mapping.
Optionally, the device further include:
Local data's management of computing module, is deployed in calculating task node, is configured as being referred to according to data calculating
It enables and obtains corresponding data；Alternatively, the data obtained to other calculating task nodes are converged according to the data computations
It is total to calculate.
Optionally, local data's management of computing module includes:
Summarize computing unit, is configured as receiving the data to be summarized of other calculating task nodes transmission, and according to institute
State data computations, data missing value polishing carried out to the data to be summarized and dimensional normalization is handled, and to processing after
Data carry out summarizing calculating.
Optionally, the computing unit that summarizes includes:
Missing value polishing subelement uses mean value interpolation if being configured as the data that data to be summarized are different densities
Polishing, sparse alignment or distribution interpolation polishing carry out missing value polishing to the data to be summarized；If data to be summarized are not
With the data that method of summary obtains, then heuristic calculating polishing is carried out to the data to be summarized.
Optionally, the data result display module is specifically used for:
According to customer analysis object, data analysis operator and the result data tissue pattern in the data computations tree
Data computations, determine the data exhibiting template of the data query result；
According to the data exhibiting template, the data query result tissue is showed into sample for the data exhibiting template
Formula obtains the data exhibiting result of the data query result；
The data exhibiting result is sent to the user terminal.
Optionally, the calculating task node includes data center and database.
About the device in above-described embodiment, wherein modules execute the concrete mode of operation in related this method
Embodiment in be described in detail, no detailed explanation will be given here.
Fig. 8 is a kind of structural block diagram of server shown according to an exemplary embodiment.Referring to Fig. 8, server 800 is wrapped
Processing component 822 is included, further comprises one or more processors, and the memory resource as representated by memory 832,
It can be by the instruction of the execution of processing component 822, such as application program for storing.The application program stored in memory 832 can
With include it is one or more each correspond to one group of instruction module.In addition, processing component 822 is configured as executing
Instruction, to execute the above method.
Server 800 can also include that a power supply module 826 be configured as the power management of execute server 800, and one
A wired or wireless network interface 850 is configured as server 800 being connected to network and input and output (I/O) interface
858.Server 800 can be operated based on the operating system for being stored in memory 1932, such as Windows ServerTM, Mac
OS XTM, UnixTM, LinuxTM, FreeBSDTM or similar.
In the exemplary embodiment, a kind of non-transitorycomputer readable storage medium including instruction, example are additionally provided
It such as include the memory 832 of instruction, above-metioned instruction can be executed by the processing component 822 of server 800 to complete the above method.Example
Such as, the non-transitorycomputer readable storage medium can be ROM, random access memory (RAM), CD-ROM, tape, soft
Disk and optical data storage devices etc..
The application also provides a kind of computer program, which realizes above-mentioned big data when being executed by processor
Querying method.
Field technical staff after considering the specification and implementing the invention disclosed here, will readily occur to of the invention other
Embodiment.This application is intended to cover any variations, uses, or adaptations of the invention, these modifications, purposes or
Adaptive change follow general principle of the invention and including the undocumented common knowledge in the art of the disclosure or
Conventional techniques.The description and examples are only to be considered as illustrative, and true scope and spirit of the invention are by following power
Benefit requires to point out.
It should be understood that the present invention is not limited to the precise structure already described above and shown in the accompanying drawings, and
And various modifications and changes may be made without departing from the scope thereof.The scope of the present invention is limited only by the attached claims.
Claims (10)
1. a kind of big data querying method characterized by comprising
Receive the data inquiry request of user terminal natural language mode；
According to the data inquiry request, the data computations tree including data computations set is generated；
According to the data computations tree, the data computations in the data computations tree are distributed into corresponding meter
Task node is calculated to be executed；
Obtain the implementing result of the calculating task node, and according to the data computations tree, to the implementing result into
Row summarizes calculating, obtains data query result；
The data query result is returned into the user terminal.
2. generating includes number the method according to claim 1, wherein described according to the data inquiry request
According to the data computations tree of computations set, comprising:
The data inquiry request is converted into structuralized query data representation sentence；
According to the structuralized query data representation sentence, the data computations tree including data computations set is generated,
The data computations tree includes data dependence relation and calculating process dependence.
3. according to the method described in claim 2, it is characterized in that, described be converted to structuring for the data inquiry request and look into
Ask data representation sentence, comprising:
Text participle and business semantics mark are carried out to the data inquiry request, segmented and semantic annotation result；
Contextual analysis is carried out to the data inquiry request, to carry out service logic benefit with semantic annotation result to the participle
Together, service logic polishing result is obtained；
According to the participle with semantic annotation result and the service logic polishing as a result, generating structuralized query data representation language
Sentence.
4. according to the method described in claim 2, it is characterized in that, described according to the structuralized query data representation sentence,
Generate the data computations tree including data computations set, comprising:
According to the structuralized query data representation sentence and business data knowledge mapping, the meter where data to be checked is determined
Task node is calculated, and determines data dependence relation and calculating process dependence, the business data knowledge mapping includes enterprise
The metadata information of data；
Calculating task node, data dependence relation and calculating process dependence where the data to be checked, really
Fixed number generates the data computations tree including the data computations set according to computations set.
5. according to the method described in claim 4, it is characterized in that, the business data knowledge mapping include Business Logic,
Analyze system layer and data indicator layer；
The Business Logic includes the relationship of service groups belonging to data and data service metadata；
The analysis system layer includes that analysis method and analysis architectural definition, the text description of analysis system and business are explained, with
And the service correlation information of analysis system；
The data target layer include the definition of data target, data target text description, data target calculating specification, with
And store path, calculating time and the history span of data target.
6. according to the method described in claim 5, it is characterized by further comprising:
The metadata information of business data is acquired, and is carried out according to metadata information of the business data knowledge mapping to acquisition
It arranges, the metadata information after arrangement is saved in the business data knowledge mapping.
7. according to the method described in claim 6, it is characterized in that, it is described acquisition business data metadata information, and according to
The business data knowledge mapping arranges the metadata information of acquisition, the metadata information after arrangement is saved in described
In business data knowledge mapping, comprising:
The metadata information of monitoring and acquisition business data；
According to the metadata information standard of enterprise, cleaning alignment is carried out to the metadata information；
According to the metadata information after cleaning alignment, the service logic in the metadata information after the cleaning is aligned is extracted, and
The service logic is saved in the Business Logic of the business data knowledge mapping；
According to the analysis architectural definition, the analysis system and corresponding point of the analysis system in the service logic are determined
Class, and the analysis system that the corresponding classification of the analysis system and the analysis system is saved in business data knowledge mapping
Layer；
Extraction alignment is carried out to the data target under the corresponding classification of the analysis system, with the name of the unified data target
Title, text description and calculating specification, and the title of the data target, text description and calculating specification are saved in business data
In the data target layer of knowledge mapping.
8. a kind of big data inquiry unit characterized by comprising
Inquiry request receiving module is configured as receiving the data inquiry request of user terminal natural language mode；
Instruction tree generation module is configured as generating the number including data computations set according to the data inquiry request
According to computations tree；
Global data management of computing module is configured as according to the data computations tree, by the data computations tree
In data computations distribute to corresponding calculating task node and executed；
As a result summarizing module is configured as obtaining the implementing result of the calculating task node, and is referred to according to data calculating
Tree is enabled, the implementing result is carried out to summarize calculating, obtains data query result；
Data result display module is configured as the data query result returning to the user terminal.
9. a kind of server characterized by comprising
Processor；
Memory for storage processor executable instruction；
Wherein, the processor is configured to executing such as the described in any item big data querying methods of claim 1-7.
10. a kind of non-transitorycomputer readable storage medium, which is characterized in that when the instruction in the storage medium is by moving
When the processor of terminal executes, so that mobile terminal is able to carry out a kind of big data querying method, the method includes such as rights
It is required that the described in any item steps of 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811526906.1A CN109739878B (en) | 2018-12-13 | 2018-12-13 | Big data query method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811526906.1A CN109739878B (en) | 2018-12-13 | 2018-12-13 | Big data query method, device, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109739878A true CN109739878A (en) | 2019-05-10 |
| CN109739878B CN109739878B (en) | 2020-12-01 |
Family
ID=66358924
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811526906.1A Active CN109739878B (en) | 2018-12-13 | 2018-12-13 | Big data query method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109739878B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110636105A (en) * | 2019-08-14 | 2019-12-31 | 北京金堤科技有限公司 | Tree graph obtaining method and device, storage medium and electronic equipment |
| CN110727526A (en) * | 2019-09-09 | 2020-01-24 | 上海陆家嘴国际金融资产交易市场股份有限公司 | Big data computing task processing method and system |
| CN111934958A (en) * | 2020-07-29 | 2020-11-13 | 深圳市高德信通信股份有限公司 | IDC resource scheduling service management platform |
| CN112434195A (en) * | 2020-11-30 | 2021-03-02 | 天津狮拓信息技术有限公司 | Data analysis method and device, electronic equipment and computer readable storage medium |
| CN113157720A (en) * | 2020-01-23 | 2021-07-23 | 阿里巴巴集团控股有限公司 | Query statement processing method, device, equipment and medium |
| CN114579545A (en) * | 2022-01-19 | 2022-06-03 | 广州新科佳都科技有限公司 | Method and device for querying data of multiple bins based on subway station system |
| CN114730313A (en) * | 2019-11-05 | 2022-07-08 | 塔谱软件公司 | Visual analysis method and user interface for data visualization with multi-line computation |
| CN116244410A (en) * | 2023-02-16 | 2023-06-09 | 北京三维天地科技股份有限公司 | Index data analysis method and system based on knowledge graph and natural language |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102073701A (en) * | 2010-12-30 | 2011-05-25 | 浪潮集团山东通用软件有限公司 | Semantic definition-based multi-data source data querying method |
| US20150347407A1 (en) * | 2014-06-03 | 2015-12-03 | Google Inc. | Dynamic current results for second device |
| CN105512226A (en) * | 2015-11-30 | 2016-04-20 | 北大方正集团有限公司 | Query optimization method and device |
| CN106776639A (en) * | 2015-11-24 | 2017-05-31 | 腾讯科技（深圳）有限公司 | Data processing method and data processing equipment based on SQL |
| CN106959820A (en) * | 2016-01-11 | 2017-07-18 | 杭州海康威视数字技术股份有限公司 | A kind of data extraction method and system |
| US20180129690A1 (en) * | 2016-11-04 | 2018-05-10 | International Business Machines Corporation | Schema-Free In-Graph Indexing |
| US20180246884A1 (en) * | 2012-04-26 | 2018-08-30 | International Business Machines Corporation | Enterprise-level data protection with variable data granularity and data disclosure control with hierarchical summarization, topical structuring, and traversal audit |
| CN108920608A (en) * | 2018-06-28 | 2018-11-30 | 百应科技（北京）有限公司 | A kind of search field knowledge mapping construction method and system towards business data |
-
2018
- 2018-12-13 CN CN201811526906.1A patent/CN109739878B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102073701A (en) * | 2010-12-30 | 2011-05-25 | 浪潮集团山东通用软件有限公司 | Semantic definition-based multi-data source data querying method |
| US20180246884A1 (en) * | 2012-04-26 | 2018-08-30 | International Business Machines Corporation | Enterprise-level data protection with variable data granularity and data disclosure control with hierarchical summarization, topical structuring, and traversal audit |
| US20150347407A1 (en) * | 2014-06-03 | 2015-12-03 | Google Inc. | Dynamic current results for second device |
| CN106776639A (en) * | 2015-11-24 | 2017-05-31 | 腾讯科技（深圳）有限公司 | Data processing method and data processing equipment based on SQL |
| CN105512226A (en) * | 2015-11-30 | 2016-04-20 | 北大方正集团有限公司 | Query optimization method and device |
| CN106959820A (en) * | 2016-01-11 | 2017-07-18 | 杭州海康威视数字技术股份有限公司 | A kind of data extraction method and system |
| US20180129690A1 (en) * | 2016-11-04 | 2018-05-10 | International Business Machines Corporation | Schema-Free In-Graph Indexing |
| CN108920608A (en) * | 2018-06-28 | 2018-11-30 | 百应科技（北京）有限公司 | A kind of search field knowledge mapping construction method and system towards business data |
Non-Patent Citations (2)
| Title |
|---|
| YISHAN: "Constructing target-aware results for keyword search on knowledge graphs", 《DATA & KNOWLEDGE ENGINEERING》 * |
| 张建: "大规模语义数据存储和查询技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110636105A (en) * | 2019-08-14 | 2019-12-31 | 北京金堤科技有限公司 | Tree graph obtaining method and device, storage medium and electronic equipment |
| CN110727526A (en) * | 2019-09-09 | 2020-01-24 | 上海陆家嘴国际金融资产交易市场股份有限公司 | Big data computing task processing method and system |
| CN110727526B (en) * | 2019-09-09 | 2023-02-03 | 未鲲(上海)科技服务有限公司 | Method and system for processing big data computing task |
| CN114730313A (en) * | 2019-11-05 | 2022-07-08 | 塔谱软件公司 | Visual analysis method and user interface for data visualization with multi-line computation |
| CN113157720A (en) * | 2020-01-23 | 2021-07-23 | 阿里巴巴集团控股有限公司 | Query statement processing method, device, equipment and medium |
| CN111934958A (en) * | 2020-07-29 | 2020-11-13 | 深圳市高德信通信股份有限公司 | IDC resource scheduling service management platform |
| CN111934958B (en) * | 2020-07-29 | 2022-03-29 | 深圳市高德信通信股份有限公司 | IDC resource scheduling service management platform |
| CN112434195A (en) * | 2020-11-30 | 2021-03-02 | 天津狮拓信息技术有限公司 | Data analysis method and device, electronic equipment and computer readable storage medium |
| CN114579545A (en) * | 2022-01-19 | 2022-06-03 | 广州新科佳都科技有限公司 | Method and device for querying data of multiple bins based on subway station system |
| CN116244410A (en) * | 2023-02-16 | 2023-06-09 | 北京三维天地科技股份有限公司 | Index data analysis method and system based on knowledge graph and natural language |
| CN116244410B (en) * | 2023-02-16 | 2023-10-20 | 北京三维天地科技股份有限公司 | Index data analysis method and system based on knowledge graph and natural language |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109739878B (en) | 2020-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109739878A (en) | Big data querying method, device, server and storage medium | |
| JP7030707B2 (en) | Systems and methods for generating metadata-driven external interfaces for application programming interfaces | |
| Hellerstein et al. | Ground: A Data Context Service. | |
| CN103733194A (en) | Dynamically organizing cloud computing resources to facilitate discovery | |
| US20130173643A1 (en) | Providing information management | |
| CN103425740B (en) | A kind of material information search method based on Semantic Clustering of internet of things oriented | |
| Fonseca et al. | A framework for measuring the interoperability of geo-ontologies | |
| Debattista et al. | Linked'Big'Data: towards a manifold increase in big data value and veracity | |
| Petermann et al. | BIIIG: enabling business intelligence with integrated instance graphs | |
| CN103577931A (en) | Reusable-software-service management system based on semanteme and implementation method thereof | |
| Imanbayev et al. | Analysis and mathematical modeling of big data processing | |
| CN115221337A (en) | Data weaving processing method and device, electronic equipment and readable storage medium | |
| Becker et al. | Semantically standardized and transparent process model collections via process building blocks | |
| Salih et al. | Data quality issues in big data: a review | |
| Pushpalatha et al. | Enterprise data | |
| CN109242431A (en) | A kind of enterprise management method and its system based on data system | |
| Sassi et al. | Supporting ontology adaptation and versioning based on a graph of relevance | |
| Palopoli et al. | Experiences using DIKE, a system for supporting cooperative information system and data warehouse design | |
| Rana et al. | Analysis of web mining technology and their impact on semantic web | |
| Singh et al. | SSMDM: An approach of big data for semantically master data management | |
| Mouhoub et al. | A framework for searching semantic data and services with SPARQL | |
| Salguero et al. | Spatio-temporal ontology based model for data warehousing | |
| Cheng et al. | Smart Home Service Experience Strategic Foresight Using the Social Network Analysis and Future Triangle | |
| Bhattacharjee et al. | Ontology based spatial clustering framework for implicit knowledge discovery | |
| van Dijk et al. | Maturing Pay-as-you-go Data Quality Management: Towards Decision Support for Paying the Larger Bills |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |