CN106095952A - In space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying - Google Patents
In space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying Download PDFInfo
- Publication number
- CN106095952A CN106095952A CN201610423093.8A CN201610423093A CN106095952A CN 106095952 A CN106095952 A CN 106095952A CN 201610423093 A CN201610423093 A CN 201610423093A CN 106095952 A CN106095952 A CN 106095952A
- Authority
- CN
- China
- Prior art keywords
- time
- key assignments
- concordance list
- space
- tree
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
In the present invention relates to a kind of space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying, and wherein said method includes structure time and the key assignments concordance list of space segment point；The key assignments concordance list of structure time and space segment tree；And the time of structure and space segment tree update and delete key assignments concordance list.Use this kind of method, by introducing space-time waypoint concordance list, be greatly improved car record quick search efficiency excessively based on space-time single-point；By introducing space-time segmentation tree concordance list, it is greatly improved car record quick search efficiency excessively based on space-time unique；Updating concordance list by introducing space-time segmentation tree, dynamic reconstruct space-time segmentation tree concordance list, for supporting that the car record real-time query that flows through of magnanimity provides possible；Preserve above-mentioned concordance list by introducing key assignments cloud storage data base, and utilize Hadoop cloud platform to generate above-mentioned concordance list, substantially reduce the time building and safeguarding index, and improve query processing performance.
Description
Technical field
The present invention relates to computer database technology field, particularly relate to distributed index and the inquiry of cloud platform data base
Process field, in specifically referring to a kind of space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying.
Background technology
In space-time unique based on key assignments cloud storage magnanimity cross car record method for quickly querying mainly realize magnanimity cross car knot
Structure record is based respectively on space-time unique and the quick search of space-time single-point in cloud platform.In recent years, along with MapReduce and
The further investigation of the key value database technology of hadoop and Nosql of hdfs so that above-mentioned two class inquiry problems are expected to be solved
Determine, and the quality of indexing means directly affects car record queries performance.Now frequently with based on building B-on MapReduce
The mutation indexes such as Tree or R-Tree go to realize space-time unique and class inquiry event two.
R.G.Y et al. uses at its paper delivered and goes to build time index, such as TimeIndex side based on B-Tree mutation
Method, the method computation complexity is O (n2), the TimeIndex+ that V.I.V.R.S et al. proposes in its paper delivered subsequently
Method, although performance increases, but need to optimize；
A.M et al. proposes SpatialHadoop in its paper delivered, and uses and goes to process based on MapReduce framework
Spatial data.
J.R et al. proposes Parallel-Secondo in its paper delivered, and uses parallel spatial relational database, this
Data base manipulation Hadoop is as distributed task dispatching device.
C.G.S.S et al. proposes distributed segmentation tree in its paper delivered, and can support model based on P2P DHT network
Enclose inquiry, but P2P DHT infrastructure is different from the distributed cloud platform that this patent proposes.
In summary, existing space-time unique and space-time single-point cross car record quick search mostly use based on Hadoop put down
Platform builds B-Tree and R-Tree mutation index, takes full advantage of Hadoop distributed variable-frequencypump and key value database is efficiently located
Reason characteristic, but magnanimity is crossed car record and is inquired about the index such as the mutation that uses B-Tree or R-Tree based on space-time unique and space-time single-point
Differ and be set to suitable technical scheme.
Summary of the invention
It is an object of the invention to the shortcoming overcoming above-mentioned prior art, it is provided that one is capable of solving space-time unique
In the space-time unique based on key assignments cloud storage of quick search and space-time single-point quick search two class problem, to cross car record fast for magnanimity
Speed querying method.
To achieve these goals, the present invention has a following composition:
In being somebody's turn to do space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying, and it is mainly characterized by, institute
The method stated comprises the steps:
(1) magnanimity is crossed car record data to import in key value database；
(2) according to crossing car record time and space granule, time and the key assignments concordance list of space segment point are built；
(3) on the basis of the key assignments concordance list of time and space segment point, time and the key assignments of space segment tree are built
Concordance list；
(4) when the key assignments concordance list of reconstitution time and space segment tree, structure time and space segment tree update and delete
Except key assignments concordance list.
It is preferred that build the key assignments concordance list of time slice point, comprise the following steps:
(2-A-1) time range belonging to the car record time was determined；
(2-A-2) based on time granularity size, time range is carried out segmentation, obtain time slice interval；
(2-A-3) initial value and stop value to time slice interval set up key assignments respectively；
(2-A-4) MapReduce or Spark framework is utilized to cross car record by belonging in each time slice interval
Line unit gathering is stored in corresponding train value, forms the key assignments concordance list of time slice point.
It is preferred that build the key assignments concordance list of space segment point, comprise the following steps:
(2-B-1) spatial dimension belonging to traffic network vehicle monitoring point is determined；
(2-B-2) based on space granularity size, spatial dimension is carried out segmentation, obtain space segment interval；
(2-B-3) initial value and stop value to space segment interval set up key assignments respectively；
(2-B-4) MapReduce or Spark framework is utilized to cross car record by belonging in each space segment interval
Line unit gathering is stored in corresponding train value, forms the key assignments concordance list of space segment point.
More preferably, build the key assignments concordance list of time slice tree, comprise the following steps:
(3-A-1) key assignments concordance list based on time slice point, the time slice that each two is adjacent builds one tree, uses
Bottom-up method, utilizes MapReduce or Spark framework, forms a complete binary tree only containing a root node；
(3-A-2) identify all nodes of binary tree, formed and have belonging to the key assignments of time slice point identification value, waypoint
Initial value and the time of the line unit of stop value, corresponding left subtree and right subtree ident value and mistake car record that time slice is interval divide
Section tree concordance list.
It is preferred that build the key assignments concordance list of space segment tree, comprise the following steps:
(3-B-1) key assignments concordance list based on space segment point, the space segment that each two is adjacent builds one tree, uses
Bottom-up method, utilizes MapReduce or Spark framework, forms a complete binary tree only containing a root node；
(3-B-2) identify all nodes of binary tree, formed and have belonging to the key assignments of space segment point identification value, waypoint
Initial value and the space of the line unit of stop value, corresponding left subtree and right subtree ident value and mistake car record that space segment is interval are divided
Section tree concordance list.
Update it is preferred that build time slice tree and delete key assignments concordance list, comprising the following steps:
(4-A-1) key assignments concordance list based on time slice point, is stored in one by car record of crossing that is newly inserted and that delete
Individually in row, form time slice tree and update and delete key assignments concordance list；
(4-A-2) when needing the key assignments concordance list rebuilding time slice tree, update from time slice tree and delete
The newly inserted car record excessively with deletion of quick obtaining in key assignments concordance list.
Update it is preferred that build space segment tree and delete key assignments concordance list, comprising the following steps:
(4-B-1) key assignments concordance list based on space segment point, is stored in one by car record of crossing that is newly inserted and that delete
Individually in row, form space segment tree and update and delete key assignments concordance list；
(4-B-2) when needing the key assignments concordance list rebuilding space segment tree, update from space segment tree and delete
The newly inserted car record excessively with deletion of quick obtaining in key assignments concordance list.
In have employed the space-time unique based on key assignments cloud storage in this invention, magnanimity crosses car record method for quickly querying, tool
Have the advantages that:
(1) by introducing space-time waypoint concordance list, it is greatly improved car record quick search of crossing based on space-time single-point and imitates
Rate；
(2) by introducing space-time segmentation tree concordance list, it is greatly improved car record quick search of crossing based on space-time unique and imitates
Rate；
(3) by introducing space-time segmentation tree renewal concordance list, dynamic reconstruct space-time segmentation tree concordance list, for supporting magnanimity
Flow through car record real-time query provide may；
(4) preserve above-mentioned concordance list by introducing key assignments cloud storage data base, and it is above-mentioned to utilize Hadoop cloud platform to generate
Concordance list, substantially reduces the time building and safeguarding index, and improves query processing performance.
Accompanying drawing explanation
Fig. 1 is the flow chart of the method for the time slice point index construct of the present invention.
Fig. 2 is the flow chart of the method for the space segment point index construct of the present invention.
Fig. 3 is the flow chart of the method for the time slice tree index construct of the present invention.
Fig. 4 is the flow chart of the method for the space segment tree index construct of the present invention.
Fig. 5 is the flow chart of the method for the time slice tree renewal index construct of the present invention.
Fig. 6 is the flow chart of the method for the space segment tree renewal index construct of the present invention.
Detailed description of the invention
In order to more clearly describe the technology contents of the present invention, carry out further below in conjunction with specific embodiment
Describe.
The space-time unique quick search problem that the present invention solves relates to time and space segment point index construct algorithm；Space-time
Single-point quick search problem relates to the renewal index of time and space segment tree index construct algorithm and correspondence and deletes index structure
Build algorithm.First magnanimity is crossed in key-value (key assignments) data base that car record imports to Nosql；Then utilize
MapReduce or Spark framework, according to crossing car record time and space granule, builds time and the key-of space segment point
Value concordance list；Afterwards on the basis of the key-value concordance list of time and space segment point, structure time and space are divided
Section tree key-value concordance list, simultaneously during reconstitution time and space segment tree key-value concordance list, build the time with
Space segment tree updates and deletes key-value concordance list.
In space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying, including following algorithm: 1) time
Between waypoint index construct algorithm；2) space segment point index construct algorithm；3) time slice tree index construct algorithm；4) space
Segmentation tree index construct algorithm；5) time slice tree updates Index Algorithm；6) space segment tree updates Index Algorithm.Above-mentioned algorithm
Import in key value database assuming that magnanimity crosses car record.Magnanimity is crossed car log and is shown as R (C, S, T).Wherein C represented that car was remembered
Record major key mark rowkey (line unit), S representation space scope, T express time scope.S=(Si, Sj ... Sn) representation space model
Enclose by traffic network vehicle monitoring point numbering (i, j ... n) composition.T=(Ta, Tb) (a≤b) express time scope is by time point
A, to the scope of time b, works as a=b, becomes time single-point.Time slice granularity of division with real needs be as the criterion (with sky, hour,
Minute etc. thickness granularity), it is assumed that time interval is expressed as timespan；Space segment granularity of division is distributed close dilute journey with control point
Degree is the criteria for classifying (example: with 500 meters × 500 meters, 1000 meters × 1000 meters, the thickness granularities such as 2000 meters × 2000 meters are as the criterion),
Assume that space interval is expressed as spacespan.The present invention as a example by key value database hbase, index creation platform with
As a example by MapReduce, algorithm Constructed wetlands is explained.
1. time slice point index construct algorithm:
1) supposed that the time range belonging to the car record time was [Tbegin,Tend], then time slice interval is [Tbegin,
Tbegin+timespan],[Tbegin+timespan,Tbegin+2×timespan],[Tbegin+2×timespan,Tbegin+3×
timespan],...[Tbegin+(n-1)×timespan,Tend]。
2) hbase data base is crossed car log create the time row bunch under as Time begin and end row, make importing
Any begin and end train value crossing car record is above-mentioned 1) in the initial value of some piecewise interval and stop value.
3) utilize MapReduce framework, any bar in data base is crossed begin and the end train value belonging to car record, shape
Become with the interval initial value (and stop value) of time slice as key, all rowkey crossing car records in this interval affiliated and right
The key-value time slice point concordance list that stop value (and initial value) is value that the time slice answered is interval.
2. space segment point index construct algorithm:
1) suppose that the spatial dimension belonging to traffic network vehicle monitoring point is [Sstartpos,Sendpos], then space segment is interval
For [Sstartpos, Sstartpos+spacespan],[Sstartpos+spacespan,Sstartpos+2×spacespan],[Sstartpos+
2×spacespan,Sstartpos+3×spacespan],..[Sstartpos+(n-1)×spacespan,Sendpos]。
2) hbase data base is crossed car log create space row bunch under as Space startpos and endpos row, make
Any startpos and the endpos train value crossing car record imported is above-mentioned 1) in the initial value of some piecewise interval
And stop value.
3) utilize MapReduce framework, any bar in data base is crossed startpos and endpos belonging to car record
Train value, is formed with the interval initial value (and stop value) of space segment as key, and this interval affiliated all cross car records
The key-value space segment point that stop value (and initial value) the is value index in rowkey and corresponding space segment interval
Table.
2. time slice tree index construct algorithm:
1) on the basis of having built time slice point concordance list, arranging sequentially in time, every 2 adjacent time divide
Section point forms 1 binary tree, and the time slice point identification value of left subtree is the intermediate value in corresponding time slice region, right subtree
Time slice point identification value is also the intermediate value in corresponding time slice region, and the time slice point identification value of father's node is corresponding
The intermediate value of left and right subtree ident value.
2) according to above-mentioned rule, utilize MapReduce framework, be recorded as defeated with in time slice point concordance list each
Enter, formed with each time slice point identification value as key, the initial value (begin) in this interval affiliated and stop value (end),
Corresponding left subtree and right subtree ident value and correspondence cross the key-value time slice tree that rowkey is value of car record
Concordance list.
3) contribute by bottom-up circulation, eventually form 1 time slice tree concordance list only containing 1 root node.
3. space segment tree index construct algorithm:
1) on the basis of having built space segment point concordance list, according to Spacial domain decomposition size, affiliated every 2
Space segment point forms 1 binary tree, and the space segment point identification value of left subtree is corresponding space segment regional center coordinate figure,
The space segment point identification value of right subtree is also corresponding space segment regional center coordinate figure, the space segment point mark of father's node
Knowledge value is the centre coordinate value of corresponding left and right subtree ident value.
2) according to above-mentioned rule, utilize MapReduce framework, be recorded as defeated with in space segment point concordance list each
Enter, formed with each space segment point identification value as key, the initial value (startpos) in this interval affiliated and stop value
(endpos) key-value that rowkey is value that, corresponding left subtree and right subtree ident value and correspondence cross car record is empty
Between segmentation tree concordance list.
3) contribute by bottom-up circulation, eventually form 1 space segment tree concordance list only containing 1 root node.
4. time slice tree renewal Index Algorithm:
1) on the basis of building time slice point concordance list, increasing corresponding arranging as Update arranges bunch lower UI, UI arranges use
In recording car record excessively that is newly inserted and that delete.
2) car record excessively that is newly inserted and that delete, is saved under the correspondence UI row of affiliated time slice point interval.
5. space segment tree renewal Index Algorithm:
1) on the basis of building space segment point concordance list, increasing corresponding arranging as Update arranges bunch lower UI, UI arranges use
In recording car record excessively that is newly inserted and that delete.
2) traffic network vehicle monitoring point that is newly inserted and that delete, is saved in affiliated space segment point interval correspondence UI row
Under.
Enforcement to the present invention elaborates below in conjunction with the accompanying drawings.Implement step as follows:
Above-mentioned algorithm supposes that magnanimity is crossed car record and imported in key value database.Magnanimity cross car log be shown as R (C, S,
T).Wherein C represented that car record major key identified rowkey, S representation space scope, T express time scope.S=(Si, Sj,
... Sn) representation space scope by traffic network vehicle monitoring point number (i, j ... n) composition.T=(Ta, Tb) (a≤b) represents
Time range to the scope of time b, is worked as a=b by time point a, becomes time single-point.Time slice granularity of division is with real needs
Be as the criterion (with sky, hour, minute etc. thickness granularity), it is assumed that time interval is expressed as timespan；Space segment granularity of division with
Control point be distributed close dilute degree be the criteria for classifying (example with 500 meters × 500 meters, 1000 meters × 1000 meters, 2000 meters × 2000 meters etc.
Thickness granularity), it is assumed that space interval is expressed as spacespan.Patent of the present invention is as a example by key value database hbase, and index is created
Algorithm Constructed wetlands is explained as a example by MapReduce by Jianping platform.
1, time slice point concordance list:
In time slice point concordance list, determined the time range (101) belonging to the car record time, then according to the time
Granular size, determines time interval timespan (with sky, hour, minute be unit), and it is interval (102) to divide time slice,
It is 2016-01-01 00:00:00 to 2016-04-30 00:00:00 as crossed car record time range, with 1 day as time interval,
Then time slice interval is [20160101000000,20160102000000], [20160102000000,
20160103000000],[20160103000000,20160104000000]...[20160429000000,
20160430000000]. be then each cross car record create begin and end row, as former cross car log be shown as R (C,
S, T), then adding the representation after row is R (C, S, T, begin, end), and wherein begin and end value is affiliated time slice
Interval initial value and stop value (103).Utilize MapReduce framework, read in each and cross car record begin and end value, shape
Becoming the Map stage is ＜ begin, " r " | rowkey=end ＞, ＜ end, " l " | and the output form of rowkey=begin ＞, pass through
Collecting of Reduce stage forms ＜ begin, " r " | rowkey1 | rowkey2 | rowkey3 | ... rowkeyk |=end ＞,
＜ end, " l " | rowkey1 | rowkey2 | rowkey3 | ... rowkeyk | time slice point concordance list (104) of=begin ＞.
Wherein rowkey1, rowkey2 ... rowkeyk was car record time car record major key excessively in [begin, end] section,
" l " and " r " and termination identifier initial for the time period.Through too much taking turns the calculation process of MapReduce, form time slice point rope
Draw table 105.
2, space segment point concordance list:
In space segment point concordance list, determine the spatial dimension (201) belonging to traffic network vehicle monitoring point, then root
According to space granular size, determine that space interval is expressed as spacespan, and it is interval (202) to divide space segment, as crossed car record
Belonging to traffic network vehicle monitoring point S=(Si, Sj ... Sn), representation space scope by traffic network vehicle monitoring point number
(i, j ... n) composition.Spatial dimension belonging to above-mentioned vehicle monitoring point is that upper left point coordinates is (0,0), lower-right most point coordinate
For (200000,200000), with 1000 meters × 1000 meters as space interval, forming 1 to 200 space segment intervals is [0,1],
[1,2],[2,3]...[199,200].Then it is each and crosses car record establishment startpos and endpos row, as crossed car former
Log is shown as R (C, S, T), then adding the representation after row is R (C, S, T, startpos, endpos), wherein
Startpos and endpos value is initial value and the stop value (203) in affiliated space segment interval. utilize MapReduce framework,
Reading in each and cross car record startpos and endpos value, forming the Map stage is ＜ startpos, " e " | rowkey=
Endpos ＞, ＜ endpos, " s " | the output form of rowkey=startpos ＞, collects formation through the Reduce stage
＜ startpos, " e " | rowkey1 | rowkey2 | rowkey3 | ... rowkeyk |=endpos ＞, ＜ endpos, " s " |
Rowkey1 | rowkey2 | rowkey3 | ... rowkeyk | space segment point concordance list (204) of=startpos ＞.Wherein
Rowkey1, rowkey2 ... rowkeyk was that the space belonging to car record control point is in [startpos, endpos] section
Crossing car record major key, " s " and " e " is initial for space segment and terminates identifier.Through too much taking turns the calculation process of MapReduce, shape
Become space segment point concordance list (205).
3, time slice tree concordance list
On the basis of building time slice point concordance list (105), to cross car record time range as 2016-01-
01 00:00:00 to 2016-04-30 00:00:00, with 1 day as time interval as a example by, then time slice point concordance list is a length of
121, the most adjacent 2 time slice points form the father node of a stalk binary tree, by taking turns iteration more, form one and contain only one
The complete binary tree (302) of individual root node, height of this tree is 7 (more than or equal to (log2121)) layer.Then in mark tree
Each node, from leaf node mark, bottom-up, eventually form time slice tree concordance list (303). node identification is such as
Under: as a example by first leaf node, take the intermediate value in [20160101000000,20160102000000]
20160101500000L is the value l=20160101000000 ∪ r=20160102000000 ∪ L=of key, value " ∪ R
=" ∪ rowkey1 | rowkey2 | rowkey3 | ... | rowkeyk.Key key wherein comprises " L " and is expressed as leaf node, " R "
For root node, it it is otherwise intermediate node；The starting point that identifier " l " is time slice, the terminal that " r " is time slice, " L " is left
The ident value of child nodes, " R " is the ident value of right child nodes, and rowkey1, rowkey2, rowkey3...rowkeyk are
The vehicle pass-through time belongs to the vehicle of this time segmentation and crosses car record major key.
4, space segment tree concordance list
On the basis of building space segment point concordance list (205), with spatial dimension upper left point coordinates for (0,
0), lower-right most point coordinate is (200000,200000), and 1000 meters × 1000 meters is space interval, then space segment point concordance list
A length of 200, the most adjacent 2 space segment points form the father node of a stalk binary tree, by taking turns iteration, only form one more
Complete binary tree (402) containing a root node, the height of this tree is 8, then each node in mark tree, from leaf
Child node identifies, bottom-up, eventually forms time slice tree concordance list (403). and node identification is as follows: with first leaf
As a example by node, take the value l=0 ∪ r=1 ∪ L=that intermediate value 0.5L is key, value in [0,1] " ∪ R=" ∪ rowkey1 |
Rowkey2 | rowkey3 | ... | comprising " L " in rowkeyk. wherein key key and be expressed as leaf node, " R " is root node, otherwise
For intermediate node；The starting point that identifier " l " is space segment, the terminal that " r " is space segment, " L " is the mark of left child nodes
Knowledge value, " R " is the ident value of right child nodes, and rowkey1, rowkey2, rowkey3...rowkeyk are vehicle pass-through monitoring
Point belongs to the vehicle of this space segment and crosses car record major key.
5, time slice tree updates concordance list
On the basis of building time slice point concordance list (105), increase update row bunch lower UI row, and record new
That inserts and delete crosses car record (501), newly inserted cross car record with containing "+" represents, deletion cross car record containing "-" table
Showing, as a example by being 2016-01-01 08:23:20 major key m such as the newly inserted car record time excessively, the key of UI train value is
20160101500000L, rm+=20160101000000；It is 2016-01-01 11:23:20 master as deleted the car record time
As a example by key n, the key of UI train value is 20160101500000L, rn-=20160101000000.
6, space segment tree updates concordance list
On the basis of building space segment point concordance list (205), increase update row bunch lower UI row, and record new
That inserts and delete crosses car record (601), newly inserted cross car record with containing "+" represents, deletion cross car record containing "-" table
Show, if newly inserted car record of crossing is as a example by control point spatial dimension belongs to space segment point [0,1] major key n, the key of UI train value
For 0.5L, rn+=1；As deleted car record as a example by control point spatial dimension belongs to space segment point [0,1] major key k, UI
The key of train value is 0.5L, rk-=1.
In have employed the space-time unique based on key assignments cloud storage in this invention, magnanimity crosses car record method for quickly querying, tool
Have the advantages that:
(1) by introducing space-time waypoint concordance list, it is greatly improved car record quick search of crossing based on space-time single-point and imitates
Rate；
(2) by introducing space-time segmentation tree concordance list, it is greatly improved car record quick search of crossing based on space-time unique and imitates
Rate；
(3) by introducing space-time segmentation tree renewal concordance list, dynamic reconstruct space-time segmentation tree concordance list, for supporting magnanimity
Flow through car record real-time query provide may；
(4) preserve above-mentioned concordance list by introducing key assignments cloud storage data base, and it is above-mentioned to utilize Hadoop cloud platform to generate
Concordance list, substantially reduces the time building and safeguarding index, and improves query processing performance.
In this description, the present invention is described with reference to its specific embodiment.But it is clear that still may be made that
Various modifications and alterations are without departing from the spirit and scope of the present invention.Therefore, specification and drawings is considered as illustrative
And it is nonrestrictive.
Claims (7)
1. in a space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying, it is characterised in that described
Method comprise the steps:
(1) magnanimity is crossed car record data to import in key value database；
(2) according to crossing car record time and space granule, time and the key assignments concordance list of space segment point are built；
(3) on the basis of the key assignments concordance list of time and space segment point, the key assignments index of time and space segment tree is built
Table；
(4) when the key assignments concordance list of reconstitution time and space segment tree, build the time and space segment tree updates and deletes key
Value concordance list.
In space-time unique based on key assignments cloud storage the most according to claim 1, magnanimity crosses car record method for quickly querying,
It is characterized in that, build the key assignments concordance list of time slice point, comprise the following steps:
(2-A-1) time range belonging to the car record time was determined；
(2-A-2) based on time granularity size, time range is carried out segmentation, obtain time slice interval；
(2-A-3) initial value and stop value to time slice interval set up key assignments respectively；
(2-A-4) MapReduce or Spark framework is utilized will to belong to the line unit crossing car record in each time slice interval
Assemble and be stored in corresponding train value, form the key assignments concordance list of time slice point.
In space-time unique based on key assignments cloud storage the most according to claim 1, magnanimity crosses car record method for quickly querying,
It is characterized in that, build the key assignments concordance list of space segment point, comprise the following steps:
(2-B-1) spatial dimension belonging to traffic network vehicle monitoring point is determined；
(2-B-2) based on space granularity size, spatial dimension is carried out segmentation, obtain space segment interval；
(2-B-3) initial value and stop value to space segment interval set up key assignments respectively；
(2-B-4) MapReduce or Spark framework is utilized will to belong to the line unit crossing car record in each space segment interval
Assemble and be stored in corresponding train value, form the key assignments concordance list of space segment point.
In space-time unique based on key assignments cloud storage the most according to claim 2, magnanimity crosses car record method for quickly querying,
It is characterized in that, build the key assignments concordance list of time slice tree, comprise the following steps:
(3-A-1) key assignments concordance list based on time slice point, the time slice that each two is adjacent builds one tree, uses the end of from
Method upwards, utilizes MapReduce or Spark framework, forms a complete binary tree only containing a root node；
(3-A-2) identify all nodes of binary tree, formed and there is the time belonging to the key assignments of time slice point identification value, waypoint
The time slice tree of the line unit of the initial value of piecewise interval and stop value, corresponding left subtree and right subtree ident value and mistake car record
Concordance list.
In space-time unique based on key assignments cloud storage the most according to claim 1, magnanimity crosses car record method for quickly querying,
It is characterized in that, build the key assignments concordance list of space segment tree, comprise the following steps:
(3-B-1) key assignments concordance list based on space segment point, the space segment that each two is adjacent builds one tree, uses the end of from
Method upwards, utilizes MapReduce or Spark framework, forms a complete binary tree only containing a root node；
(3-B-2) identify all nodes of binary tree, formed and there is space belonging to the key assignments of space segment point identification value, waypoint
The space segment tree of the line unit of the initial value of piecewise interval and stop value, corresponding left subtree and right subtree ident value and mistake car record
Concordance list.
In space-time unique based on key assignments cloud storage the most according to claim 1, magnanimity crosses car record method for quickly querying,
It is characterized in that, build time slice tree and update and delete key assignments concordance list, comprise the following steps:
(4-A-1) key assignments concordance list based on time slice point, is stored in one individually by car record of crossing that is newly inserted and that delete
Row in, formed time slice tree update and deletion key assignments concordance list；
(4-A-2) when needing the key assignments concordance list rebuilding time slice tree, update from time slice tree and delete key assignments
The newly inserted car record excessively with deletion of quick obtaining in concordance list.
In space-time unique based on key assignments cloud storage the most according to claim 1, magnanimity crosses car record method for quickly querying,
It is characterized in that, build space segment tree and update and delete key assignments concordance list, comprise the following steps:
(4-B-1) key assignments concordance list based on space segment point, is stored in one individually by car record of crossing that is newly inserted and that delete
Row in, formed space segment tree update and deletion key assignments concordance list；
(4-B-2) when needing the key assignments concordance list rebuilding space segment tree, update from space segment tree and delete key assignments
The newly inserted car record excessively with deletion of quick obtaining in concordance list.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610423093.8A CN106095952A (en) | 2016-06-15 | 2016-06-15 | In space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610423093.8A CN106095952A (en) | 2016-06-15 | 2016-06-15 | In space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106095952A true CN106095952A (en) | 2016-11-09 |
Family
ID=57846024
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610423093.8A Pending CN106095952A (en) | 2016-06-15 | 2016-06-15 | In space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106095952A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106777003A (en) * | 2016-12-07 | 2017-05-31 | 安徽大学 | A kind of search index method and system towards Key Value storage systems |
| CN108595720A (en) * | 2018-07-12 | 2018-09-28 | 中国科学院深圳先进技术研究院 | A kind of block chain spatiotemporal data warehouse method, system and electronic equipment |
| CN109241126A (en) * | 2018-06-29 | 2019-01-18 | 武汉理工大学 | A kind of space-time trajectory accumulation mode mining algorithm based on R* tree index |
| CN109542912A (en) * | 2018-12-04 | 2019-03-29 | 北京锐安科技有限公司 | Interval censored data storage method, device, server and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102955843A (en) * | 2012-09-20 | 2013-03-06 | 北大方正集团有限公司 | Method for realizing multi-key finding of key value database |
| CN104750708A (en) * | 2013-12-27 | 2015-07-01 | 华为技术有限公司 | Spatio-temporal data index building and searching methods, a spatio-temporal data index building and searching device and spatio-temporal data index building and searching equipment |
| CN104978334A (en) * | 2014-04-04 | 2015-10-14 | 华为技术有限公司 | Processing method and device of spatiotemporal behavior data |
| CN105426491A (en) * | 2015-11-23 | 2016-03-23 | 武汉大学 | Space-time geographic big data retrieval method and system |
-
2016
- 2016-06-15 CN CN201610423093.8A patent/CN106095952A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102955843A (en) * | 2012-09-20 | 2013-03-06 | 北大方正集团有限公司 | Method for realizing multi-key finding of key value database |
| CN104750708A (en) * | 2013-12-27 | 2015-07-01 | 华为技术有限公司 | Spatio-temporal data index building and searching methods, a spatio-temporal data index building and searching device and spatio-temporal data index building and searching equipment |
| CN104978334A (en) * | 2014-04-04 | 2015-10-14 | 华为技术有限公司 | Processing method and device of spatiotemporal behavior data |
| CN105426491A (en) * | 2015-11-23 | 2016-03-23 | 武汉大学 | Space-time geographic big data retrieval method and system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106777003A (en) * | 2016-12-07 | 2017-05-31 | 安徽大学 | A kind of search index method and system towards Key Value storage systems |
| CN106777003B (en) * | 2016-12-07 | 2020-04-03 | 安徽大学 | Key-Value storage system oriented index query method and system |
| CN109241126A (en) * | 2018-06-29 | 2019-01-18 | 武汉理工大学 | A kind of space-time trajectory accumulation mode mining algorithm based on R* tree index |
| CN109241126B (en) * | 2018-06-29 | 2021-09-14 | 武汉理工大学 | Spatio-temporal trajectory aggregation mode mining algorithm based on R-tree index |
| CN108595720A (en) * | 2018-07-12 | 2018-09-28 | 中国科学院深圳先进技术研究院 | A kind of block chain spatiotemporal data warehouse method, system and electronic equipment |
| CN109542912A (en) * | 2018-12-04 | 2019-03-29 | 北京锐安科技有限公司 | Interval censored data storage method, device, server and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100523735C (en) | Fast map matching method based on small lattice road network organization and structure | |
| CN103020204B (en) | A kind of method and its system carrying out multi-dimensional interval query to distributed sequence list | |
| Abuhashim et al. | Smart contract designs on blockchain applications | |
| CN103412897B (en) | A kind of parallel data processing method based on distributed frame | |
| CN106095952A (en) | In space-time unique based on key assignments cloud storage, magnanimity crosses car record method for quickly querying | |
| CN105956015A (en) | Service platform integration method based on big data | |
| CN106407208B (en) | A kind of construction method and system of city management ontology knowledge base | |
| CN104346444B (en) | A kind of the best site selection method based on the anti-spatial key inquiry of road network | |
| CN106790468A (en) | A kind of distributed implementation method for analyzing user's WiFi event trace rules | |
| CN102521364B (en) | Method for inquiring shortest path between two points on map | |
| CN105205247B (en) | A kind of emulation road net data management method based on tree construction | |
| CN107247799A (en) | Data processing method, system and its modeling method of compatible a variety of big data storages | |
| CN106599040A (en) | Layered indexing method and search method for cloud storage | |
| CN104574965B (en) | A kind of urban transportation hot spot region based on magnanimity traffic flow data division methods | |
| CN106528793A (en) | Spatial-temporal fragment storage method for distributed spatial database | |
| CN103699648A (en) | Tree-form data structure used for quick retrieval and implementation method of tree-form data structure | |
| CN106503214A (en) | A kind of complex rule matching process based on Redis memory databases | |
| CN106777163A (en) | IP address institute possession querying method and system based on RBTree | |
| CN107870949A (en) | Data analysis job dependence relation generation method and system | |
| CN104615734B (en) | A kind of community management service big data processing system and its processing method | |
| CN108280159A (en) | A method of converting chart database to relational database | |
| CN104156635B (en) | The OPSM method for digging of the gene chip expression data based on common subsequence | |
| CN104050291B (en) | A kind of method for parallel processing and system of account balance data | |
| CN105912665A (en) | Method for model conversion and data migration of Neo4j to relational database | |
| CN108009265A (en) | A kind of space data index method under cloud computing environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161109 |