CN104182409B - A kind of method and device optimized to multistage Hash - Google Patents
A kind of method and device optimized to multistage Hash Download PDFInfo
- Publication number
- CN104182409B CN104182409B CN201310196974.7A CN201310196974A CN104182409B CN 104182409 B CN104182409 B CN 104182409B CN 201310196974 A CN201310196974 A CN 201310196974A CN 104182409 B CN104182409 B CN 104182409B
- Authority
- CN
- China
- Prior art keywords
- data
- colliding
- successful
- colliding data
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000001514 detection method Methods 0.000 claims abstract description 90
- 238000003780 insertion Methods 0.000 claims description 20
- 230000037431 insertion Effects 0.000 claims description 20
- 241001269238 Data Species 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 11
- 238000010586 diagram Methods 0.000 description 18
- 241000544061 Cuculus canorus Species 0.000 description 6
- 230000000052 comparative effect Effects 0.000 description 6
- 230000006872 improvement Effects 0.000 description 6
- 230000008520 organization Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 239000000686 essence Substances 0.000 description 1
- 238000001125 extrusion Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000004080 punching Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention proposes a kind of method and device optimized to multistage Hash, and wherein method includes：When inserting new data to multistage Hash, if this, which is inserted into all addressed locations of data, has preserved colliding data, the colliding data in all addressed locations is detected to higher order；If there is the successful colliding data of detection, the minimum colliding data of exponent number for detecting successful location then is transferred into the successful position of the detection to be preserved, and by the initial position for being inserted into data and being stored in the minimum colliding data of exponent number of the detection successful location.The present invention can improve the filling rate of multistage Hash.
Description
Technical field
The present invention relates to multistage salted hash Salted field, more particularly to a kind of method optimized to multistage Hash and dress
Put.
Background technology
Multistage Hash is a kind of outstanding data structure, its have high-performance, can reentry, stably, healthy and strong, perform stream peace more
Entirely, can be from the advantages that eliminating.The implementation process of multistage Hash is as follows：
N number of Big prime is preset, each Big prime is as the length per single order Hash bucket；
When needing to store new data, using the length remainder of the current rank Hash bucket of the key-value pair of the new data, sought
Location position, if having preserved colliding data in the addressed location, downward single order carries out Hash calculation；Otherwise, it is this is new
Data be stored in the addressed location；
During to multistage Hash data storage, it is maximum can conflict number be multistage Hash exponent number.
Such as the storage organization schematic diagram that Fig. 1 is a multistage Hash in the prior art.The exponent number of the multistage Hash is 6 ranks,
Exponent number from top to bottom increases successively.Include multiple storage locations per single order Hash bucket, its bend lattice represents to have preserved punching
The position (using position) of prominent data, blank cell are represented without the position (i.e. available position) for preserving colliding data.
When inserting new data A to the multistage Hash, address bits of the A in the 1st rank is calculated according to A key assignments (being designated as KeyA)
Put, find the addressed location to use position, then continue to address to the 2nd rank；Addressed locations of the A in the 2nd rank is calculated according to KeyA,
It was found that the addressed location then continues to address to the 3rd rank to use position；Addressed locations of the A in the 3rd rank, hair are calculated according to KeyA
Now the addressed location is available position, then is stored in A in the available position of the 3rd rank.If highest is addressed to always to A
Rank, it is found that the addressed location of most high-order still to use position, then judges the insertion failure to data A.
From above procedure, existing hash algorithm to the new data that inserts of needs when addressing, if per single order
Addressed location is unavailable, then directly judges the insertion failure to new data；However, it may now be remained in multistage Hash in sky
Not busy position does not use, and the filling rate that this results in existing multistage Hash is low.
The content of the invention
The present invention proposes the method and device that a kind of multistage Hash optimizes, it is possible to increase the filling rate of multistage Hash.
The technical proposal of the invention is realized in this way：
A kind of method optimized to multistage Hash, including：
When inserting new data to multistage Hash, if this, which is inserted into all addressed locations of data, has preserved number of collisions
According to then the colliding data in all addressed locations is detected to higher order；
If there is successful colliding data is detected, then the minimum colliding data of the exponent number for detecting successful location is transferred to
The successful position of the detection is preserved, and by it is described be inserted into data and be stored in the exponent number of the detection successful location minimum rush
The initial position of prominent data.
When inserting new data to multistage Hash, if this, which is inserted into all addressed locations of data, has preserved number of collisions
According to then the colliding data in all addressed locations is detected to higher order；
If there is successful colliding data is detected, then the minimum colliding data of the exponent number for detecting successful location is transferred to
The successful position of the detection is preserved, and by it is described be inserted into data and be stored in the exponent number of the detection successful location minimum rush
The initial position of prominent data.
A kind of device optimized to multistage Hash, including：
Detecting module, for when inserting new data to multistage Hash, if this is inserted into all addressed locations of data
On preserved colliding data, then the colliding data in all addressed locations is detected to higher order；
Preserving module, for when exist detect successful colliding data when, the exponent number for detecting successful location minimum is rushed
Prominent data are transferred to the successful position of the detection and preserved, and the new data are stored in the rank of the detection successful location
The initial position of the minimum colliding data of number.
It can be seen that the method and apparatus proposed by the present invention optimized to multistage Hash, in the new data inserted to needs
During addressing, if each addressed location is unavailable, judge whether colliding data in its all addressed location can be to
Higher order shifts, if it can, then shifting the colliding data to higher order, and the new data that are inserted into is inserted into the colliding data
Initial position.In this way, it is possible to increase the filling rate of multistage Hash.
Brief description of the drawings
Fig. 1 is the storage organization schematic diagram of a multistage Hash in the prior art；
Fig. 2 is the method flow diagram proposed by the present invention optimized to multistage Hash；
Fig. 3 is the schematic diagram 1 of the embodiment of the present invention one；
Fig. 4 is the schematic diagram 2 of the embodiment of the present invention one；
Fig. 5 is the schematic diagram 3 of the embodiment of the present invention one；
Fig. 6 is the schematic diagram 4 of the embodiment of the present invention one；
Fig. 7 is the schematic diagram 1 of the embodiment of the present invention two；
Fig. 8 is the schematic diagram 2 of the embodiment of the present invention two；
Fig. 9 is the schematic diagram 3 of the embodiment of the present invention two；
Figure 10 is to after existing multistage Hash, the multistage Hash proposed by the present invention based on detection, further improvement
The filling rate comparative result schematic diagram of class Cuckoo Hash based on detection；
Figure 11 is to after existing multistage Hash, the multistage Hash proposed by the present invention based on detection, further improvement
Class Cuckoo Hash based on detection writes efficiency (per second) comparative result schematic diagram；
Figure 12 is to after existing multistage Hash, the multistage Hash proposed by the present invention based on detection, further improvement
Reading efficiency (per second) comparative result schematic diagram of class Cuckoo Hash based on detection；
Figure 13 be to data scale 2700W 20 ranks, 5 ranks that commonly multistage Hash, data scale are 700W be based on detection
Class Cuckoo Hash and data scale be 2700W class Cuckoo Hash of 5 ranks based on detection filling rate comparative result
Schematic diagram；
Figure 14 be to data scale 2700W 20 ranks, 5 ranks that commonly multistage Hash, data scale are 700W be based on detection
Class Cuckoo Hash and data scale be 2700W class Cuckoo Hash of 5 ranks based on detection the ratio for writing efficiency (per second)
Compared with result schematic diagram；
Figure 15 be to data scale 2700W 20 ranks, 5 ranks that commonly multistage Hash, data scale are 700W be based on detection
Class Cuckoo Hash and data scale be 2700W class Cuckoo Hash of 5 ranks based on detection reading efficiency (per second) ratio
Compared with result schematic diagram.
Embodiment
The present invention proposes a kind of method optimized to multistage Hash, if Fig. 2 is this method implementation process figure, including：
Step 201：When inserting new data to multistage Hash, protected if this is inserted into all addressed locations of data
Colliding data is deposited, then is detected the colliding data in all addressed locations to higher order；
Step 202：If there is successful colliding data is detected, then by the minimum number of collisions of the exponent number for detecting successful location
Preserved according to the successful position of the detection is transferred to, and by the exponent number for being inserted into data and being stored in the detection successful location
The initial position of minimum colliding data.
In said process, colliding data is specifically as follows to the mode that higher order is detected：According to the number of collisions
According to key assignments calculate addressed location of the colliding data in higher order, when finding idle addressed location, judge the conflict
Data snooping success, successful position is detected using the idle addressed location as the colliding data.
During specific implementation, by the colliding data in all addressed locations to higher order detected when, one can be safeguarded
Variable, the initial value of the variable add 1 for the exponent number of the multistage Hash；If colliding data detects successfully, the conflict is judged
Whether the exponent number of data snooping successful location is less than the variable, if it is less, the value of the variable is revised as into the number of days from now on
According to the exponent number of detection successful location, continue the detection to higher order colliding data；After the detection of all colliding datas finishes, such as
The value of variable described in fruit is not more than the exponent number of the multistage Hash, then judges the successful colliding data of detection be present.
That is, for the data newly inserted, address bit of the data in every single order is calculated according to its key assignments key
Put, if giving insertion per all no room of single order Hash, then every single order is produced into the value to conflict with key and goes it higher
The place trial of rank, sees if there is room and is inserted.If so, then taking out, a replaceable exponent number is minimum to be replaced,
The value replaced out is put on the position of detection again.
Specific embodiment is lifted below to be described in detail the above method.
Embodiment one：
In the present embodiment, it is necessary to which new data A is inserted into a 6 rank Hash.As Fig. 3-Fig. 6 shows the present embodiment pair
Data A insertion process.
In figure 3, the addressed location in every single order is calculated according to data A key value Key A, found in all addressed locations
Colliding data has been preserved, has been B, C, D, E, F and G respectively.
Because all addressed locations are unavailable, then the colliding data in all addressed locations is visited to higher order
Survey.As shown in Figure 4 and Figure 5.
In Fig. 4, data B is detected to higher order, i.e., calculated according to data B key value Key B in higher order
Addressed location, it is found that the addressed location on the 5th rank is idle, then it is assumed that data B detection success, and record detection success position
The exponent number 5 put.
In Figure 5, continue to be detected data C to higher order, i.e., calculated according to data C key value Key C in higher order
On addressed location, it is found that addressed location on the 3rd rank is idle, then it is assumed that data C detection success, and record the detection into
The exponent number 3 of work(position.
Afterwards, continue to be detected D, E, F, G to higher order, the addressed location in these data higher orders is occupied
With, therefore detect failure.
After detection is completed, insertion is performed to data A.As shown in fig. 6, the exponent number of the detection successful location due to data C
It is minimum, therefore data C is transferred to the successful position of the detection and preserved, and data A is stored in data C initial position.
If all data detect failure, then it is assumed that the insertion failure to data A.
In specific perform, if be addressed to most high-order in N rank Hash does not find gap insertion yet, then can safeguard
One variable minRow (initial value N+1), each data of the line number on addressing path less than minRow are entered to higher order
(for a key assignments of the i-th rank, why it can be located at the i-th rank, be the position because on its path before i ranks for row detection
Put and be necessarily occupied, therefore only detected to higher order), if detecting exponent number residing for a position is less than minRow, that
Renewal minRow is the rank, until detection terminates.After detection terminates, if minRow<=N is then replaced, otherwise it is assumed that
Insertion failure.
Above is the optimization method proposed by the present invention to multistage Hash, the multistage Hash referred to as based on detection.Due to right
The insertion of new data has carried out some filling rates for changing and improving whole multistage Hash.The present invention can also be done to this method
Further improve, the thought for the key assignments " extrusion " that will conflict in existing Cuckoo hash algorithms is used for reference, when the data newly inserted exist
The position that can be replaced is not found after once detecting, it tries all colliding datas are replaced, and rushed to what is be replaced
(thresholding for replacing the number of plies is value set in advance, is such as arranged to 3 layers, number of plies mistake for the recurrence detection of the same scheme of prominent data progress
More interrogatorys show).Method after this improvement is referred to as the class Cuckoo Hash based on detection.
Specifically, may further include after above-mentioned steps 202：
If there is no the successful colliding data of detection, then the conflict being inserted into for this in first addressed location of data
Data perform following steps：
A, data replacement conflicts data are inserted into using described；
B, data are inserted into using the colliding data as new, by being inserted into all addressed locations of data newly
Colliding data is detected to higher order；If there is successful colliding data is detected, then the exponent number of successful location will be detected most
Low colliding data is transferred to the successful position of the detection and preserved, and the new data that are inserted into are stored in into the detection
The initial position of the minimum colliding data of the exponent number of successful location, terminates current process；If there is no detection successfully conflict
Data, then step A is performed successively for the new colliding data being inserted into all addressed locations of data, until replacing the number of plies
When reaching the threshold value pre-set, successful colliding data is detected if be still not present, by all number of collisions being replaced
Preserved according to its initial position is returned, and the colliding data being inserted into for this in the next addressed location of data performs step
A；
Until when being performed both by finishing to the colliding data being inserted into all addressed locations of data, visited if be still not present
Successful colliding data is surveyed, then the insertion for judging to be inserted into this data fails, and the colliding data that this is inserted into data returns
Its initial position is preserved.
Wherein, to the colliding data in all addressed locations of colliding data to the mode that higher order is detected with it is above-mentioned right
The mode that the new legacy data being inserted into all addressed locations of data is detected to higher order is identical.
Specific embodiment is lifted below to be described in detail above-mentioned improved method.
Embodiment two：
In the present embodiment, it is necessary to which new data A is inserted into a 6 rank Hash.As Fig. 7-Fig. 9 shows the present embodiment pair
Data A insertion process.In the present embodiment, illustrated so that the threshold value for replacing the number of plies is 2 layers as an example.
In the figure 7, the addressed location in every single order is calculated according to data A key value Key A, found in all addressed locations
Colliding data has been preserved, has been B, C, D, E, F and G respectively.Also, colliding data B, C, D, E, F and G detection not into
Work(, that is, addressed location of these colliding datas in higher order is occupied that (Fig. 7 only show data B detection
Journey).
Afterwards, all colliding data B, C, D, E, F and G on its addressing path are replaced using A successively, are rushed what is be replaced
Prominent data carry out Cuckoo Hash insertion operation.
As shown in figure 8, with data A replacement data B, now data B turns into new and is inserted into data, data B is carried out same
The Cuckoo Hash of sample insertion operation.The replacement number of plies now is 1 layer (i.e. A replaces B).In B all addressed locations
Colliding data is followed successively by A → H → I → J → K → L, then is detected successively with A, H, I, J, K, L value, if there is detection
Successful colliding data, then the minimum colliding data of the exponent number for detecting successful location is transferred to the successful position of the detection and carried out
Preserve, B is stored in the initial position of the colliding data.As shown in figure 9, B colliding data H detection success, and H detection into
The exponent number of work(position is minimum, then B is stored in H initial position, and H is stored in into it detects successful position.So far, to new number
According to A insertion success, terminate current process.
After if all colliding datas detection to B is completed, still it is not present and detects successful colliding data, then successively
All colliding datas on its addressing path are replaced using B, number is inserted into using the colliding data that this is replaced as new
According to carrying out Cuckoo Hash insertion operation to it.The number of plies now replaced is 2 layers, and (i.e. A replaces B, and B replaces rushing for it
Prominent data), reach threshold value set in advance.
For B colliding data H, it is assumed that the colliding data in H all addressed locations be followed successively by W → B → M → P → Q →
V, then detected successively with W, B, M, P, Q, V to higher order, if there is no successful colliding data is detected, due to now
The replacement number of plies reached threshold value set in advance, therefore H can not replace its colliding data, then B continues to replace its next
Colliding data in addressed location.Until B had replaced all colliding datas in its addressed location, be still not present detection into
During the colliding data of work(, illustrate the detection failure to B, then all colliding datas being replaced are returned into its initial position and protected
Deposit.
Afterwards, A continues to replace next colliding data, i.e. data C, and now data C turns into the new data that are inserted into, right
Data C carries out the operation same with above-mentioned data B.If still detecting failure, A continues to replace next colliding data, directly
To after being finished to the detection of all colliding datas, if not detecting the insertion failure successfully, judged to data A yet.
In above process, when being inserted into colliding datas of the data A as certain data, can add mark make its not by
Replace；Certainly, if too many extra consumption will not also be increased (because necessarily all conflicts on its detective path by not labelling
, so only can once be detected and then be retracted).
Class Cuckoo Hash based on detection further improve the filling rate of multistage Hash, and readwrite performance has matter
Surmount.
Below to the base after existing multistage Hash, the multistage Hash proposed by the present invention based on detection, further improvement
Test comparison is carried out in the class Cuckoo Hash of detection.Test environment is single thread+B6, and data scale is about 2700W, is breathed out per rank
About 136W is wished, totally 20 rank.As Figure 10-12 be respectively to three kinds of method filling rates, write efficiency (per second) and reading efficiency (per second)
Comparative result schematic diagram.
Can clearly it find out from chart, the class Cuckoo hash performances based on detection are very in terms of filling rate
Outstanding, more than 96% filling rate can have been reached in 5 rank.We are with the class Cuckoo hash based on detection of 5 ranks
Contrasted with 20 rank Hash of plaintext：
Utilization rate height (96.5-83.3)/83.3=15.8%
Write performance improves (2668997-1391161)/1391161=91.9%
Reading performance improves (2462777-614965)/614965=300.5%
The performance why write also improvesIt is in fact not unexpected, because when exponent number falls too low and has conflict
Also it is and few.
So, the class Cuckoo hash conceptual data scales based on detection of 5 ranks are only 136*5 ≈ 700W, and common 20
The data scale of rank Hash is 2700W, such more significantIn fact, filling rate and the read-write of multistage Hash
With having no much relations per the scale of single order, we increase class Cuckoo hash single-order scale is carried out pair performance for 540W
Than 20 ranks, 5 ranks that commonly multistage Hash, data scale are 700W if Figure 13-15 is respectively 2700W to data scale are based on
The class Cuckoo Hash and data scale of detection are the filling rate of 2700W class Cuckoo Hash of 5 ranks based on detection, write efficiency
(per second) and the comparative result schematic diagram for reading efficiency (per second).By this three charts, we have observed that, the scale per single order is to entirety
The influence of efficiency is not especially big, and relative to common multistage hash data structure, the class Cuckoo hash based on detection are being filled
There is excellent performance in rate, readwrite performance.
In actual mailbox Batch Processing, when storing some sparse data, it can be wasted greatly using conventional bitmap
The internal memory of amount, and using the method proposed by the present invention optimized to multistage Hash, the filling rate of multistage Hash can be lifted,
So as to extremely efficient alleviate memory pressure.From read-write efficiency, method proposed by the present invention is also fully able to meet industry
Business demand.
The present invention also proposes a kind of device optimized to multistage Hash, including：
Detecting module, for when inserting new data to multistage Hash, if this is inserted into all addressed locations of data
On preserved colliding data, then the colliding data in all addressed locations is detected to higher order；
Preserving module, for when exist detect successful colliding data when, the exponent number for detecting successful location minimum is rushed
Prominent data are transferred to the successful position of the detection and preserved, and the new data are stored in the rank of the detection successful location
The initial position of the minimum colliding data of number.
In said apparatus, detecting module is additionally operable to, if the colliding data being inserted into all addressed locations of data
In be not present and detect successful colliding data, then the colliding data being inserted into for this in first addressed location of data perform with
Lower step：
A, data replacement conflicts data are inserted into using described；
B, data are inserted into using the colliding data as new, by being inserted into all addressed locations of data newly
Colliding data is detected to higher order；If there is successful colliding data is detected, then the preserving module is notified to detect
The minimum colliding data of the exponent number of successful location is transferred to the successful position of the detection and preserved, and described new is inserted into
Data are stored in the initial position of the minimum colliding data of exponent number of the detection successful location, terminate current process；If do not deposit
Successful colliding data is being detected, then is performing step successively for the new colliding data being inserted into all addressed locations of data
Rapid A, until when the replacement number of plies reaches the threshold value pre-set, successful colliding data is detected if be still not present, is notified
All colliding datas being replaced are returned to its initial position and preserved by the preserving module, and are inserted into for this under data
Colliding data in one addressed location performs step A；
Until when being performed both by finishing to the colliding data being inserted into all addressed locations of data, visited if be still not present
Successful colliding data is surveyed, then the insertion for judging to be inserted into this data fails, and notifies the preserving module that this is inserted into number
According to colliding data return to its initial position and preserved.
In said apparatus, colliding data is by detecting module to the mode that higher order is detected：According to the number of collisions
According to key assignments calculate addressed location of the colliding data in higher order, when finding idle addressed location, judge the conflict
Data snooping success, successful position is detected using the idle addressed location as the colliding data.
Detecting module for a colliding data for being inserted into data, by the colliding data in all addressed locations to
When higher order is detected, a variable can be safeguarded, the initial value of the variable adds 1 for the exponent number of the multistage Hash；If
Colliding data detects successfully, then judges whether the exponent number of colliding data detection successful location is less than the variable, if it is less,
The value of the variable is revised as to the exponent number of colliding data detection successful location, continues the detection to higher order colliding data；When
After all colliding data detections finish, if the value of the variable is not more than the exponent number of the multistage Hash, judge exist
Detect successful colliding data.As fully visible, the method and apparatus proposed by the present invention optimized to multistage Hash, to needing
During the new data to be inserted addressing, if each addressed location is unavailable, the conflict in its all addressed location is judged
Whether data can shift to higher order, if it can, then shifting the colliding data to higher order, and will be inserted into data insertion
The initial position of the colliding data.Further, if detection failure, can be attempted all conflicts on its detective path
Data replace, using the colliding data being replaced as the new recursive operation for being inserted into data, carrying out same scheme, it is allowed to
The level of replacement can be pre-set.The filling rate of multistage Hash can be improved in this way, while ensures higher reading
Write performance.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
God any modification, equivalent substitution and improvements done etc., should be included within the scope of protection of the invention with principle.
Claims (8)
- A kind of 1. method optimized to multistage Hash, it is characterised in that methods described includes：When inserting new data to multistage Hash, if this, which is inserted into all addressed locations of data, has preserved colliding data, Then the colliding data in all addressed locations is detected to higher order；If there is successful colliding data is detected, then the minimum colliding data of the exponent number for detecting successful location is transferred to the spy Successful position is surveyed to be preserved, and by the minimum number of collisions of exponent number for being inserted into data and being stored in the detection successful location According to initial position.
- 2. according to the method for claim 1, it is characterised in that methods described further comprises：If there is no the successful colliding data of detection, then the colliding data being inserted into for this in first addressed location of data Perform following steps：A, data replacement conflicts data are inserted into using described；B, data are inserted into using the colliding data as new, by the new conflict being inserted into all addressed locations of data Data are detected to higher order；It is if there is successful colliding data is detected, then the exponent number for detecting successful location is minimum Colliding data is transferred to the successful position of the detection and preserved, and the new data that are inserted into are stored in into detection success The initial position of the minimum colliding data of the exponent number of position, terminates current process；If there is no detecting successful colliding data, Step A is then performed successively for the new colliding data being inserted into all addressed locations of data, is reached until replacing the number of plies During the threshold value pre-set, successful colliding data is detected if be still not present, all colliding datas being replaced are returned Return its initial position to be preserved, and the colliding data being inserted into the next addressed location of data for this performs step A；Until when being performed both by finishing to the colliding data being inserted into all addressed locations of data, if be still not present detection into The colliding data of work(, then the insertion for judging to be inserted into this data fails, at the beginning of the colliding data for being inserted into data is returned into it Beginning, position preserved.
- 3. method according to claim 1 or 2, it is characterised in that described to be detected colliding data to higher order Mode is：Addressed location of the colliding data in higher order is calculated according to the key assignments of the colliding data, when the discovery free time During addressed location, judge that the colliding data detects successfully, detected using the idle addressed location as the colliding data successful Position.
- 4. method according to claim 1 or 2, it is characterised in that for a colliding data for being inserted into data, By the colliding data in all addressed locations to higher order detected when, safeguard a variable, the initial value of the variable is institute The exponent number for stating multistage Hash adds 1；If colliding data, which detects, successfully, judges that the exponent number that the colliding data detects successful location is It is no to be less than the variable, if it is less, the value of the variable to be revised as to the exponent number of colliding data detection successful location, continue pair The detection of higher order colliding data；After the detection of all colliding datas finishes, if the value of the variable is no more than described more The exponent number of rank Hash, then judge to exist to detect successful colliding data.
- 5. a kind of device optimized to multistage Hash, it is characterised in that described device includes：Detecting module, for when inserting new data to multistage Hash, if this is inserted into all addressed locations of data Colliding data has been preserved, then has been detected the colliding data in all addressed locations to higher order；Preserving module, for when exist detect successful colliding data when, the minimum number of collisions of exponent number of successful location will be detected Preserved according to the successful position of the detection is transferred to, and the new data are stored in the exponent number of the detection successful location most The initial position of low colliding data.
- 6. device according to claim 5, it is characterised in that the detecting module is additionally operable to, if described be inserted into number Successful colliding data is detected according to being not present in the colliding data in all addressed locations, then is inserted into data first for this Colliding data in addressed location performs following steps：A, data replacement conflicts data are inserted into using described；B, data are inserted into using the colliding data as new, by the new conflict being inserted into all addressed locations of data Data are detected to higher order；If there is successful colliding data is detected, then the preserving module is notified to detect successfully The minimum colliding data of the exponent number of position is transferred to the successful position of the detection and preserved, and new is inserted into data by described The initial position of the minimum colliding data of exponent number of the detection successful location is stored in, terminates current process；If there is no spy Successful colliding data is surveyed, then performs step A successively for the new colliding data being inserted into all addressed locations of data, Until when the replacement number of plies reaches the threshold value pre-set, successful colliding data is detected if be still not present, described in notice All colliding datas being replaced are returned to its initial position and preserved by preserving module, and to be inserted into data next for this Colliding data in addressed location performs step A；Until when being performed both by finishing to the colliding data being inserted into all addressed locations of data, if be still not present detection into The colliding data of work(, the then insertion for judging to be inserted into this data fail, and notify the preserving module that this is inserted into data Colliding data returns to its initial position and preserved.
- 7. the device according to claim 5 or 6, it is characterised in that the detecting module enters colliding data to higher order Row detection mode be：Addressed location of the colliding data in higher order is calculated according to the key assignments of the colliding data, works as hair During the addressed location of existing free time, judge that the colliding data detects successfully, visited the idle addressed location as the colliding data Survey successful position.
- 8. the device according to claim 5 or 6, it is characterised in that the detecting module is inserted into data for described A colliding data, by the colliding data in all addressed locations to higher order detected when, safeguard a variable, the change The initial value of amount adds 1 for the exponent number of the multistage Hash；If colliding data detects successfully, judge the colliding data detection into Whether the exponent number of work(position is less than the variable, if it is less, the value of the variable is revised as into the colliding data detects successfully position The exponent number put, continue the detection to higher order colliding data；After the detection of all colliding datas finishes, if the variable Value then judges to exist to detect successful colliding data no more than the exponent number of the multistage Hash.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310196974.7A CN104182409B (en) | 2013-05-24 | 2013-05-24 | A kind of method and device optimized to multistage Hash |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310196974.7A CN104182409B (en) | 2013-05-24 | 2013-05-24 | A kind of method and device optimized to multistage Hash |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104182409A CN104182409A (en) | 2014-12-03 |
| CN104182409B true CN104182409B (en) | 2018-01-19 |
Family
ID=51963460
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310196974.7A Active CN104182409B (en) | 2013-05-24 | 2013-05-24 | A kind of method and device optimized to multistage Hash |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104182409B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108804234B (en) * | 2017-04-28 | 2023-05-09 | 腾讯科技（深圳）有限公司 | Data storage system and method of operation thereof |
| CN113297209B (en) * | 2021-02-10 | 2024-03-08 | 阿里巴巴集团控股有限公司 | Method and device for database to execute hash connection |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101782922A (en) * | 2009-12-29 | 2010-07-21 | 山东山大鸥玛软件有限公司 | Multi-level bucket hashing index method for searching mass data |
| CN103064906A (en) * | 2012-12-18 | 2013-04-24 | 华为技术有限公司 | File management method and device |
-
2013
- 2013-05-24 CN CN201310196974.7A patent/CN104182409B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101782922A (en) * | 2009-12-29 | 2010-07-21 | 山东山大鸥玛软件有限公司 | Multi-level bucket hashing index method for searching mass data |
| CN103064906A (en) * | 2012-12-18 | 2013-04-24 | 华为技术有限公司 | File management method and device |
Non-Patent Citations (1)
| Title |
|---|
| Cuckoo hashing;Pagh R等;《Springer》;20010817;正文第124-125页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104182409A (en) | 2014-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105242871B (en) | A kind of method for writing data and device | |
| US20230317165A1 (en) | Global responder signal circuitry for memory arrays | |
| CN104899297B (en) | Create the method with the hybrid index of storage perception | |
| CN102663090B (en) | Method and device for inquiry metadata | |
| CN107153707A (en) | A kind of Hash table construction method and system for nonvolatile memory | |
| US7565482B1 (en) | Method and device for scalable multiple match extraction from search data | |
| CN104991738B (en) | A kind of solid-state disk and its read-write operation method | |
| CN103440207A (en) | Caching method and caching device | |
| CN104750432B (en) | A kind of date storage method and device | |
| CN107346212A (en) | Method for screening damaged data row and data storage device with damaged data row general table | |
| US7584173B2 (en) | Edit distance string search | |
| CN100476824C (en) | Method and system for storing element and method and system for searching element | |
| CN104182409B (en) | A kind of method and device optimized to multistage Hash | |
| CN104050057B (en) | Historical sensed data duplicate removal fragment eliminating method and system | |
| CN105279108A (en) | Method for writing data of solid-state drive | |
| CN107544749A (en) | Method for screening damaged data line of data storage medium | |
| US20170147712A1 (en) | Memory equipped with information retrieval function, method for using same, device, and information processing method | |
| CN105988719B (en) | Storage device and its method for handling data | |
| CN104951403A (en) | Low-overhead and error-free cold and hot data recognition method | |
| CN104142979B (en) | A kind of indexing means for realizing RFID tag storage management | |
| KR20170065374A (en) | Method for Hash collision detection that is based on the sorting unit of the bucket | |
| CN100399338C (en) | A sorting method of data record | |
| CN104239329B (en) | The preservation of map road shape point data and loading method and device | |
| CN104348571A (en) | Data portioning method and apparatus | |
| TWI761440B (en) | memory access method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231229 Address after: 518057, 35th Floor, Tencent Building, Keji Middle Road, High tech Zone, Shenzhen, Guangdong Province Patentee after: TENCENT TECHNOLOGY (SHENZHEN) Co.,Ltd. Patentee after: TENCENT CLOUD COMPUTING (BEIJING) Co.,Ltd. Address before: 2, 518044, East 403 room, SEG science and Technology Park, Zhenxing Road, Shenzhen, Guangdong, Futian District Patentee before: TENCENT TECHNOLOGY (SHENZHEN) Co.,Ltd. |
|
| TR01 | Transfer of patent right |