CN106407376A - Index reconstruction method and device - Google Patents
Index reconstruction method and device Download PDFInfo
- Publication number
- CN106407376A CN106407376A CN201610817528.7A CN201610817528A CN106407376A CN 106407376 A CN106407376 A CN 106407376A CN 201610817528 A CN201610817528 A CN 201610817528A CN 106407376 A CN106407376 A CN 106407376A
- Authority
- CN
- China
- Prior art keywords
- burst
- index
- copy
- main
- initial data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2272—Management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides an index reconstruction method and device. The index reconstruction method comprises the following steps of: receiving an index reconstruction request which carries a new index structure, determining a main shard corresponding to the index reconstruction request, and creating a replica shard for the main shard; copying original data recorded in the main shard to the replica shard, and establishing an index for the original data according to the new index structure to obtain new index data; and deleting the main shard, and switching the replica shard to the main shard. By use of the index reconstruction method, the original data of the main shard is synchronized to the corresponding replica shard to carry out index reconstruction on the original data, and a process that an external client side reads and writes the original data through a query request and a write request is not required so as to reduce the consumption of network resources.
Description
Technical field
The application is related to communication technical field, and more particularly, to one kind rebuilds indexing means and device.
Background technology
ElasticSearch is a search server based on Lucene.This search server comprises multiple index databases,
Client first by initial data and index structure write index database in, then search server according to this index structure to original
Data sets up index, obtains index data.But when index structure changes, old index data can not meet user's
Demand, search server needs the initial data to index database to rebuild index, to obtain new index data.
In the prior art, search server is first that this old index database rebuilds a new index database, by client
Read the initial data in this old index database, and the initial data reading out and new index structure are write new index
In storehouse, then index is rebuild to this initial data according to new index structure, obtain new index data, and indexed by modification
Another name mode is switched to new index database from this old index database, and deletes this old index database.However, because client is being read
During taking and writing initial data, it is required for by network transmission, network resource consumption ratio is larger, lead to rebuild index effect
Rate is low.
Content of the invention
In view of this, the application provides a kind of reconstruction indexing means and device, rebuilds rope to solve existing reconstruction mode
Draw the low problem of efficiency.
According to the embodiment of the present application in a first aspect, provide a kind of rebuild indexing means, methods described includes：
Receive the reconstruction index request carrying new index structure, determine that corresponding main burst asked in described index of rebuilding,
And create a Copy burst for described main burst；
The initial data of record in described main burst is copied on described copy burst, and according to described new index knot
Structure sets up index to described initial data, obtains new index data；
Delete described main burst, and by described copy burst switch based on burst.
According to the second aspect of the embodiment of the present application, provide a kind of reconstruction indexing unit, described device includes：
Receiving unit, for receiving the reconstruction index request carrying new index structure；
Creating unit, for determining the described reconstruction index corresponding main burst of request, and creates a Copy for described main burst
Burst；
Lock unit, for copying to described copy burst by the initial data of record in described main burst；
Set up indexing units, for index being set up to described initial data according to described new index structure, obtain new
Index data；
Switch unit, for deleting described main burst, and by described copy burst switch based on burst.
Application the embodiment of the present application, search server receive carry new index structure reconstruction index request when,
Determine that corresponding main burst asked in this reconstruction index, and create a Copy burst for this main burst；Former by record in this main burst
Beginning data duplication is on copy burst, and sets up index according to new index structure to this initial data, obtains new index number
According to；Delete this main burst, and by copy burst switch based on burst.Based on above-mentioned implementation, search server passes through to lead
The initial data of burst is synchronized in corresponding copy burst, to rebuild index to initial data, without external client
By inquiry request and write request, read and write the process of initial data, thus decreasing the consumption of Internet resources.
Brief description
Fig. 1 is a kind of exemplary existing reconstruction index structure figure according to an exemplary embodiment for the application；
Fig. 2A is a kind of embodiment flow chart of reconstruction indexing means according to an exemplary embodiment for the application；
Fig. 2 B is one of Fig. 2A illustrated embodiment exemplary main burst and copy burst scattergram；
Fig. 2 C is one of Fig. 2A illustrated embodiment exemplary main slicing synchronization data procedures figure；
Fig. 3 is a kind of hardware structure diagram of search server according to an exemplary embodiment for the application；
Fig. 4 is a kind of example structure figure of reconstruction indexing unit according to an exemplary embodiment for the application.
Specific embodiment
Here will in detail exemplary embodiment be illustrated, its example is illustrated in the accompanying drawings.Explained below is related to
During accompanying drawing, unless otherwise indicated, the same numbers in different accompanying drawings represent same or analogous key element.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistent with the application.On the contrary, they be only with such as appended
The example of the consistent apparatus and method of some aspects being described in detail in claims, the application.
It is the purpose only merely for description specific embodiment in term used in this application, and be not intended to be limiting the application.
" a kind of ", " described " and " being somebody's turn to do " of singulative used in the application and appended claims is also intended to including most
Form, unless context clearly shows that other implications.It is also understood that term "and/or" used herein refers to and wraps
Containing one or more associated any or all possible combination listing project.
It will be appreciated that though various information may be described using term first, second, third, etc. in the application, but this
A little information should not necessarily be limited by these terms.These terms are only used for same type of information is distinguished from each other out.For example, without departing from
In the case of the application scope, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as
One information.Depending on linguistic context, word as used in this " if " can be construed to " ... when " or " when ...
When " or " in response to determining ".
Fig. 1 is a kind of exemplary existing reconstruction index structure figure according to an exemplary embodiment for the application, such as
Shown in Fig. 1, search server includes three search nodes node1, node2 and node3, and old index database A comprises A1, A2
And tri- main bursts of A3, a new index database A ' of reconstruction is corresponding to create A ' 1, A ' 2 and 3 three main bursts of A '.Visitor
Family end first sends, to search server, the inquiry request carrying index database A mark, and search server selects one from index database A
Search node, such as node1, this inquiry request is sent to node1, and inquiry request is forwarded to A2 place by node1 respectively
On the search node node3 that search node node2 and A3 is located, initial data 2 He that A2 is recorded by node2 and node3 respectively
The initial data 3 of A3 record is sent on node1, initial data 1, initial data 2 and initial data that A1 is recorded by node1
3 are collected, and return to client；Then client will carry initial data 1, initial data 2, initial data 3 and new
The write request of index structure be sent to search server, search server is by initial data 1, initial data 2, initial data
3 and new index structure be sent to node1, node1 replicates two parts of new index structures, and will be new to initial data 1 and portion
Index structure store in A ' 1, and initial data 2 and a new index structure are sent to node2, node2 will be original
Data 2 and a new index structure store in A ' 2, and initial data 3 and a new index structure are sent to
Initial data 3 and a new index structure are stored in A ' 3 by node3, node3, and each search node is all according to new rope
Guiding structure sets up index on each main burst, when node1 receives the write success response of node2 and node3 return, will write
Enter success response and return to client；Index database A is cut by last client with changing index another name mode command deployment server
Change to index database A ', and delete index database A.
It follows that the existing process rebuilding index needs external client to assist to complete, external client needs first to
Initial data reads out, and then again by index database new to initial data and the write of new index structure, so will certainly disappear
Consume substantial amounts of network resource transmission data, and inquiry request and write request are all to be grasped on same search node
Make, and initial data and new index structure are forwarded to again by other search nodes by this search node, operating efficiency is low.
Fig. 2A is a kind of embodiment flow chart of reconstruction indexing means according to an exemplary embodiment for the application, should
Embodiment is applied on search server, and in the embodiment of the present application, search server is distributed server, comprises multiple searching
Socket point, and this search server comprises index database, and this index database comprises multiple main bursts, and each main burst is likely distributed in
It is also possible to be distributed on identical search node on different search nodes, and each main burst be to should have index structure, and
Record has initial data and index data, and as shown in Figure 2 A, this embodiment comprises the following steps：
Step 201：Receive the reconstruction index request carrying new index structure, determine that this reconstruction index request is corresponding
Main burst, and create a Copy burst for this main burst.
When index structure changes, in search server, original index data just cannot meet the use need of user
Ask, need search server to re-establish index data, the application passes through external client and sends reconstruction rope to search server
Draw request, indexed with the reconstruction triggering search server permissible in flow process, and the reconstruction index request of external client transmission
Carry new index structure, to rebuild index using this new index structure.
Ask corresponding main burst for determining that this reconstruction indexes, and the process of the burst that creates a Copy for this main burst, should
Rebuild and index database mark in index request, can also be carried, search server can first obtain this index database and identify corresponding index
Storehouse, all main burst in this index database is defined as rebuilding the index corresponding main burst of request, is then in this index database again
Each main burst create corresponding copy burst.
Wherein, include multiple index databases in search server, be stored with each index database different types of data, lead to
Cross index database mark and can distinguish different index databases, therefore rebuild and need in index request to carry index database mark, to refer to
Show that the index database of index rebuild by needs, this index database mark is referred to as index another name.Additionally, including multiple in index database
Main burst, each main burst stores a part of data of index database, and these data can be evenly distributed in each main burst
On, accordingly, it would be desirable to using all main burst in this index database as needing to rebuild the object of index, will institute in this index database
Main burst is had to be defined as rebuilding the index corresponding main burst of request.
For the process of the burst that creates a Copy for this main burst, in one example, search server can be at this main point
On the search node that piece is located, create corresponding copy burst for this main burst.
Wherein, because search server comprises multiple search nodes, the main burst of each in index database is likely distributed in not
It is also possible to be distributed on identical search node on same search node, the main burst in usual index database is with load balancing side
Formula is distributed on search node, and for example, search server has 3 search nodes, and index database comprises 3 main bursts, then 3 masters
Burst is respectively distributed on different search nodes, therefore, consumes to reduce communication between search node, search server can
, on the search node that main burst is located, to create corresponding copy burst for this main burst, and each main burst be in difference
Search node on, corresponding copy burst can be created simultaneously, improve rebuild index efficiency.Entered with an example below
Row explanation.
Fig. 2 B is one of Fig. 2A illustrated embodiment exemplary main burst and copy burst scattergram, as Fig. 2 B institute
Show, index database comprises 3 main burst A0, A1, A2, each main burst is respectively positioned on different search nodes, and A0 is in search node
On node1, on search node node2, on search node node3, search server is the pair that 3 main bursts create to A2 to A1
This burst all on the search node that each main burst is located, respectively A0 ＇, A1 ＇, A2 ＇.
Description based on this example understand, search server by index database mark get corresponding index database it
Afterwards, the copy burst creating for each main burst of this index database, is the pair creating on each main burst place search node
This burst, it is not necessary to the communication resource consuming between search node just can achieve main burst and copy burst in subsequent step
Interaction, such that it is able to reduce the communication consumption between search node, additionally, on different search nodes can simultaneously based on
Burst creates corresponding copy burst, improves the efficiency rebuilding index.
In another example, search server can also be in the other search outside the search node that this main burst is located
On node, create corresponding copy burst for this main burst.
Step 202：The initial data of record in this main burst is copied on copy burst, and according to new index structure
Index is set up to this initial data, obtains new index data.
Specifically, on each search node, the initial data of the main burst that this index database is comprised is same for search server
Walk in corresponding copy burst, and according to new index structure, index is set up to this initial data in copy burst, obtain
New index data.
It should be noted that search server based on burst create a Copy after burst, can be this copy burst wound
Build one and rebuild index mark, to make a distinction with existing copy burst.Search server is needing in described main burst
It can be determined that whether this copy burst has reconstruction index mark when the initial data of record copies on copy burst；If it has,
Then the initial data of this main burst record is copied on copy burst, if it is not, the original number by this main burst record
According to copying to together with index data on copy burst.
Wherein, rebuild index mark for the type of this copy burst of labelling, main burst is in synchrodata, if copy
Burst has reconstruction index mark, represents that this copy burst is used to rebuild index, only replicates the initial data in main burst, with
Re-establish index using new index structure, obtain new index data；Without rebuilding index mark, represent this copy
Burst is existing copy burst, needs all to copy to the initial data in main burst and index data in copy burst.
As depicted at step 201, Fig. 2 C is one of Fig. 2A illustrated embodiment exemplary main slicing synchronization data procedures
Figure, in conjunction with shown in Fig. 2 B and Fig. 2 C, main burst A0, in synchrodata, marks because corresponding copy burst A0 ＇ has reconstruction to index
Know, therefore it may only be necessary to the initial data 1 on A0 is copied in A0 ＇, A1 synchrodata to A1 ＇ and A2 synchrodata are arrived
The process of A2 ＇ is similar to, and repeats no more.
For according to new index structure, index is set up to this initial data, obtain the process of new index data, new
Index structure can be to include field, the corresponding type of this field and participle mode, and initial data includes a plurality of sub- initial data, pin
To every strip initial data, search server can obtain the corresponding data content of this field from this strip initial data, and
It is set to the corresponding type of this field, then further according to the corresponding dictionary of this field corresponding participle mode, to this data content
Carry out word segmentation processing, obtain word segmentation result；The word segmentation result that all sub- initial datas are obtained is collected, and by summarized results
Store in this field corresponding index data table, what this summarized results included is the vocabulary that data content occurs in dictionary,
And the mark of the corresponding all sub- initial datas of each vocabulary.
Wherein, multiple fields can be included in new index structure, and each field is to should have participle mode and type, example
As participle mode includes Chinese word segmentation mode, English string segmentation mode, phonetic participle mode etc., and the corresponding type of field includes
There are character types, date type, value type etc.., to there being dictionary, for example, Chinese word segmentation mode is corresponding for each participle mode
Chinese vocabulary bank, English string segmentation mode corresponding English dictionary etc..Initial data is become by many strips original data set, and every strip is former
Beginning data is to should have mark.Illustrated with an example below.
For example, in copy burst storage initial data be two documents, a document be designated 000001, an another piece
Document be designated 000002, every strip initial data corresponds to a document, and the mark of therefore wherein one strip initial data can
Think 000001, the mark of another strip initial data can be 000002, and the corresponding field of this initial data includes
Filename, Title, Body, Author, wherein, are designated 000001 sub- initial data and are specially：
{
Filename：" patent writes .doc "；
Title：" the crucial points for attention of patent writing "；
Body：" have multiple points for attention in patent writing process, after omit.”；
Author：" Zhang Qiang "；
}
It is designated 000002 sub- initial data to be specially：
{
Filename：" test data .doc "；
Title：" patent protection "；
Body：" it is noted that intellectual property of protection company "；
Author：" Li Guang "；
}
Taking field title as a example, to field title, corresponding data content carries out word segmentation processing, is designated 000001
The word segmentation result that sub- initial data obtains be " patent ", " writing ", " ", " crucial ", " attention ", " item "；It is designated
The word segmentation result that 000002 sub- initial data obtains is " patent ", " protection ", after being collected, the summarized results that obtain,
And summarized results are stored in the index data table of field title, such as table 1 is a kind of index of exemplary field title
Tables of data, as shown in table 1, word segmentation result after collecting is " patent ", " writing ", " ", " crucial ", " attention ", " item ",
" protection ", the corresponding lists of documents of each participle is：" patent " corresponding document is 000001,000002；" writing " is corresponding
Document is 000001；" " corresponding document be 000001；" crucial " corresponding document is 000001；" attention " corresponding document
For 000001；" item " corresponding document is 000001；" protection " corresponding document is 000002.
| Word segmentation result | The corresponding lists of documents of participle |
| Patent | 000001、000002 |
| Writing | 000001 |
| 's | 000001 |
| Crucial | 000001 |
| Note | 000001 |
| Item | 000001 |
| Protection | 000002 |
Table 1
If it should be noted that dictionary also has renewal, search server is original to this according to new index structure
Before data sets up index, can first new dictionary be imported in each search node, for rebuilding index.
Description based on step 202 understand, search server pass through on copy burst setting rebuild index mark, with
Existing copy burst makes a distinction, thus in synchrodata, only can copying to the initial data in main burst and carry
Have on the copy burst rebuilding index mark, and on each search node, the process of synchrodata can be carried out simultaneously, and not
Need client to realize initial data is sent in new index database by inquiry request and write request, decrease network money
The consumption in source.And each search node can be operated simultaneously, that is, each main burst and copy burst can be carried out simultaneously
The process of synchrodata, thus decreasing the communication consumption between search node, improves reconstruction index efficiency.
Step 203：Delete this main burst, and by this copy burst switch based on burst.
Specifically, the main burst of this index database is deleted on each search node by search server, and will carry weight
Index burst based on the copy burst switching of mark, thus completing the reconstruction index flow process of this index database.
It should be noted that search server is after execution step 201, and before execution step 202, can will be somebody's turn to do
The status indicator of copy burst is set to the first mark, after performing step 202, by the status indicator setting of this copy burst
For the second mark.Thus, when receiving searching request, search server can inquire about the status indicator of this copy burst；If
This status indicator is the first mark, then obtain the corresponding index data of this searching request from the corresponding main burst of copy burst；If
This status indicator is the second mark, then obtain the corresponding index data of this searching request from copy burst.
Wherein, this index database mark and content to be searched for are carried in this searching request, search server is in this index
Storehouse identifies in corresponding index database, when the status indicator inquiring this copy burst is for the first mark, represents that this copy burst is
Disabled, this corresponding index data of content to be searched for can be inquired about from the corresponding main burst of this copy burst.
From above-described embodiment, search server when receiving the reconstruction index request carrying new index structure,
Determine that corresponding main burst asked in this reconstruction index, and create a Copy burst for this main burst；Former by record in this main burst
Beginning data duplication is on copy burst, and sets up index according to new index structure to this initial data, obtains new index number
According to；Delete this main burst, and by copy burst switch based on burst.Based on above-mentioned implementation, search server passes through to lead
The initial data of burst is synchronized in corresponding copy burst, to rebuild index to initial data, without external client
By inquiry request and write request, read and write the process of initial data, thus decreasing the consumption of Internet resources.
Corresponding with the embodiment of aforementioned reconstruction indexing means, present invention also provides rebuilding the embodiment of indexing unit.
The embodiment that the application rebuilds indexing unit can be applied on search server.Device embodiment can be passed through soft
Part is realized it is also possible to be realized by way of hardware or software and hardware combining.As a example implemented in software, as a logical meaning
On device, be by the processor of its place equipment, computer program instructions corresponding in nonvolatile memory to be read
Formation is run in internal memory.For hardware view, as shown in figure 3, for the application according to an exemplary embodiment
A kind of hardware structure diagram of search server, except the processor shown in Fig. 3, internal memory, network interface and non-volatile memories
Outside device, the equipment that in embodiment, device is located, generally according to the actual functional capability of this equipment, can also include other hardware, to this
Repeat no more.
Fig. 4 is a kind of example structure figure of reconstruction indexing unit according to an exemplary embodiment for the application, should
Embodiment is applied on search server, and described search server comprises main burst, described main burst to should have index structure, and
Record has initial data and index data, as shown in figure 4, described device includes：Receiving unit 410, creating unit 420, synchronization
Unit 430, set up indexing units 440, switch unit 450.
Wherein, receiving unit 410, for receiving the reconstruction index request carrying new index structure；
Creating unit 420, for determining the described reconstruction index corresponding main burst of request, and creates pair for described main burst
This burst；
Lock unit 430, for copying to described copy burst by the initial data of record in described main burst；
Set up indexing units 440, for index being set up to described initial data according to described new index structure, obtain new
Index data；
Switch unit 450, for deleting described main burst, and by described copy burst switch based on burst.
In an optional implementation, described search server comprises index database, and described index database comprises multiple masters
Burst, also carries index database mark in described reconstruction index request, described creating unit 420, specifically for obtaining described rope
Draw storehouse and identify corresponding index database, all main burst in described index database is defined as described index of rebuilding and asks corresponding master
Burst；Create corresponding copy burst for the main burst of each in described index database.
In another optional implementation, described search server comprises multiple search nodes, described main burst position
On a search node, described creating unit 420, also particularly useful for the process in the burst that creates a Copy for described main burst
In, on the search node that described main burst is located, it is that described main burst creates corresponding copy burst；Or, in described master
On other search nodes outside the search node that burst is located, it is that described main burst creates corresponding copy burst.
In another optional implementation, described device also includes (not shown in Fig. 4)：
Search query unit, after creating a Copy burst for described main burst in described creating unit 420,
Described set up indexing units 440 according to described new index structure to described initial data set up index before, by described copy
The status indicator of burst is set to the first mark；Set up indexing units 440 described according to described new index structure to described
After initial data sets up index, the status indicator of described copy burst is set to the second mark；Receiving searching request
When, inquire about the status indicator of described copy burst；If described status indicator is the first mark, corresponding from described copy burst
Main burst obtains the corresponding index data of described searching request；If described status indicator is the second mark, divide from described copy
The corresponding index data of described searching request is obtained on piece.
In another optional implementation, described device also includes (not shown in Fig. 4)：
Rebuild index mark unit, specifically for described creating unit 420 for described main burst create a Copy burst it
Afterwards, it is that described copy burst creates a reconstruction index mark；Needing to replicate the initial data of record in described main burst
During on copy burst, judge whether described copy burst has reconstruction index mark；If it has, then by described main burst record
Initial data copies on described copy burst；If it is not, by the initial data of described main burst record and index data
Copy on described copy burst.
In said apparatus, the process of realizing of the function of unit and effect specifically refers to corresponding step in said method
Realize process, will not be described here.
For device embodiment, because it corresponds essentially to embodiment of the method, thus real referring to method in place of correlation
The part applying example illustrates.Device embodiment described above is only schematically, wherein said as separating component
The unit illustrating can be or may not be physically separate, as the part that unit shows can be or can also
It is not physical location, you can with positioned at a place, or can also be distributed on multiple NEs.Can be according to actual
Need to select the purpose to realize application scheme for some or all of module therein.Those of ordinary skill in the art are not paying
In the case of going out creative work, you can to understand and to implement.
From above-described embodiment, search server when receiving the reconstruction index request carrying new index structure,
Determine that corresponding main burst asked in this reconstruction index, and create a Copy burst for this main burst；Former by record in this main burst
Beginning data duplication is on copy burst, and sets up index according to new index structure to this initial data, obtains new index number
According to；Delete this main burst, and by copy burst switch based on burst.Based on above-mentioned implementation, search server passes through to lead
The initial data of burst is synchronized in corresponding copy burst, to rebuild index to initial data, without external client
By inquiry request and write request, read and write the process of initial data, thus decreasing the consumption of Internet resources.
The foregoing is only the preferred embodiment of the application, not in order to limit the application, all essences in the application
Within god and principle, any modification, equivalent substitution and improvement done etc., should be included within the scope of the application protection.
Claims (10)
1. a kind of rebuild indexing means, methods described applies on search server it is characterised in that described search server bag
Containing main burst, described main burst is to should have index structure, and records and have initial data and index data, and methods described includes：
Receive the reconstruction index request carrying new index structure, determine that corresponding main burst asked in described index of rebuilding, and be
Described main burst creates a Copy burst；
The initial data of record in described main burst is copied on described copy burst, and according to described new index structure pair
Described initial data sets up index, obtains new index data；
Delete described main burst, and by described copy burst switch based on burst.
2. method according to claim 1 is it is characterised in that described search server comprises index database, described index database
Comprise multiple main bursts, in described reconstruction index request, also carry index database mark, described determination is described to rebuild index request
Corresponding main burst, and the process of the burst that creates a Copy for described main burst, specifically include：
Obtain described index database and identify corresponding index database, all main burst in described index database is defined as described reconstruction rope
Draw the corresponding main burst of request；
Create corresponding copy burst for the main burst of each in described index database.
3. method according to claim 1 is it is characterised in that described search server comprises multiple search nodes, described
Main burst is located on a search node, and the process of the described burst that creates a Copy for described main burst specifically includes：
On the search node that described main burst is located, it is that described main burst creates corresponding copy burst；Or, in described master
On other search nodes outside the search node that burst is located, it is that described main burst creates corresponding copy burst.
4. method according to claim 1 is it is characterised in that methods described also includes：
After the burst that creates a Copy for described main burst, according to described new index structure, rope is being set up to described initial data
Before drawing, the status indicator of described copy burst is set to the first mark；According to described new index structure to described former
After beginning data sets up index, the status indicator of described copy burst is set to the second mark；
When receiving searching request, inquire about the status indicator of described copy burst；
If described status indicator is the first mark, obtains described searching request from the corresponding main burst of described copy burst and correspond to
Index data；
If described status indicator is the second mark, the corresponding index number of described searching request is obtained from described copy burst
According to.
5. method according to claim 1 is it is characterised in that methods described further includes：
After the burst that creates a Copy for described main burst, it is that described copy burst creates a reconstruction index mark；
When needing to copy to the initial data of record in described main burst on copy burst, whether judge described copy burst
There is reconstruction index mark；
If it has, then the initial data of described main burst record is copied on described copy burst；
If it is not, the initial data of described main burst record and index data are copied on described copy burst.
6. a kind of rebuild indexing unit, described device applies on search server it is characterised in that described search server bag
Containing main burst, described main burst is to should have index structure, and records and have initial data and index data, and described device includes：
Receiving unit, for receiving the reconstruction index request carrying new index structure；
Creating unit, for determining that described index of rebuilding asks corresponding main burst, and creates a Copy burst for described main burst；
Lock unit, for copying to described copy burst by the initial data of record in described main burst；
Set up indexing units, for index being set up to described initial data according to described new index structure, obtain new index
Data；
Switch unit, for deleting described main burst, and by described copy burst switch based on burst.
7. device according to claim 6 is it is characterised in that described search server comprises index database, described index database
Comprise multiple main bursts, in described reconstruction index request, also carry index database mark,
Described creating unit, identifies corresponding index database specifically for obtaining described index database, will be all in described index database
Main burst is defined as described index of rebuilding asks corresponding main burst；Create corresponding for the main burst of each in described index database
Copy burst.
8. device according to claim 6 is it is characterised in that described search server comprises multiple search nodes, described
Main burst is located on a search node,
Described creating unit, also particularly useful for during the burst that creates a Copy for described main burst, in described main burst institute
Search node on, be that described main burst creates corresponding copy burst；Or, the search node being located in described main burst
Outside other search nodes on, be that described main burst creates corresponding copy burst.
9. device according to claim 6 is it is characterised in that described device also includes：
Search query unit, after creating a Copy burst for described main burst in described creating unit, builds described
Before vertical indexing units set up index according to described new index structure to described initial data, by the state of described copy burst
Mark is set to the first mark；According to described new index structure, rope is set up to described initial data in described indexing units of setting up
After drawing, the status indicator of described copy burst is set to the second mark；When receiving searching request, inquire about described copy
The status indicator of burst；If described status indicator is the first mark, obtain described from the corresponding main burst of described copy burst
The corresponding index data of searching request；If described status indicator is the second mark, search described in acquisition from described copy burst
Corresponding index data asked by rope.
10. device according to claim 6 is it is characterised in that described device also includes：
Rebuild index mark unit, after creating a Copy burst for described main burst in described creating unit, be institute
State copy burst and create a reconstruction index mark；In needs, the initial data of record in described main burst is copied to copy to divide
When on piece, judge whether described copy burst has reconstruction index mark；If it has, then by the initial data of described main burst record
Copy on described copy burst；If it is not, the initial data of described main burst record and index data are copied to institute
State on copy burst.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610817528.7A CN106407376B (en) | 2016-09-12 | 2016-09-12 | Index reconstruction method and device |
| CN201911129763.5A CN110990399B (en) | 2016-09-12 | 2016-09-12 | Reconstruction index method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610817528.7A CN106407376B (en) | 2016-09-12 | 2016-09-12 | Index reconstruction method and device |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911129763.5A Division CN110990399B (en) | 2016-09-12 | 2016-09-12 | Reconstruction index method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106407376A true CN106407376A (en) | 2017-02-15 |
| CN106407376B CN106407376B (en) | 2019-12-20 |
Family
ID=57999212
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911129763.5A Active CN110990399B (en) | 2016-09-12 | 2016-09-12 | Reconstruction index method and device |
| CN201610817528.7A Active CN106407376B (en) | 2016-09-12 | 2016-09-12 | Index reconstruction method and device |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911129763.5A Active CN110990399B (en) | 2016-09-12 | 2016-09-12 | Reconstruction index method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN110990399B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110321322A (en) * | 2019-07-02 | 2019-10-11 | 深信服科技股份有限公司 | Data re-establishing method, device, equipment and computer readable storage medium |
| CN110442645A (en) * | 2019-07-11 | 2019-11-12 | 新华三大数据技术有限公司 | Data index method and device |
| CN110609865A (en) * | 2018-05-29 | 2019-12-24 | 优信拍（北京）信息科技有限公司 | Information synchronization method, device and system |
| CN110765125A (en) * | 2018-07-25 | 2020-02-07 | 杭州海康威视数字技术股份有限公司 | Data storage method and device |
| CN111061431A (en) * | 2019-11-28 | 2020-04-24 | 曙光信息产业股份有限公司 | Distributed storage method, server and client |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101196935A (en) * | 2008-01-03 | 2008-06-11 | 中兴通讯股份有限公司 | System and method for creating index database |
| CN102779160A (en) * | 2012-06-14 | 2012-11-14 | 中金数据***有限公司 | Mass data information indexing system and indexing construction method |
| CN103258036A (en) * | 2013-05-15 | 2013-08-21 | 广州一呼百应网络技术有限公司 | Distributed real-time search engine based on p2p |
| CN103605657A (en) * | 2013-10-14 | 2014-02-26 | 华为技术有限公司 | Method and device for reconstructing index online |
| CN104156367A (en) * | 2013-05-14 | 2014-11-19 | 阿里巴巴集团控股有限公司 | Search engine capacity expansion method and search service system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101295323B (en) * | 2008-06-30 | 2011-11-02 | 腾讯科技（深圳）有限公司 | Processing method and system for index updating |
| US8239389B2 (en) * | 2008-09-29 | 2012-08-07 | International Business Machines Corporation | Persisting external index data in a database |
| JP2013073557A (en) * | 2011-09-29 | 2013-04-22 | Hitachi Solutions Ltd | Information search system, search server and program |
| CN103198108B (en) * | 2013-03-27 | 2016-08-10 | 新浪网技术(中国)有限公司 | A kind of index data update method, retrieval server and system |
| CN103310023A (en) * | 2013-07-05 | 2013-09-18 | 深圳中兴网信科技有限公司 | Distributed searching system and method |
| CN103488687A (en) * | 2013-09-02 | 2014-01-01 | 用友软件股份有限公司 | Searching system and searching method of big data |
-
2016
- 2016-09-12 CN CN201911129763.5A patent/CN110990399B/en active Active
- 2016-09-12 CN CN201610817528.7A patent/CN106407376B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101196935A (en) * | 2008-01-03 | 2008-06-11 | 中兴通讯股份有限公司 | System and method for creating index database |
| CN102779160A (en) * | 2012-06-14 | 2012-11-14 | 中金数据***有限公司 | Mass data information indexing system and indexing construction method |
| CN104156367A (en) * | 2013-05-14 | 2014-11-19 | 阿里巴巴集团控股有限公司 | Search engine capacity expansion method and search service system |
| CN103258036A (en) * | 2013-05-15 | 2013-08-21 | 广州一呼百应网络技术有限公司 | Distributed real-time search engine based on p2p |
| CN103605657A (en) * | 2013-10-14 | 2014-02-26 | 华为技术有限公司 | Method and device for reconstructing index online |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110609865A (en) * | 2018-05-29 | 2019-12-24 | 优信拍（北京）信息科技有限公司 | Information synchronization method, device and system |
| CN110609865B (en) * | 2018-05-29 | 2022-04-15 | 优信拍（北京）信息科技有限公司 | Information synchronization method, device and system |
| CN110765125A (en) * | 2018-07-25 | 2020-02-07 | 杭州海康威视数字技术股份有限公司 | Data storage method and device |
| CN110321322A (en) * | 2019-07-02 | 2019-10-11 | 深信服科技股份有限公司 | Data re-establishing method, device, equipment and computer readable storage medium |
| CN110442645A (en) * | 2019-07-11 | 2019-11-12 | 新华三大数据技术有限公司 | Data index method and device |
| CN111061431A (en) * | 2019-11-28 | 2020-04-24 | 曙光信息产业股份有限公司 | Distributed storage method, server and client |
| CN111061431B (en) * | 2019-11-28 | 2023-06-23 | 曙光信息产业股份有限公司 | Distributed storage method, server and client |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106407376B (en) | 2019-12-20 |
| CN110990399B (en) | 2023-04-28 |
| CN110990399A (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106407376A (en) | Index reconstruction method and device | |
| US11853334B2 (en) | Systems and methods for generating and using aggregated search indices and non-aggregated value storage | |
| US9558203B2 (en) | Data mover discovery of object extent | |
| CN104978151B (en) | Data reconstruction method in the data de-duplication storage system perceived based on application | |
| US8301588B2 (en) | Data storage for file updates | |
| US9660962B2 (en) | Network-attached storage gateway validation | |
| CN106663047A (en) | Systems and methods for oprtimized signature comparisons and data replication | |
| US8527556B2 (en) | Systems and methods to update a content store associated with a search index | |
| Zhu et al. | Efficient single-source shortest path and distance queries on large graphs | |
| CN105593804A (en) | Method and apparatus for virtualization of file system, data storage system for virtualization of file system, and file server for use in data storage system | |
| JP2011516989A (en) | Search result ranking using edit distance and document information | |
| CN107148616A (en) | The efficient annotation system controlled for distributed version | |
| CN107710201A (en) | Data storage and from bit vector search index fetch data | |
| US20130311517A1 (en) | Representing Incomplete and Uncertain Information in Graph Data | |
| CN107851108A (en) | Use the matching document of bit vector search index | |
| US20130246365A1 (en) | Methods and systems for searching a backup volume | |
| CN104636401B (en) | Method and device for data rollback of SCADA (supervisory control and data acquisition) system | |
| CN109189778A (en) | A kind of method of online modification database table structure | |
| CN107944041A (en) | A kind of storage organization optimization method of HDFS | |
| CN107783770A (en) | Page configuration update method, device, server and medium | |
| CN110019038A (en) | File reference analysis method, apparatus, equipment and computer readable storage medium | |
| CN107209707A (en) | Hierarchy system based on cloud is preserved | |
| CN110209780A (en) | A kind of question template generation method, device, server and storage medium | |
| CN109325005A (en) | A kind of data processing method and electronic equipment | |
| KR100834293B1 (en) | Document processing system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |