CN104182534A - Data search method and device - Google Patents
Data search method and device Download PDFInfo
- Publication number
- CN104182534A CN104182534A CN201410433364.9A CN201410433364A CN104182534A CN 104182534 A CN104182534 A CN 104182534A CN 201410433364 A CN201410433364 A CN 201410433364A CN 104182534 A CN104182534 A CN 104182534A
- Authority
- CN
- China
- Prior art keywords
- equipment
- storage medium
- kernel
- data
- search
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24532—Query optimisation of parallel queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data search method and device which is used for reducing search time and increasing search efficiency. The method includes: storing data segmentation to the storage media of at least two equipment; after a search task is received, transmitting the search task to at least two equipment; instructing the two equipment to respectively search the data in respective storage media. By the method, data search is achieved, load balance is achieved, search speed is increased, and search time is reduced.
Description
Technical field
The present invention relates to field of computer technology, particularly a kind of method of data search and device.
Background technology
In the prior art, while searching for, in storage space, adopt the mode of serial operation to search for.For example, while searching in these 1000 data, 1000 data are mated with the key word of input one by one, search out result.So operation, in the large numbers of situations of data number, makes to search for overlong time used, and search efficiency is lower.
Summary of the invention
The invention provides a kind of method and device of data search, in order to reduce search duration used, improve search efficiency.
A kind of method that the invention provides data search, is characterized in that, comprising:
By data block storage in the storage medium of at least two equipment;
Receive after search mission at least two equipment described in described search mission is sent to;
Described in indication, at least two equipment is searched for the data in storage medium separately respectively.
Wherein, described data block storage is specifically comprised in the storage medium of at least two equipment:
Data are numbered;
Utilize and number data block storage in the storage medium of described at least two equipment.
Wherein, described method also comprises:
For each equipment in described at least two equipment, according to the quantity of the kernel of described equipment, the storage medium of described equipment is carried out to subregion;
Described data block storage is specifically comprised in the storage medium of at least two equipment:
By described data block storage in the subregion of the storage medium of described at least two equipment.
Wherein, described method also comprises:
For each equipment in described at least two equipment, the subregion of the storage medium of described equipment is associated with the kernel of described equipment;
Described in described indication, at least two equipment is searched for specifically and is comprised the data in storage medium separately respectively:
Described in indication, at least two equipment are distributed to kernel separately by described search mission respectively, check data in subregion associated with described kernel in storage medium and search in calling.
Wherein, described the subregion of the storage medium of described equipment be associated and specifically comprised with the kernel of described equipment:
For each kernel of described equipment, according to the processing speed of described kernel, the subregion of the storage medium of described equipment is associated with the kernel of described equipment;
The processing speed of described kernel is directly proportional with the size of the subregion of associated storage medium.
The device that the invention also discloses a kind of data search, comprising:
Data memory module, for the storage medium at least two equipment by data block storage;
Task sending module, for receiving after search mission, at least two equipment described in described search mission is sent to;
Search indicating module, described in being used to indicate, at least two equipment is searched for the data in storage medium separately respectively.
Wherein, described data memory module specifically comprises:
Numbering submodule, for being numbered data;
Sub module stored, numbers by data block storage the storage medium to described at least two equipment for utilizing.
Wherein, described device also comprises:
Partition holding module, for for described each equipment of at least two equipment, carries out subregion according to the quantity of the kernel of described equipment to the storage medium of described equipment;
Described data memory module specifically for by described data block storage in the subregion of the storage medium of described at least two equipment.
Wherein, described device also comprises:
Kernel relating module, for for described each equipment of at least two equipment, is associated the subregion of the storage medium of described equipment with the kernel of described equipment;
Described search indicating module is distributed to kernel separately by described search mission respectively specifically at least two equipment described in indicating, and checks data in subregion associated with described kernel in storage medium and search in calling.
Wherein, described kernel relating module, specifically for each kernel for described equipment, is associated the subregion of the storage medium of described equipment according to the processing speed of described kernel with the kernel of described equipment;
The processing speed of described kernel is directly proportional with the size of the subregion of associated storage medium.
Beneficial effect of the present invention is: by data block storage in the storage medium of at least two equipment, receive after search mission, search mission is sent to at least two equipment, indicate at least two equipment respectively the data in storage medium separately to be searched for; So can carry out parallel search operation to data, realize load balancing, improve search speed, reduce search duration used.
Other features and advantages of the present invention will be set forth in the following description, and, partly from instructions, become apparent, or understand by implementing the present invention.Object of the present invention and other advantages can be realized and be obtained by specifically noted structure in the instructions write, claims and accompanying drawing.
Below by drawings and Examples, technical scheme of the present invention is described in further detail.
Accompanying drawing explanation
Accompanying drawing is used to provide a further understanding of the present invention, and forms a part for instructions, for explaining the present invention, is not construed as limiting the invention together with embodiments of the present invention.In the accompanying drawings:
Fig. 1 is the process flow diagram of the method for data search in one embodiment of the invention;
Fig. 2 is the process flow diagram of the method for data search in another embodiment of the present invention;
Fig. 3 is the process flow diagram of the method for data search in the embodiment of the present invention one;
Fig. 4 is the block diagram of the device of data search in one embodiment of the invention;
Fig. 5 is the block diagram of the device of data search in another embodiment of the present invention;
Fig. 6 is the block diagram of the device of data search in another embodiment of the present invention.
Embodiment
Below in conjunction with accompanying drawing, the preferred embodiments of the present invention are described, should be appreciated that preferred embodiment described herein, only for description and interpretation the present invention, is not intended to limit the present invention.
Fig. 1 is the method for data search in one embodiment of the invention.The method comprises the steps:
In step S101, by data block storage in the storage medium of at least two equipment.
In an embodiment, step S101 can be embodied as: data are numbered; Utilize and number data block storage in the storage medium of at least two equipment.
For example, data are numbered, numbering is carried out to Hash operation, adopt the mode of key-value (key-value pair) to store data.
So storage, makes data number corresponding one by one with address data memory, can easy-to-look-up data, further improve search speed.
In step S102, receive after search mission, search mission is sent to at least two equipment.
For example, utilize two physical equipment A and B to store, data are divided into two parts, a part stores in the storage medium of physical equipment A, and another part stores in the storage medium of physical equipment B.In step S102, receive after search mission, search mission is sent to physical equipment A and physical equipment B.For example, receive the key word for searching for of input, key word is sent to physical equipment A and physical equipment B.
In step S103, at least two equipment of indication search for the data in storage medium separately respectively.
For example, indication physics device A and physical equipment B searches for the data in storage medium separately respectively.
The beneficial effect of embodiments of the invention is: by data block storage in the storage medium of at least two equipment, receive after search mission, search mission is sent to at least two equipment, indicate at least two equipment respectively the data in storage medium separately to be searched for; So can carry out parallel search operation to data, improve search data, reduce search duration used.
Fig. 2 is the method for data search in another embodiment of the present invention, and the method comprises the steps:
In step S201, for each equipment at least two equipment, according to the quantity of the kernel of equipment, the storage medium of equipment is carried out to subregion.
In an embodiment, the storage medium of equipment is marked off to the subregion identical with the number of cores of equipment.So, be convenient to subregion corresponding with kernel.This embodiment is only exemplary illustration, and the embodiment of step S201 is not limited to this.For example, the storage medium of equipment can be marked off to the subregion more than the number of cores of equipment.
For example, utilize two physical equipment A and B to store, in physical equipment A, there are 2 kernels, be divided into for core A 1 and core A 2, in physical equipment B, there are 3 kernels, be divided into for kernel B1, kernel B2 and kernel B3.For physical equipment A, the storage medium of physical equipment A is divided into 2 subregions; For physical equipment B, the storage medium of physical equipment B is divided into 4 subregions.
In step S202, for each equipment at least two equipment, the subregion of the storage medium of equipment is associated with the kernel of equipment.
In an embodiment, step S203 may be embodied as: for each kernel of equipment, according to the processing speed of kernel, the subregion of the storage medium of equipment is associated with the kernel of equipment.
Wherein, the processing speed of kernel is directly proportional with the size of the subregion of associated storage medium.
For example, the storage medium of physical equipment A is divided into 2 subregions, subregion C1 and subregion C2, wherein, the space of subregion C1 is larger than the space of subregion C2.In physical equipment A, the processing speed of core A 1 is faster than the processing speed of core A 2, in step S202, core A 1 is associated with subregion C1, and core A 2 is associated with subregion C2.The storage medium of physical equipment B is divided into 4 subregions that space is identical, is respectively subregion D1, subregion D2, subregion D3 and D4, has 3 kernels in physical equipment B, and kernel B1 is identical with kernel B2 processing speed, and kernel B3 processing speed is faster than kernel B1.Kernel B1 is associated with subregion D1, kernel B2 is associated with subregion D2, kernel B3 is associated with subregion D3 and subregion D4.
Adopt above-mentioned embodiment, the processing speed of kernel and the space size of subregion are matched, can give full play to the function of kernel, further improve data search speed.
In step S203, by data block storage in the subregion of the storage medium of at least two equipment.
In step S204, receive after search mission, search mission is sent to at least two equipment.
In step S205, indication at least two equipment respectively search mission is distributed to kernel separately, in calling, check data in subregion associated with kernel in storage medium and search for.
The beneficial effect of embodiments of the invention is: adopt the equipment with multinuclear to search for data, in a plurality of interior internuclear load balancing of carrying out, can further improve search data, reduce search duration used.
Embodiment mono-
Fig. 3 is the process flow diagram of the method for data search in the embodiment of the present invention one, in embodiment mono-, method is applied in following scene: use two equipment, be respectively physical equipment A and physical equipment B memory search data used, physical equipment A has 2 kernels, be respectively core A 1 and core A 2, in physical equipment B, there are 3 kernels, be divided into for kernel B1, kernel B2 and kernel B3.In embodiment mono-, method is carried out following steps:
In step S301, for physical equipment A, according to number of cores 2, the storage medium of physical equipment A is divided into 2 subregions; For physical equipment B, according to number of cores 3, the storage medium of physical equipment B is divided into 3 subregions.
In step S302,2 kernels in physical equipment A are associated with 2 subregions respectively, 3 kernels in physical equipment B are associated with 3 subregions respectively.
In step S303, by data block storage in 2 subregions of physical equipment A, and in 3 subregions of physical equipment B.
In step S304, receive after search mission, key word in search mission is sent to physical equipment A and physical equipment B.
In step S305, indication physics device A distributes to core A 1 and core A 2 by search mission, calling core A 1 and core A 2 uses key words to search for data in the subregion of each auto correlation, indication physics equipment B is distributed to kernel B1, kernel B2 and kernel B3 by search mission, calls kernel B1, kernel B2 and kernel B3 and uses key word to search for data in the subregion of each auto correlation.
The beneficial effect of embodiments of the invention is: adopt the equipment with multinuclear to search for data, in a plurality of interior internuclear load balancing of carrying out, can further improve search data, reduce search duration used.
Fig. 4 is the block diagram of the device of a kind of data search of the present invention, and as shown in Figure 4, device comprises:
Data memory module 41, for the storage medium at least two equipment by data block storage;
Task sending module 42, for receiving after search mission, sends at least two equipment by search mission;
Search indicating module 43, is used to indicate at least two equipment and respectively the data in storage medium is separately searched for.
In one embodiment, as shown in Figure 5, data memory module 41 specifically comprises:
Numbering submodule 51, for being numbered data;
Sub module stored 52, numbers by data block storage the storage medium at least two equipment for utilizing.
In one embodiment, as shown in Figure 6, device also comprises:
Partition holding module 61, for for each equipment of at least two equipment, carries out subregion according to the quantity of the kernel of equipment to the storage medium of equipment;
Data memory module 41 specifically for by data block storage in the subregion of the storage medium of at least two equipment.
In one embodiment, as shown in Figure 6, device also comprises:
Kernel relating module 62, for for each equipment of at least two equipment, is associated the subregion of the storage medium of equipment with the kernel of equipment;
Search indicating module 43 is distributed to search mission respectively kernel separately specifically at least two equipment of indication, checks data in subregion associated with kernel in storage medium and search in calling.
In one embodiment, kernel relating module, specifically for each kernel for equipment, is associated the subregion of the storage medium of equipment according to the processing speed of kernel with the kernel of equipment;
The processing speed of kernel is directly proportional with the size of the subregion of associated storage medium.
Beneficial effect of the present invention is: by data block storage in the storage medium of at least two equipment, receive after search mission, search mission is sent to at least two equipment, indicate at least two equipment respectively the data in storage medium separately to be searched for; So can carry out parallel search operation to data, realize load balancing, improve search speed, reduce search duration used.
About the device in above-described embodiment, wherein the concrete mode of modules executable operations have been described in detail in the embodiment of relevant the method, will not elaborate explanation herein.
Those skilled in the art should understand, embodiments of the invention can be provided as method, system or computer program.Therefore, the present invention can adopt complete hardware implementation example, implement software example or in conjunction with the form of the embodiment of software and hardware aspect completely.And the present invention can adopt the form that wherein includes the upper computer program of implementing of computer-usable storage medium (including but not limited to magnetic disk memory and optical memory etc.) of computer usable program code one or more.
The present invention is with reference to describing according to process flow diagram and/or the block scheme of the method for the embodiment of the present invention, equipment (system) and computer program.Should understand can be in computer program instructions realization flow figure and/or block scheme each flow process and/or the flow process in square frame and process flow diagram and/or block scheme and/or the combination of square frame.Can provide these computer program instructions to the processor of multi-purpose computer, special purpose computer, Embedded Processor or other programmable data processing device to produce a machine, the instruction of carrying out by the processor of computing machine or other programmable data processing device is produced for realizing the device in the function of flow process of process flow diagram or a plurality of flow process and/or square frame of block scheme or a plurality of square frame appointments.
These computer program instructions also can be stored in energy vectoring computer or the computer-readable memory of other programmable data processing device with ad hoc fashion work, the instruction that makes to be stored in this computer-readable memory produces the manufacture that comprises command device, and this command device is realized the function of appointment in flow process of process flow diagram or a plurality of flow process and/or square frame of block scheme or a plurality of square frame.
These computer program instructions also can be loaded in computing machine or other programmable data processing device, make to carry out sequence of operations step to produce computer implemented processing on computing machine or other programmable devices, thereby the instruction of carrying out is provided for realizing the step of the function of appointment in flow process of process flow diagram or a plurality of flow process and/or square frame of block scheme or a plurality of square frame on computing machine or other programmable devices.
Obviously, those skilled in the art can carry out various changes and modification and not depart from the spirit and scope of the present invention the present invention.Like this, if within of the present invention these are revised and modification belongs to the scope of the claims in the present invention and equivalent technologies thereof, the present invention is also intended to comprise these changes and modification interior.
Claims (10)
1. a method for data search, is characterized in that, comprising:
By data block storage in the storage medium of at least two equipment;
Receive after search mission at least two equipment described in described search mission is sent to;
Described in indication, at least two equipment is searched for the data in storage medium separately respectively.
2. the method for claim 1, is characterized in that, described data block storage is specifically comprised in the storage medium of at least two equipment:
Data are numbered;
Utilize and number data block storage in the storage medium of described at least two equipment.
3. the method for claim 1, is characterized in that, described method also comprises:
For each equipment in described at least two equipment, according to the quantity of the kernel of described equipment, the storage medium of described equipment is carried out to subregion;
Described data block storage is specifically comprised in the storage medium of at least two equipment:
By described data block storage in the subregion of the storage medium of described at least two equipment.
4. method as claimed in claim 3, is characterized in that, described method also comprises:
For each equipment in described at least two equipment, the subregion of the storage medium of described equipment is associated with the kernel of described equipment;
Described in described indication, at least two equipment is searched for specifically and is comprised the data in storage medium separately respectively:
Described in indication, at least two equipment are distributed to kernel separately by described search mission respectively, check data in subregion associated with described kernel in storage medium and search in calling.
5. method as claimed in claim 4, is characterized in that, described the subregion of the storage medium of described equipment is associated and is specifically comprised with the kernel of described equipment:
For each kernel of described equipment, according to the processing speed of described kernel, the subregion of the storage medium of described equipment is associated with the kernel of described equipment;
The processing speed of described kernel is directly proportional with the size of the subregion of associated storage medium.
6. a device for data search, is characterized in that, comprising:
Data memory module, for the storage medium at least two equipment by data block storage;
Task sending module, for receiving after search mission, at least two equipment described in described search mission is sent to;
Search indicating module, described in being used to indicate, at least two equipment is searched for the data in storage medium separately respectively.
7. device as claimed in claim 6, is characterized in that, described data memory module specifically comprises:
Numbering submodule, for being numbered data;
Sub module stored, numbers by data block storage the storage medium to described at least two equipment for utilizing.
8. device as claimed in claim 6, is characterized in that, described device also comprises:
Partition holding module, for for described each equipment of at least two equipment, carries out subregion according to the quantity of the kernel of described equipment to the storage medium of described equipment;
Described data memory module specifically for by described data block storage in the subregion of the storage medium of described at least two equipment.
9. device as claimed in claim 8, is characterized in that, described device also comprises:
Kernel relating module, for for described each equipment of at least two equipment, is associated the subregion of the storage medium of described equipment with the kernel of described equipment;
Described search indicating module is distributed to kernel separately by described search mission respectively specifically at least two equipment described in indicating, and checks data in subregion associated with described kernel in storage medium and search in calling.
10. device as claimed in claim 9, is characterized in that, described kernel relating module, specifically for each kernel for described equipment, is associated the subregion of the storage medium of described equipment according to the processing speed of described kernel with the kernel of described equipment;
The processing speed of described kernel is directly proportional with the size of the subregion of associated storage medium.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410433364.9A CN104182534A (en) | 2014-08-28 | 2014-08-28 | Data search method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410433364.9A CN104182534A (en) | 2014-08-28 | 2014-08-28 | Data search method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104182534A true CN104182534A (en) | 2014-12-03 |
Family
ID=51963573
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410433364.9A Pending CN104182534A (en) | 2014-08-28 | 2014-08-28 | Data search method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104182534A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104517084A (en) * | 2014-12-26 | 2015-04-15 | 张君 | Data checking system based on temporary storage area and used for filling liquid food |
| CN104516972A (en) * | 2014-12-26 | 2015-04-15 | 张君 | Data search method for filling liquid food based on temporary storage area |
| CN106776945A (en) * | 2016-11-30 | 2017-05-31 | 努比亚技术有限公司 | Mobile terminal and garbage files searching method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101183368A (en) * | 2007-12-06 | 2008-05-21 | 华南理工大学 | Method and system for distributed calculating and enquiring magnanimity data in on-line analysis processing |
| CN103365658A (en) * | 2013-06-28 | 2013-10-23 | 华为技术有限公司 | Resource access method and computer equipment |
-
2014
- 2014-08-28 CN CN201410433364.9A patent/CN104182534A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101183368A (en) * | 2007-12-06 | 2008-05-21 | 华南理工大学 | Method and system for distributed calculating and enquiring magnanimity data in on-line analysis processing |
| CN103365658A (en) * | 2013-06-28 | 2013-10-23 | 华为技术有限公司 | Resource access method and computer equipment |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104517084A (en) * | 2014-12-26 | 2015-04-15 | 张君 | Data checking system based on temporary storage area and used for filling liquid food |
| CN104516972A (en) * | 2014-12-26 | 2015-04-15 | 张君 | Data search method for filling liquid food based on temporary storage area |
| CN104517084B (en) * | 2014-12-26 | 2017-07-21 | 张君 | The data query system for pouring in liquid food based on temporary storage area |
| CN104516972B (en) * | 2014-12-26 | 2017-11-28 | 张君 | The data retrieval method for pouring in liquid food based on temporary storage area |
| CN106776945A (en) * | 2016-11-30 | 2017-05-31 | 努比亚技术有限公司 | Mobile terminal and garbage files searching method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220292163A1 (en) | Dilated convolution using systolic array | |
| EP3496008B1 (en) | Method and apparatus for processing convolution operation in neural network | |
| US10324644B2 (en) | Memory side accelerator thread assignments | |
| CN103309893A (en) | Character string comparing method and device | |
| CN104063226A (en) | Information processing method and electronic equipment | |
| CN110188100A (en) | Data processing method, device and computer storage medium | |
| EP3640813A1 (en) | Random walk method, apparatus and device, and cluster-based random walk method, apparatus and device | |
| CN102915344B (en) | SQL (structured query language) statement processing method and device | |
| CN104182534A (en) | Data search method and device | |
| WO2023065983A1 (en) | Computing apparatus, neural network processing device, chip, and data processing method | |
| CN108304460B (en) | Improved database positioning method and system | |
| EP2840499A1 (en) | Parallel computer system, control program for management device, and method of controlling parallel computer system | |
| CN103488504A (en) | Method and device for loading data | |
| CN102932416B (en) | A kind of intermediate data storage method of information flow task, processing method and device | |
| US9753769B2 (en) | Apparatus and method for sharing function logic between functional units, and reconfigurable processor thereof | |
| US9384449B2 (en) | Parallel hardware searching system for building artificial intelligent computer | |
| CN109389213B (en) | Storage device and method, data processing device and method, and electronic device | |
| CN110688223B (en) | Data processing method and related product | |
| CN103699627B (en) | A kind of super large file in parallel data block localization method based on Hadoop clusters | |
| CN110221778A (en) | Processing method, system, storage medium and the electronic equipment of hotel's data | |
| CN104572275A (en) | Process loading method, process loading device and process loading system | |
| CN102542525A (en) | Information processing equipment and information processing method | |
| CN105068850A (en) | Resource bundle loading device, method and computing device | |
| CN109308327A (en) | Figure calculation method device medium apparatus based on the compatible dot center's model of subgraph model | |
| CN109947702A (en) | Index structuring method and device, electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20141203 |