CN104123300B - Data distribution formula storage system and method - Google Patents
Data distribution formula storage system and method Download PDFInfo
- Publication number
- CN104123300B CN104123300B CN201310150539.0A CN201310150539A CN104123300B CN 104123300 B CN104123300 B CN 104123300B CN 201310150539 A CN201310150539 A CN 201310150539A CN 104123300 B CN104123300 B CN 104123300B
- Authority
- CN
- China
- Prior art keywords
- data
- back end
- unit
- internal memory
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Present invention is disclosed a kind of data distribution formula storage system and method, the system includes node cluster module, data import modul, memory module；Node cluster module by the back end in cluster to connect corresponding management node；Data import modul is scanned according to the data block being sized to the data to input and is loaded into internal memory, and the data in internal memory are grouped according to the characteristic value of data, and the data after packet then are sent into corresponding data node；Data fragmentation to be retained in internal memory by memory module after back end receives file fragmentation, back end output journal to hard disk；Judge whether the size of data in internal memory exceedes set threshold values, data are reorganized as more than if, hard disk are write after compression, and delete the journal file of corresponding user memory data recovery.The present invention can realize cluster of the acceleration based on internal memory computing capability；The real-time loading and disposal ability to large-scale data, the response time of lifting system can be improved.
Description
Technical field
The invention belongs to database storage techniques field, it is related to a kind of distributed memory system, more particularly to a kind of data
Distributed memory system；Meanwhile, the invention further relates to a kind of data distribution formula storage method.
Background technology
At present, the data storage method of database has：1. unit data storage method；2. master-slave back-up storage mode；3.
Utilize the storage mode of distributed file system.However, no matter using which kind of mode above, all there is certain deficiency.
Although unit data storage method is easy to manage and use, but scalability exist major defect be difficult to meet work as
The access of modern mass data needs, and there is also problem for the security of data.Master-slave back-up storage mode solve only security and ask
Topic, other problemses are still present.Utilize the database purchase mode of distributed file system, although solve the security of data
With the access requirement of mass data, but data access and processing that those require low latency are not appropriate for.
In view of this, nowadays in the urgent need to designing a kind of new distributed memory system and method for database, with
Just the drawbacks described above of existing storage system is solved.
The content of the invention
The technical problems to be solved by the invention are：A kind of distributed memory system for database is provided, be can be achieved
Cluster and lifting based on rapid memory computing capability accelerate whole system to large-scale data real-time loading and disposal ability
Response time.
In addition, the present invention also provides a kind of data distribution formula storage method, it can be achieved based on rapid memory computing capability
Cluster and lifting accelerate the response time of whole system to large-scale data real-time loading and disposal ability.
In order to solve the above technical problems, the present invention is adopted the following technical scheme that：
A kind of data distribution formula storage system, the system includes：
Registering modules, the back end in cluster is registered into management node by client；
Data import modul, is scanned according to the data block being sized to the data to input and is loaded into internal memory,
Data in internal memory are grouped according to the characteristic value of data, and the data after packet then are sent into corresponding back end；
The data import modul specifically includes data scanning unit, packet rule match unit, data packet units, data hair
Send unit；The data scanning unit is scanned according to the data block being sized with the data to input and is loaded into internal memory,
And data are carried out with cutting according to data feature values and an integer numerical value is generated as the mark of data according to characteristic value
Code；The packet rule match unit is used to the identification code according to the Data Identification code of different pieces of information according to rule of classification
It is grouped；The data packet units by what is be scanned through in internal memory to be sized characteristic value of the data block according to data
It is grouped；The data transmission unit sends the data after packet to corresponding back end；
Memory module, data fragmentation is retained in internal memory after back end receives file fragmentation, judgement is
The data are backuped to other back end by no needs, if desired for then being backed up by backup module；Back end exports day
Will is to hard disk, for datarams data recovery；Judge whether the size of data in internal memory exceedes set threshold values, such as exceed
Then data are sorted out according to metadata feature, after the reorganization of data, then are compressed；To the group again of data
The mode knitted is mainly the characteristic value according to data, and similarity between data is ranked up so that the number of maximum similarity
It is that the compression storing data of next step is prepared according to can continuously deposit；After the reorganization of data, due to similar number
According to that can store together, it is compressed using LZAM algorithms, to obtain higher compression ratio, hard disk is then write afterwards again, and delete
Except the journal file of corresponding user memory data recovery；
Backup module, after in data transfer to corresponding back end, to backup number of the data according to setting
Mesh is backed up, and the data of backup will be distributed on other back end；
Module is retrieved, corresponding data is retrieved after the request to receive data retrieval in management node；Retrieve mould
Block specifically includes positioning unit, failure judging unit, request Dispatching Unit, retrieval unit, result combining unit；Management node is led to
Cross the back end involved by positioning unit location data retrieval request；Management node uses Lease by the judging unit that fails
Mechanism determines whether the back end fails, and request failure information is directly returned if failure, if effectively, management node is by asking
Dispatching Unit distribution request is asked to arrive respective nodes；Back end is received after data retrieval request, by retrieval unit to respective counts
According to returning results to client after being retrieved；Client is merged the result received using result combining unit.
A kind of data distribution formula storage system, the system includes：
Node cluster module, the back end in cluster is connected into corresponding management node；
Data import modul, is scanned according to the data block being sized to the data to input and is loaded into internal memory,
Data in internal memory are grouped according to the characteristic value of data, and the data after packet then are sent into corresponding back end；
Memory module, data fragmentation is retained in internal memory after back end receives file fragmentation, data section
Point output journal is to hard disk, for datarams data recovery；Judge whether the size of data in internal memory exceedes set valve
Value, reorganizes data as more than if, hard disk is write after compression, and delete the daily record text of corresponding user memory data recovery
Part.
As a preferred embodiment of the present invention, the data import modul specifically includes data cutting unit, file and swept
Retouch unit, packet rule match unit, data packet units, data transmission unit；
The data cutting unit is scanned according to the data block being sized to the data to input and is loaded into interior
Deposit；The packet rule match unit is used to set the feature that different rules calculate data according to different data types
Value；The data packet units are the data block being sized being scanned through to be grouped according to the feature of data；Institute
Data transmission unit is stated to send the data after packet to corresponding back end.
As a preferred embodiment of the present invention, the system also include backup module, to data transfer to accordingly
Back end on after, the data are backed up according to the backup number of setting, the data of backup will be distributed to other numbers
According on node.
As a preferred embodiment of the present invention, the system also includes retrieval module, to receive number in management node
According to being retrieved after the request of retrieval to corresponding data；
The retrieval module specifically includes positioning unit, failure judging unit, request Dispatching Unit, retrieval unit, result
Combining unit；
Management node passes through the back end involved by positioning unit location data retrieval request；Management node passes through failure
Judging unit determines whether the back end fails using Lease mechanism, request failure information is directly returned to if failure, if having
Effect, management node is by asking Dispatching Unit distribution request to arrive respective nodes；Back end is received after data retrieval request, is passed through
Retrieval unit returns results to client after being retrieved to corresponding data；Client will be received using result combining unit
As a result merge.
A kind of data distribution formula storage method, methods described comprises the following steps：
Node cluster step：Back end in cluster is connected into corresponding management node；
Data steps for importing：Data to input are scanned according to the data block being sized and are loaded into internal memory, internal memory
In data be grouped according to the characteristic value of data, the data after packet are then sent to corresponding back end；
Storing step：Data fragmentation is retained in internal memory after back end receives file fragmentation, back end is defeated
Go out daily record to hard disk, for datarams data recovery；Judge whether the size of data in internal memory exceedes set threshold values, such as
More than then data are reorganized, hard disk is write after compression, and delete the journal file of corresponding user memory data recovery.
As a preferred embodiment of the present invention, the data steps for importing includes：
Data scanning step, the data to input are scanned according to the data block being sized and are loaded into internal memory；
Packet rule match step, the feature that different rules calculate data is set according to different data types
Value；
Packet step, the data block being sized being scanned through is grouped according to the feature of data；
Data sending step, the data after packet are sent to corresponding back end.
As a preferred embodiment of the present invention, methods described also includes backup-step：In data transfer to corresponding number
After on node, the data are backed up according to the backup number of setting, the data of backup will be distributed to other data sections
Point on.
As a preferred embodiment of the present invention, methods described also includes searching step, and data inspection is received in management node
Corresponding data is retrieved after the request of rope；
The searching step is specifically included：
Back end involved by management node location data retrieval request；
Management node determines whether the back end fails using Lease mechanism, and request failure is directly returned if failure
Information, if effectively, respective nodes are arrived in management node distribution request；
Back end is received after data retrieval request, and client is returned results to after being retrieved to corresponding data；
Client merges the result received.
The beneficial effects of the present invention are：Data distribution formula storage system and method proposed by the present invention, it is possible to achieve base
The cluster calculated in internal memory；The real-time transaction management to large-scale data, the response time of lifting system can be achieved.At each
On back end, internal storage data is all backed up on disk, it is ensured that the safety of unit data；Simultaneity factor is set using redundant
Meter, each number according to all there is redundancy backup on different nodes, and the machine of delaying of any node does not influence data complete and system is available
Property.
Brief description of the drawings
Fig. 1 is the composition schematic diagram of data distribution formula storage system of the present invention.
Fig. 2 is the flow chart of importing data in data distribution formula storage method of the present invention.
Fig. 3 is the composition schematic diagram of the data import modul of present system.
Fig. 4 is the flow chart of data storage in data distribution formula storage method of the present invention.
Fig. 5 is the flow chart of data retrieval in data distribution formula storage method of the present invention.
Embodiment
The preferred embodiment that the invention will now be described in detail with reference to the accompanying drawings.
Embodiment one
Referring to Fig. 1, present invention is disclosed a kind of data distribution formula storage system, the system includes：Registering modules 1
（It is referred to as " node cluster module "）, data import modul 2, memory module 3, backup module, retrieval module 4.
Registering modules 1 by the back end in cluster by client to be registered to management node；
Data import modul 2 is scanned according to the data block being sized to the data to input and is loaded into internal memory,
Data in internal memory are grouped according to the characteristic value of data, and the data after packet then are sent into corresponding back end.
Specifically, referring to Fig. 3, in the present embodiment, the data import modul specifically includes data cutting unit, file
Scanning element, packet rule match unit, data packet units, data transmission unit.
The data cutting unit is scanned according to the data block being sized to the data to input and is loaded into interior
Deposit；The packet rule match unit is used to set the feature that different rules calculate data according to different data types
Value；The data packet units are the data block that is sized being scanned through in internal memory to be divided according to the characteristic value of data
Group；The data transmission unit sends the data after packet to corresponding back end.
Data fragmentation to be retained in internal memory by memory module 3 after back end receives file fragmentation, and judgement is
The data are backuped to other back end by no needs, if desired for then being backed up by backup module.Backup module is used to
Data transfer is backed up to after on corresponding back end to the data according to the backup number of setting, and the data of backup will
It is distributed on other back end.Back end output journal is to hard disk, for datarams data recovery；Judge in internal memory
Size of data whether exceed set threshold values, such as exceed if data are reorganized, then be compressed；To data again
The mode of tissue is mainly the characteristic value according to data, and similarity between data is ranked up so that maximum similarity
Data can be deposited continuously, be that the compression storing data of next step is prepared；After the reorganization of data, due to similar
Data can be stored together, and it is compressed using LZAM algorithms, to obtain higher compression ratio, then write hard disk afterwards again, and
Delete the journal file of corresponding user memory data recovery.
Retrieval module 4 after management node receives the request of data retrieval to corresponding data to retrieve.Retrieve mould
Block specifically includes positioning unit, failure judging unit, request Dispatching Unit, retrieval unit, result combining unit.
Specifically, management node passes through the back end involved by positioning unit location data retrieval request；Management node
Determine whether the back end fails using Lease mechanism by the judging unit that fails, request failure is directly returned if failure
Information, if effectively, management node is by asking Dispatching Unit distribution request to arrive respective nodes；Back end receives data retrieval please
After asking, client is returned results to after being retrieved by retrieval unit to corresponding data；Client utilizes result combining unit
The result received is merged.
The composition of data distribution formula storage system of the present invention is described above, it is of the invention while said system is disclosed,
Also disclose a kind of data distribution formula storage method；Fig. 2, Fig. 4 are referred to, methods described comprises the following steps：
【Step S1】Node cluster step（That is registration step）：By the corresponding management section of back end connection in cluster
Point, can complete connection by way of registration, and such as client sends log-on message, the back end in cluster is registered into pipe
Manage on node.
【Step S2】Data steps for importing：Data to input are scanned according to the data block being sized and are loaded into interior
Deposit, the data in internal memory are grouped according to the characteristic value of data, the data after packet are then sent to corresponding data section
Point.With reference to Fig. 3, the data steps for importing is specifically included：
Step S21, data scanning step, the data to input are scanned according to the data block being sized and are loaded into interior
Deposit；
Step S22, packet rule match step, set different rules according to different data types and calculate data
Characteristic value；
Step S23, packet step, the data block being sized being scanned through is divided according to the feature of data
Group；
Step S24, data sending step, the data after packet are sent to corresponding back end.
【Step S3】Storing step：As shown in figure 4, data fragmentation is retained in after back end receives file fragmentation
In internal memory, judge whether to need the data backuping to other back end, if desired for then being backed up.
After backup-step is included in data transfer to corresponding back end, to backup number of the data according to setting
Backed up, the data of backup will be distributed on other back end.Back end output journal is to hard disk, in data
Deposit data recovers.
Judge whether the size of data in internal memory exceedes set threshold values, reorganize data as more than if, then enter
Row compression；It is mainly the characteristic value according to data to the mode of the reorganizations of data, and the similarity between data is arranged
Sequence so that the data of maximum similarity can be deposited continuously, is that the compression storing data of next step is prepared；By data again
After tissue, because similar data can be stored together, it is compressed using LZAM algorithms, to obtain higher compression ratio,
Then write hard disk afterwards again, and delete the journal file of corresponding user memory data recovery.
【Step S4】Searching step, is retrieved after management node receives the request of data retrieval to corresponding data.Please
Refering to Fig. 5, the searching step is specifically included：
The request of data retrieval is sent on the node of data management by step S40, client；
Back end involved by step S41, management node location data retrieval request；
Step S42, management node determine whether the back end fails using Lease mechanism, are directly returned if failure
Failure information is asked, if effectively, respective nodes are arrived in management node distribution request；
Step S43, back end are received after data retrieval request, and client is returned results to after being retrieved to corresponding data
End；
Step S44, client merge the result received.
In summary, data distribution formula storage system and method proposed by the present invention, it is possible to achieve calculated based on internal memory
Cluster；The real-time transaction management to large-scale data, the response time of lifting system can be achieved.On each back end,
Internal storage data is all backed up on disk, it is ensured that the safety of unit data；Simultaneity factor is designed using redundant, each number
According to all there is redundancy backup on different nodes, the machine of delaying of any node does not influence data complete and system availability.
Here description of the invention and application be illustrative, be not wishing to limit the scope of the invention to above-described embodiment
In.The deformation and change of embodiments disclosed herein are possible, real for those skilled in the art
The replacement and equivalent various parts for applying example are known.It should be appreciated by the person skilled in the art that not departing from the present invention
Spirit or essential characteristics in the case of, the present invention can in other forms, structure, arrangement, ratio, and with other components,
Material and part are realized.In the case where not departing from scope and spirit of the present invention, embodiments disclosed herein can be entered
The other deformations of row and change.
Claims (7)
1. a kind of data distribution formula storage system, it is characterised in that the system includes：
Registering modules, the back end in cluster is registered into management node by client；
Data import modul, is scanned according to the data block being sized to the data to input and is loaded into internal memory, internal memory
In data be grouped according to the characteristic value of data, the data after packet are then sent to corresponding back end；It is described
Data import modul specifically includes data cutting unit, data scanning unit, packet rule match unit, packet list
Member, data transmission unit；The data cutting unit is scanned to the data to input according to the data block being sized
And it is loaded into internal memory；The packet rule match unit is used to calculate data according to different data type setting Different Rules
Characteristic value；The data packet units by what is be scanned through in internal memory to be sized characteristic value of the data block according to data
It is grouped；The data transmission unit sends the data after packet to corresponding back end；
Memory module, data fragmentation is retained in internal memory after back end receives file fragmentation, judges whether to need
The data are backuped into other back end, if desired for then being backed up by backup module；Back end output journal is extremely
Hard disk, for datarams data recovery；Judge whether the size of data in internal memory exceedes set threshold values, will if exceeding
Data are sorted out according to metadata feature, after the reorganization of data, then are compressed；To the reorganizations of data
Mode is the characteristic value according to data, and the similarity between data is ranked up so that the data of maximum similarity can be continuous
Storage, is that the compression storing data of next step is prepared；After the reorganization of data, because similar data can be deposited
Together, it is compressed using LZAM algorithms, to obtain higher compression ratio, then writes hard disk afterwards again, and delete corresponding
The journal file of user memory data recovery；
The data after in data transfer to corresponding back end, are entered by backup module according to the backup number of setting
Row backup, the data of backup will be distributed on other back end；
Module is retrieved, corresponding data is retrieved after the request to receive data retrieval in management node；Retrieve module tool
Body includes positioning unit, failure judging unit, request Dispatching Unit, retrieval unit, result combining unit；It is fixed that management node passes through
Back end involved by bit location location data retrieval request；Management node is by the judging unit that fails using Lease mechanism
Determine whether the back end fails, request failure information is directly returned if failure, if effectively, management node passes through request point
Respective nodes are arrived in bill member distribution request；Back end is received after data retrieval request, and corresponding data is entered by retrieval unit
Client is returned results to after row retrieval；Client is merged the result received using result combining unit.
2. a kind of data distribution formula storage system, it is characterised in that the system includes：
Node cluster module, the back end in cluster is connected into corresponding management node；
Data import modul, is scanned according to the data block being sized to the data to input and is loaded into internal memory, internal memory
In data be grouped according to the characteristic value of data, the data after packet are then sent to corresponding back end；
Memory module, data fragmentation is retained in internal memory after back end receives data fragmentation, back end is defeated
Go out daily record to hard disk, for datarams data recovery；Judge whether the size of data in internal memory exceedes set threshold values, such as
More than then data are reorganized, hard disk is write after compression, and delete the journal file of corresponding user memory data recovery；
The system also includes retrieval module, and corresponding data is examined after the request to receive data retrieval in management node
Rope；
The retrieval module specifically includes positioning unit, failure judging unit, request Dispatching Unit, retrieval unit, result and merged
Unit；
Management node passes through the back end involved by positioning unit location data retrieval request；Management node is judged by failing
Unit determines whether the back end fails using Lease mechanism, and request failure information is directly returned if failure, if effectively,
Management node is by asking Dispatching Unit distribution request to arrive respective nodes；Back end is received after data retrieval request, passes through inspection
Cable elements return results to client after being retrieved to corresponding data；Client is using result combining unit by the knot received
Fruit merges.
3. data distribution formula storage system according to claim 2, it is characterised in that：
The data import modul specifically includes data cutting unit, document scanning unit, packet rule match unit, number
According to grouped element, data transmission unit；
The data cutting unit is scanned according to the data block being sized to the data to input and is loaded into internal memory；Institute
State the characteristic value that packet rule match unit is used to calculate data according to different data type setting Different Rules；It is described
Data packet units are the data block being sized being scanned through to be grouped according to the feature of data；The data hair
Unit is sent to send the data after packet to corresponding back end.
4. data distribution formula storage system according to claim 2, it is characterised in that：
The system also includes backup module, after in data transfer to corresponding back end, to the data according to setting
Fixed backup number is backed up, and the data of backup will be distributed on other back end.
5. a kind of data distribution formula storage method, it is characterised in that methods described comprises the following steps：
Node cluster step：Back end in cluster is connected into corresponding management node；
Data steps for importing：Data to input are scanned according to the data block being sized and are loaded into internal memory, in internal memory
Data are grouped according to the characteristic value of data, and the data after packet then are sent into corresponding back end；
Storing step：Data fragmentation is retained in internal memory after back end receives file fragmentation, back end output day
Will is to hard disk, for datarams data recovery；Judge whether the size of data in internal memory exceedes set threshold values, such as exceed
Then data are reorganized, hard disk are write after compression, and delete the journal file of corresponding user memory data recovery；
Methods described also includes searching step, and corresponding data is retrieved after management node receives the request of data retrieval；
The searching step is specifically included：
Back end involved by management node location data retrieval request；
Management node determines whether the back end fails using Lease mechanism, and request failure information is directly returned if failure,
If effectively, respective nodes are arrived in management node distribution request；
Back end is received after data retrieval request, and client is returned results to after being retrieved to corresponding data；
Client merges the result received.
6. data distribution formula storage method according to claim 5, it is characterised in that：
The data steps for importing includes：
Data scanning step, the data to input are scanned according to the data block being sized and are loaded into internal memory；
Packet rule match step, the characteristic value that different rules calculate data is set according to different data types；
Packet step, the data block being sized being scanned through is grouped according to the feature of data；
Data sending step, the data after packet are sent to corresponding back end.
7. data distribution formula storage method according to claim 5, it is characterised in that：
Methods described also includes backup-step：After in data transfer to corresponding back end, to the data according to setting
Backup number is backed up, and the data of backup will be distributed on other back end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310150539.0A CN104123300B (en) | 2013-04-26 | 2013-04-26 | Data distribution formula storage system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310150539.0A CN104123300B (en) | 2013-04-26 | 2013-04-26 | Data distribution formula storage system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104123300A CN104123300A (en) | 2014-10-29 |
| CN104123300B true CN104123300B (en) | 2017-10-13 |
Family
ID=51768713
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310150539.0A Expired - Fee Related CN104123300B (en) | 2013-04-26 | 2013-04-26 | Data distribution formula storage system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104123300B (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10102086B2 (en) * | 2014-12-19 | 2018-10-16 | Futurewei Technologies, Inc. | Replicated database distribution for workload balancing after cluster reconfiguration |
| CN104572987B (en) * | 2015-01-04 | 2017-12-22 | 浙江大学 | A kind of method and system that simple regeneration code storage efficiency is improved by compressing |
| CN104679847B (en) * | 2015-02-13 | 2019-03-15 | 高第网络技术(北京)有限公司 | A kind of method and apparatus constructing online real-time update magnanimity audio-frequency fingerprint library |
| CN104731676A (en) * | 2015-03-24 | 2015-06-24 | 浪潮集团有限公司 | Method for accelerating data recovery of cluster system |
| CN105159818B (en) * | 2015-08-28 | 2018-01-02 | 东北大学 | Journal recovery method and its analogue system in main-memory data management |
| CN106648442A (en) * | 2015-10-29 | 2017-05-10 | 阿里巴巴集团控股有限公司 | Metadata node internal memory mirroring method and device |
| CN105335513B (en) * | 2015-10-30 | 2018-09-25 | 迈普通信技术股份有限公司 | A kind of distributed file system and file memory method |
| CN105516284B (en) * | 2015-12-01 | 2019-05-03 | 深圳市华讯方舟软件技术有限公司 | A kind of method and apparatus of Cluster Database distributed storage |
| CN107203554A (en) * | 2016-03-17 | 2017-09-26 | 北大方正集团有限公司 | A kind of distributed search method and device |
| CN105912601A (en) * | 2016-04-05 | 2016-08-31 | 国电南瑞科技股份有限公司 | Partition storage method for distributed real-time memory database of energy management system |
| CN105956190A (en) * | 2016-06-14 | 2016-09-21 | 武汉斗鱼网络科技有限公司 | RBF neural network-based search cluster optimization method and system |
| CN106649481A (en) * | 2016-09-30 | 2017-05-10 | 郑州云海信息技术有限公司 | A method and system of log optimization for SQL Server database |
| CN106886555A (en) * | 2016-12-27 | 2017-06-23 | 苏州春禄电子科技有限公司 | A kind of anti-loss of data based on block chain technology and the data-storage system for damaging |
| CN107436738B (en) * | 2017-08-17 | 2019-10-25 | 北京理工大学 | A kind of date storage method and system |
| CN110069483B (en) * | 2017-08-17 | 2023-04-28 | 阿里巴巴集团控股有限公司 | Method, node and system for loading data into distributed data warehouse |
| CN110019210B (en) * | 2017-11-24 | 2024-01-09 | 阿里云计算有限公司 | Data writing method and device |
| CN108664223B (en) * | 2018-05-18 | 2021-07-02 | 百度在线网络技术（北京）有限公司 | Distributed storage method and device, computer equipment and storage medium |
| CN108984686B (en) * | 2018-07-02 | 2021-03-30 | 中国电子科技集团公司第五十二研究所 | Distributed file system indexing method and device based on log merging |
| CN108921728B (en) * | 2018-07-03 | 2020-11-13 | 北京科东电力控制***有限责任公司 | Distributed real-time library system based on power grid dispatching system |
| CN108920215A (en) * | 2018-07-18 | 2018-11-30 | 郑州云海信息技术有限公司 | A method of passing through initramfs collection system log |
| CN109360605B (en) * | 2018-09-25 | 2020-10-20 | 安吉康尔（深圳）科技有限公司 | Genome sequencing data archiving method, server and computer readable storage medium |
| CN109522310A (en) * | 2018-11-16 | 2019-03-26 | 北京锐安科技有限公司 | Data storage, search method, system and storage medium |
| CN109885536B (en) * | 2019-02-26 | 2023-06-16 | 深圳众享互联科技有限公司 | Distributed data fragment storage and fuzzy search method |
| CN114281604B (en) * | 2022-03-02 | 2022-07-29 | 北京金山云网络技术有限公司 | Data recovery method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101079896A (en) * | 2007-06-22 | 2007-11-28 | 西安交通大学 | A multi-availability mechanism coexistence framework of concurrent storage system |
| CN102906751A (en) * | 2012-07-25 | 2013-01-30 | 华为技术有限公司 | Method and device for data storage and data query |
| CN103020077A (en) * | 2011-09-24 | 2013-04-03 | 国家电网公司 | Method for managing memory of real-time database of power system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120150824A1 (en) * | 2010-12-10 | 2012-06-14 | Inventec Corporation | Processing System of Data De-Duplication |
-
2013
- 2013-04-26 CN CN201310150539.0A patent/CN104123300B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101079896A (en) * | 2007-06-22 | 2007-11-28 | 西安交通大学 | A multi-availability mechanism coexistence framework of concurrent storage system |
| CN103020077A (en) * | 2011-09-24 | 2013-04-03 | 国家电网公司 | Method for managing memory of real-time database of power system |
| CN102906751A (en) * | 2012-07-25 | 2013-01-30 | 华为技术有限公司 | Method and device for data storage and data query |
Non-Patent Citations (1)
| Title |
|---|
| 搜索引擎中的分布式文件***的研究和优化;黄翀民;《中国优秀硕士学位论文全文数据库 信息科技辑》;20110415;第8页2.1.2 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104123300A (en) | 2014-10-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104123300B (en) | Data distribution formula storage system and method | |
| CN100565512C (en) | Eliminate the system and method for redundant file in the document storage system | |
| CN103646111B (en) | System and method for realizing real-time data association in big data environment | |
| US20220405277A1 (en) | Joining large database tables | |
| CN104751359B (en) | System and method for payment clearing | |
| CN103345472B (en) | De-redundant file system based on limited binary tree Bloom filter and construction method thereof | |
| CN102411637B (en) | Metadata management method of distributed file system | |
| US20200117661A1 (en) | Large scale application specific computing system architecture and operation | |
| US8510304B1 (en) | Transactionally consistent indexing for data blobs | |
| US8510344B1 (en) | Optimistically consistent arbitrary data blob transactions | |
| CN105095520B (en) | The distributed memory database indexing means of structure-oriented data | |
| CN102156727A (en) | Method for deleting repeated data by using double-fingerprint hash check | |
| US8856089B1 (en) | Sub-containment concurrency for hierarchical data containers | |
| CN107809467B (en) | Method for deleting container mirror image data in cloud environment | |
| CN100452046C (en) | Storage method and system for mass file | |
| CN104391930A (en) | Distributed file storage device and method | |
| CN103139302A (en) | Real-time copy scheduling method considering load balancing | |
| CN103067525A (en) | Cloud storage data backup method based on characteristic codes | |
| CN101814045A (en) | Data organization method for backup services | |
| CN101170416A (en) | Network data storage system and data access method | |
| CN103377100B (en) | A kind of data back up method, network node and system | |
| CN102541918A (en) | Method and equipment for identifying repeated information | |
| CN108399199A (en) | A kind of collection of the application software running log based on Spark and service processing system and method | |
| CN108108476A (en) | The method of work of highly reliable distributed information log system | |
| CN108717457B (en) | Electronic commerce platform big data processing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20171013 Termination date: 20180426 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |