CN107577436A - A kind of date storage method and device - Google Patents
A kind of date storage method and device Download PDFInfo
- Publication number
- CN107577436A CN107577436A CN201710842915.0A CN201710842915A CN107577436A CN 107577436 A CN107577436 A CN 107577436A CN 201710842915 A CN201710842915 A CN 201710842915A CN 107577436 A CN107577436 A CN 107577436A
- Authority
- CN
- China
- Prior art keywords
- data
- current
- row
- storage file
- piecemeal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003860 storage Methods 0.000 title claims abstract description 248
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000015654 memory Effects 0.000 claims abstract description 117
- 238000013500 data storage Methods 0.000 claims abstract description 38
- 238000005192 partition Methods 0.000 claims abstract description 18
- 230000006837 decompression Effects 0.000 claims description 17
- 230000000694 effects Effects 0.000 abstract description 2
- 238000007906 compression Methods 0.000 description 5
- 230000006835 compression Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 241001269238 Data Species 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000013144 data compression Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of date storage method, when carrying out data storage, the maximum data in the specified partition section of table subregion is only stored in being indexed due to subregion, minimum data, block index only stores the maximum data in row storage file, minimum data, canned data total amount, empty data total amount, the average value of each canned data sum and canned data, so subregion index and the memory space shared by block index are very small, when mass data storage is carried out, subregion is indexed and block index can also be stored in internal memory substantially, avoid due to index file is huge and caused by internal memory frequently replace.Therefore, notebook data storage method, when carrying out mass data storage, subregion can be indexed and block index is completely stored in internal memory, improve the inquiry velocity of data.In addition, the invention also discloses a kind of data storage device, effect is as above.
Description
Technical field
The present invention relates to field of data storage, more particularly to a kind of date storage method and device.
Background technology
With the fast development of internet, data volume becomes increasing, more such as buries a daily record and monitoring day
The noncore data of will etc. becomes very huge, if being continuing with traditional data storage method carries out these mass datas
Storage, not only results in the waste of a large amount of storage resources, can also cause the inquiry velocity of data to become very slow.
For example, it is larger to employ committed memory space using the relatively low Mysql databases of relatively broad compression ratio at present
B+ trees index.Because the compression ratio of Mysql databases is relatively low, if when the size of stored data is a, it stores this and deposited
The storage size stored up shared by data is approximately then a, then B+ trees index corresponding with the data storage is shared to be deposited
Storage space size is also approximately at a.This is equivalent to the data that Mysql database purchase sizes are a are, it is necessary to which it is 2a's to take size
Memory headroom.Therefore, although Mysql databases are used for the storage of small amount of data, although B+ tree ropes corresponding with small amount of data
It is larger, can be due to data volume very little itself, so still complete B+ trees corresponding with small data quantity data can be indexed
All be stored in internal memory, internal memory will not be caused to replace so that the inquiry velocity of data quickly, still, when Mysql databases use
When the storage of mass data, because B+ trees corresponding with mass data index is larger, so can not be by complete B+ trees Suo Quan
Portion is stored in internal memory, is replaced so as to cause internal memory, and then cause the inquiry velocity of data to become very slow.
Therefore, how when carrying out mass data storage, the inquiry velocity for improving data is that those skilled in the art are current
The technical issues that need to address.
The content of the invention
It is an object of the invention to provide a kind of date storage method and device, when carrying out mass data storage, Neng Gouti
The inquiry velocity of high data.
In order to solve the above-mentioned technical problem, a kind of date storage method provided by the invention, including：
Before current data write-in, when without piecemeal is not filled with, is established in table subregion and include row storage file
Piecemeal；
The current data is split by row, and by row, concurrently forefront storage file is worked as in write-in by the current data；
The current data write after, in current piecemeal establish with it is each it is described work as forefront storage file it is corresponding
Block index, and record the maximum data worked as in the storage file of forefront, minimum data in the block index, deposited number
According to the average value of total amount, empty data total amount, each canned data sum and the canned data；
After write-in data all write, establish in current table subregion and divide with specifying in the current table subregion
Subregion corresponding to section indexes, and maximum data and minimum number in the specified partition section is recorded during the subregion indexes
According to.
Preferably, it is described when without be not filled with piecemeal when, in table subregion establish include row storage file piecemeal it is specific
Including：
When without table subregion is not filled with, newly-built current table subregion, and establish in the current table subregion include it is described
The piecemeal of row storage file.
Preferably, the date storage method also includes：
After the write-in of said write data finishes, the file data being had been written under currently stored catalogue is scanned；
When scanning is stored with uncompressed data to the row storage file being filled with piecemeal, compression is described uncompressed
Data.
Preferably, the date storage method also includes：
After the current data writes, calculate and described work as maximum data and minimum data in the storage file of forefront
Difference, and the difference is evenly dividing as N number of range section；
Established in the current piecemeal with it is each it is described work as the corresponding scope segment index of forefront storage file, and in the model
Enclose in segment index, the mark distribution situation for working as numeric type data in the storage file of forefront；
Wherein, N is positive integer.
Preferably, the date storage method also includes：
After the current data writes, established in the current piecemeal and described work as forefront storage file pair with each
The character position index answered, and in the character position index, the mark distribution for working as character type data in the storage file of forefront
Situation.
Preferably, the date storage method also includes：
, will be with described when storage file corresponding each index in forefront is loaded onto JAVA after the current data writes
Virutal machine memory；
When receiving inquiry request, if be stored with the JAVA virtual machine internal memory and the institute where current queries data
The index corresponding to row storage file is stated, then is gone out according to the indexed search stored in the JAVA virtual machine internal memory described
Current queries data.
Preferably, the date storage method also includes：
After inquiry data are obtained, by all decompression data in the row storage file where the inquiry data
Store to JAVA out-pile internal memories；
When receiving inquiry request, if being stored with current queries data in the JAVA out-piles internal memory, from described
The current queries data are obtained in JAVA out-pile internal memories.
Preferably, it is described by it is described inquiry data where the row storage file in all decompression data storages extremely
JAVA out-pile internal memories specifically include：
The decompression data are stored in the form of byte stream to the JAVA out-piles internal memory；
It is then corresponding, it is described the current queries data are obtained from the JAVA out-piles internal memory to specifically include：
The current queries data are retrieved in the JAVA out-piles internal memory in the form of the byte stream；
The current queries data are changed into the current queries data of character string forms, and obtained the character string shape
The current queries data of formula.
Preferably, the date storage method also includes：
After inquiry data are obtained, by the inquiry data write-in JAVA virtual machine internal memory, and if the JAVA is empty
The inquiry data stored in plan machine internal memory are not used in preset time, then remove and stored in the JAVA virtual machine internal memory
The inquiry data；
It is then corresponding, when receiving the inquiry request for obtaining the inquiry data in preset time, directly from described
The inquiry data are obtained in JAVA out-pile internal memories.
In order to solve the above-mentioned technical problem, a kind of data storage device that the present invention also provides, including：
Module is established, for before current data writes, when without piecemeal is not filled with, establishing and including in table subregion
The piecemeal of row storage file；
Writing module, for the current data to be split by row, and by the current data, by row, concurrently write-in is current
Row storage file；
Block index module is created, after being write in the current data, is established and each institute in current piecemeal
State and work as block index corresponding to the storage file of forefront, and described work as in the storage file of forefront most is recorded in the block index
Big data, minimum data, canned data total amount, empty data total amount, each canned data sum and the canned data it is flat
Average；
Create the division index module, after all being write in write-in data, foundation and institute in current table subregion
State subregion corresponding to the specified partition section in current table subregion to index, and the specified partition section is recorded in subregion index
In maximum data and minimum data.
As can be seen here, date storage method provided by the invention, when carrying out data storage, only deposited in being indexed due to subregion
Maximum data, the minimum data in the specified partition section of table subregion are stored up, block index is only stored in row storage file most
Big data, minimum data, canned data total amount, empty data total amount, the average value of each canned data sum and canned data, so
Memory space shared by subregion index and block index is very small, when mass data storage is carried out, index file
Can also be stored in substantially in internal memory, avoid due to index file is huge and caused by internal memory frequently replace.Therefore, user is worked as
When carrying out data query, fast positioning directly can be indexed to the table subregion for meeting querying condition using the subregion in internal memory,
After navigating to table subregion, the block index fast positioning being stored in internal memory is recycled to store text to the row for meeting querying condition
Part, the data for meeting querying condition are only then retrieved in related column storage file.So data provided by the invention
Storage method, when carrying out mass data storage, number can not only be improved by the way that index file is completely stored in internal memory
According to inquiry velocity, can also reduce the range of search of data by the only retrieval and inquisition data in related column storage file, enter
One step improves the inquiry velocity of data.In addition, present invention also offers a kind of data storage device, effect is as above.
Brief description of the drawings
In order to illustrate the embodiments of the present invention more clearly, the required accompanying drawing used in embodiment will be done simply below
Introduce, it should be apparent that, drawings in the following description are only some embodiments of the present invention, for ordinary skill people
For member, on the premise of not paying creative work, other accompanying drawings can also be obtained according to these accompanying drawings.
Fig. 1 is a kind of flow chart of date storage method provided in an embodiment of the present invention；
Fig. 2 is pair of table subregion in date storage method provided in an embodiment of the present invention, piecemeal, row storage file and index
The schematic diagram that should be related to；
Fig. 3 is the flow chart of another date storage method provided in an embodiment of the present invention；
Fig. 4 is the flow chart of another date storage method provided in an embodiment of the present invention；
Fig. 5 is the flow chart of another date storage method provided in an embodiment of the present invention；
Fig. 6 is a kind of structure chart of data storage device provided in an embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.Based on this
Embodiment in invention, those of ordinary skill in the art on the premise of creative work is not paid, obtained it is all its
His embodiment, belongs to the scope of the present invention.
It is an object of the invention to provide a kind of date storage method and device, when carrying out mass data storage, Neng Gouti
The inquiry velocity of high data.
In order that those skilled in the art is better understood from technical solution of the present invention, below in conjunction with the accompanying drawings and it is embodied
The present invention is described in further detail for mode.
Fig. 1 is a kind of flow chart of date storage method provided in an embodiment of the present invention.As shown in figure 1, data storage side
Method includes：
S10：Before current data write-in, when without piecemeal is not filled with, is established in table subregion and include row storage text
The piecemeal of part.
S11：Current data is split by row, and concurrently forefront storage file is worked as in write-in by row.
S12：After current data writes, established in current piecemeal with respectively when the corresponding piecemeal of forefront storage file
Index, and the maximum data worked as in the storage file of forefront, minimum data, canned data total amount, empty number are recorded in block index
According to the average value of total amount, each canned data sum and canned data.
S13：After write-in data whole write-in, establish in current table subregion and divide with specifying in current table subregion
Subregion corresponding to section indexes, and maximum data and minimum data in subregion index in record specified partition section.
It should be noted that the present invention is preferentially used for the storage of basic ordered data.Current data refers to that row write will be entered
The data of the fixed data amount entered, and fixed data amount be less than or equal to a row storage file at most can data storage row
Number.Such as, when can at most store 100,000 row data in a row storage file, and write data volume and be more than 100,000 row, Gu
It is 100,000 rows to determine data volume；When a row storage file can at most store 100,000 row data, and write data volume and be less than or wait
When 100,000 row, fixed data amount is identical with write-in data volume.Wherein, total data of the data volume for all write-in data is write
Amount, and for write data volume be more than fixed data amount in the case of for, be will write data volume be divided into multiple fixed data amounts,
Recurrent wrIting is carried out using the write-in data of fixed data amount as current data.
Fig. 2 is pair of table subregion in date storage method provided in an embodiment of the present invention, piecemeal, row storage file and index
The schematic diagram that should be related to.As shown in Fig. 2 piecemeal refers to comprising row storage file and the block indexed corresponding with row storage file point
Area 20, table subregion are the subregion 2 for the block subregion 20 for including fixed number.
In addition, it is necessary to explanation, current piecemeal is the piecemeal of write-in current data, the row storage file in current piecemeal
For when forefront storage file.
In specific implementation, for step S10, after the write request of write-in data is received, obtain write first
Enter the information of the write-in data volume carried in request and determine to write data volume, then judge whether to have in memory space and do not deposit
Full piecemeal, if so, then entering step S11, otherwise, a piecemeal for including row storage file is newly established in table subregion
As current piecemeal with when storing current data in the storage file of forefront.Also, it is to be understood that without not in memory space
Being filled with the situation of piecemeal has two kinds, and the first situation is：Without table subregion in memory space, i.e., without piecemeal in memory space, or deposit
Existing table subregion in space is stored up, and existing table subregion has been filled with；Second of situation be：Existing table subregion in memory space, though
Right table subregion is not filled with, but all piecemeal of table subregion has been filled with., it is necessary to newly-built table subregion for the first situation
As current table subregion, then piecemeal is established in current table subregion again；For second of situation, then without newly-built table again
Subregion, newly-built piecemeal in table subregion is not directly filled with existing.That is, after write request is received, working as
Before preceding data write-in, table subregion or the piecemeal belonging to current data can be first determined, has only been filled with without piecemeal or piecemeal
In the case of, the just newly-built piecemeal for including row storage file in table subregion.Wherein, current table subregion, which refers to, includes current point
The table subregion of block.
During in order to there is no table subregion in memory space, it still can will write that data are orderly to be stored, as excellent
Selection of land embodiment, when without piecemeal is not filled with, is established in table subregion and include the piecemeal of row storage file and specifically include：When
When nothing is not filled with table subregion, newly-built current table subregion, and the piecemeal for including row storage file is established in current table subregion.
For step S11, after data check success is write, current data is split (before will be per the trade by row
Data are divided into different row according to data type, and same row includes the data of same data type) after, by current data by row
Concurrent write-in is worked as in the storage file of forefront, and after the completion of current data write-in, each row storage file stores a kind of data,
Such as, current data is divided into three row according to time, age and number, then when being only stored with the row storage file of storage time
Between data, storage the age row storage file in be only stored with age data, storage number row storage file in only
Number is stored with, and it is mutually corresponding with the data of a line.Wherein, verification is carried out to write-in data and refers to whether proof list is deposited
, field whether there is, row whether there is, whether data type meets and data length whether overlength etc..
If a row storage file can store 100,000 row data, a table subregion can at most include 100 points
Block, include in a piecemeal it is different and mutually corresponding to row storage file.In a specific implementation, currently stored sky
Between in there is no table subregion, then receive request write-in 1,010,000 row data write request after, 100,000 row writes can be entered data work
For current data, and before current data write-in, can newly-built 1 table subregion is as current table subregion in advance, and current
A piecemeal is established in table subregion as current piecemeal, then current data is concurrently write in current piecemeal by row and works as forefront
In storage file, after current data writes, it may proceed to establish new piecemeal in current table subregion as current piecemeal,
Then 100,000 rows are not write into data again and concurrently writes working as in the storage file of forefront in newly-built current piecemeal by row, successively class
Push away, be respectively written into until 100,000 row writes are entered into data in 10 piecemeals, remaining 10,000 row data do not write memory space, at this moment still
It is so a newly-built piecemeal in current table subregion, remaining 10,000 row is not write into data is concurrently write as current data by row
Enter in the row storage file in the piecemeal, most write data all write-in memory spaces at last, complete data storage.
In another embodiment, currently there is 1 table subregion for including 11 piecemeals, and in the 11st piecemeal only
Store 10,000 row data, then, then can be first by the write-in number of 90,000 rows when receiving the write request of 2,060,000 row data of write-in
Working as in the storage file of forefront in the 11st piecemeal is concurrently write by row according to as current data, is finished in current data write-in
Afterwards, then may proceed to a newly-built piecemeal in current table subregion, as current piecemeal, the write-in data that 100,000 rows are not write are made
Working as in the storage file of forefront in newly-built current piecemeal is concurrently write by row for current data, the like, until by 1,990,000
Row write enters data and is respectively written into 20 piecemeals, is still in current table when remaining 70,000 row writes enter data and do not write memory space
A newly-built piecemeal in subregion, remaining 70,000 row is not write into data and concurrently write in the piecemeal by row as current data
In row storage file, data all write-in memory spaces are most write at last, complete data storage.
In another embodiment, there are a table subregion for including 31 piecemeals, and in current memory space
70,000 row data are only stored in 31 piecemeals, then, then can be by 30,000 when receiving the write request of 9,000,000 row data of write-in
Capable write-in data are concurrently write in the row storage file in the 31st piecemeal as current data by row, are write in current data
After, then it may proceed in current table subregion that a newly-built piecemeal is as current piecemeal, the write-in number that 100,000 rows are not write
Working as in the storage file of forefront in newly-built current piecemeal is concurrently write by row according to as current data, the like, until will
6930000 row writes enter data and are respectively written into 70 piecemeals, and current table subregion has been write completely, and remaining 2,070,000 row writes enter data and do not write
, it is necessary to newly establish a table subregion as current table subregion when entering memory space, and one is established in newly-built current table subregion
Individual piecemeal is as current piecemeal, and using 100,000 row data in remaining 2,070,000 row data as current data, by row, concurrently write-in is new
Working as in the storage file of forefront in the current piecemeal built, a piecemeal then is established as current point in current table subregion again
Block, concurrently write in newly-built current piecemeal by row using 100,000 row data in remaining 1,970,000 row data as current data
Work as in the storage file of forefront, the like, until 2,000,000 row writes entered into data be respectively written into 20 piecemeals, remaining 70,000 rows
Data do not write memory space, are at this moment still a newly-built piecemeal in current table subregion, remaining 70,000 row is not write into number
Concurrently write in the row storage file in the piecemeal by row according to as current data, most writing data at last, all write-in storage is empty
Between, complete data storage.
It is, of course, understood that when receiving other write requests, according to the rule of above three specific embodiment
Rule, before current data write-in, when without piecemeal is not filled with, the piecemeal for including row storage file is established in table subregion.
In addition, it is necessary to illustrate, the specified partition section mentioned in step S13 is necessary for store basic ordered data
Data arrange.
For step S12, after current data writes, it is meant that current piecemeal has write full or write-in data
Through all writing memory spaces, then in order to after inquiry request is received, quickly retrieve inquiry data, can establish immediately with
In current piecemeal respectively when block index corresponding to the storage file of forefront, i.e. each row storage file have a block index
Correspond to therewith, and maximum data, minimum data, canned data in corresponding row storage file are recorded in block index
Total amount, empty data total amount, the average value of each canned data sum and canned data, thus it is determined that inquiry data in which table
, can be with by judging that inquiry data whether both less than the maximum recorded in block index, are more than piecemeal rope again after subregion
Draw the minimum value of middle record, and judge according to the other information of the record in block index in corresponding row storage file
Whether data meet other querying conditions, it is determined that inquiry data whether may in row storage file corresponding to the block index,
It is achieved thereby that according to the row storage file where block index fast positioning to inquiry data, to be examined in the row storage file
Rope, which goes out, inquires about data, reduces query context, lifts the inquiry velocities of data, wherein, inquiry data refer to meeting querying condition
Data.
Similarly, for step S13, after write-in data all write-in, in order to after inquiry request is received,
Inquiry data are quickly retrieved, subregion index corresponding with the specified partition section in table subregion can be established immediately, and due to referring to
Determine the data row that sectional corresponds to store basic ordered data, so, it is only necessary to record in subregion index and correspond to therewith
Table subregion specified partition section in maximum data and minimum data, can be to react the data of the storage in the section of specified partition
Which substantially has, so by whether judging to inquire about data both less than maximum data is recorded in subregion index, is more than subregion rope again
Draw the minimum data of middle record, it is determined that inquiry data whether may be indexed in the subregion corresponding in table subregion, so as to
After receiving inquiry request, the table subregion where fast positioning to inquiry data is indexed according to subregion, is examined in corresponding table subregion
Rope inquires about data, reduces query context, lifts the inquiry velocity of data.Wherein, maximum data and minimum data refer both to numeric type
Data, it is, of course, understood that the average value of canned data sum and canned data also refer both to it is numeric type data and and flat
Average.
Moreover, it is noted that when the data renewal in row storage file, subregion rope corresponding with row storage file
Drawing can real-time update therewith with block index.
As can be seen here, the date storage method that the present embodiment provides, when carrying out data storage, in being indexed due to subregion only
Maximum data, the minimum data in the specified partition section of table subregion are stored, block index is only stored in row storage file
Maximum data, minimum data, canned data total amount, empty data total amount, the average value of each canned data sum and canned data, institute
It is very small with the memory space shared by subregion index and block index, when mass data storage is carried out, index text
Part can also be stored in internal memory substantially, avoid due to index file is huge and caused by internal memory frequently replace.Therefore, when with
When family carries out data query, fast positioning directly can be indexed to the table subregion for meeting querying condition using the subregion in internal memory,
After table subregion is navigated to, the block index fast positioning being stored in internal memory is recycled to store text to the row for meeting querying condition
Part, the data for meeting querying condition are only then retrieved in related column storage file.So number that the present embodiment provides
According to storage method, when carrying out mass data storage, can not only be improved by the way that index file is completely stored in internal memory
The inquiry velocity of data, the range of search of data by the only retrieval and inquisition data in related column storage file, can also be reduced,
Further increase the inquiry velocity of data.
In specific implementation, if during data write row storage file, the data in row storage file are entered
The synchronous squeeze operation of row, can not only reduce the writing speed of data, can also likely cause cpu spike, especially write
When the concurrency of data is higher, it frequently can lead to CPU and rise violently, so as to reduce the inquiry velocity of data.Therefore, in order to save
Memory space, the writing speed and inquiry velocity for lifting data, the present embodiment further change on the basis of above-described embodiment
Enter, line asynchronous compression can be entered to writing the data in full row storage file.
Fig. 3 is the flow chart of another date storage method provided in an embodiment of the present invention.As shown in figure 3, as preferred
Ground embodiment, on the basis of Fig. 1, after execution of step S13, in addition to：
S30：After write-in data write-in finishes, the file data being had been written under currently stored catalogue is scanned.
S31：When scanning is stored with uncompressed data to the row storage file being filled with piecemeal, uncompressed number is compressed
According to.
It should be noted that for step S30, daemon thread can be utilized by scanning under currently stored catalogue
File, the last modification time of file is filtered, the last modification time for the file that Current Scan arrives, which is less than, to be preset
During time (such as 1 day), then the compressing mark of the row storage file being filled with piecemeal inside this document folder is continued to scan on.For
For step S31, if scanning to the compressing mark of the row storage file being filled with piecemeal be uncompressed mark when, to this
The uncompressed data stored in row storage file are compressed, and after the completion of data compression, change the row storage file
Compressing mark is compressing mark, deletes the uncompressed data stored in the row storage file.Wherein, preset time is according to real
Border needs what is pre-set.
Furthermore, it should be noted that when carrying out data query, if only needing to obtain a certain data, only need
A certain row storage file is decompressed, can without carrying out decompression operation to all row storage files
Further lift the speed of data query.
As can be seen here, the date storage method that the present embodiment provides, by using asynchronous compression, can not only not influence
On the premise of data write, memory space is saved, moreover, will not also cause cpu spike and cause the inquiry velocity of data slack-off,
So notebook data storage method can lift the inquiry velocity of data.
In specific implementation, in order to lift the inquiry velocity of data, range of search is navigated to except being indexed by subregion
Table subregion, then narrowed down to range of search outside storage file by block index, more careful index can also be established, will be examined
Rope scope is reduced to smaller.Therefore, the present embodiment is further improved on the basis of above-described embodiment, is stored by establishing with row
Scope segment index corresponding to file, the data of which range section included in corresponding row storage file for marking, when
When user carries out data query, then the row storage file where inquiry data can be locked by scope segment index.Such as, a row
The minimum data stored in storage file is 0, maximum data 749, but not 400 this data of storage, then works as user
When inquiring about data 400, if according to block index retrieval and inquisition data, then will be considered that in row storage file to have
400 this data, and all data to being stored in whole row storage file are retrieved, but finally without retrieving
400 this data.And if establishing scope segment index, then whether can have in note row storage file is got the bid in scope segment index
There are the data of which range section, such as in scope segment index, marked the number for not having 250-499 range sections in row storage file
According to, then when user inquires about data, although maximum data and minimum number that inquiry data 400 store in row storage file
Between, but due to having remembered in the row storage file there is no the data of 250-499 range sections in scope segment index acceptance of the bid, then recognize
There is no 400 this data in the row storage file, it is not necessary to search operaqtion is carried out to all data in the row storage file,
And directly retrieve 250-499 range sections in other scope segment index labeled as the data that store in 1 row storage file,
So as to reduce the range of search of data, the inquiry velocity of data can be further lifted.
Fig. 4 is the flow chart of another date storage method provided in an embodiment of the present invention.As shown in figure 4, as preferred
Ground embodiment, on the basis of Fig. 1, in addition to：
S40：After current data writes, the difference of the maximum data and minimum data worked as in the storage file of forefront is calculated
Value, and difference is evenly dividing as N number of range section.
S41：Established in current piecemeal with respectively when the corresponding scope segment index of forefront storage file, and in scope segment index
In, mark works as the distribution situation of numeric type data in the storage file of forefront.
It should be noted that being performed after step S40 is arranged in into execution of step S11 in the present embodiment, step S41 is pacified
Performed after coming execution of step S12, but in specific implementation, step S40 can be in the either step after execution of step S11
Performed before or after execution, as long as and step S41 is performed after step S40.The distribution situation of numeric type data is
Which range section is the data referred in row storage file are distributed in.Moreover, when the data renewal in row storage file, with row
The also real-time update therewith of scope segment index corresponding to storage file.
In addition, the N mentioned in step S40 is positive integer, and when wishing that scope segment index is more careful, then N can be set
That puts is bigger, but simultaneously, N is bigger, and the memory space shared by scope segment index is also bigger, so N value, can be according to reality
Border needs to be pre-set.Usually, when a table subregion includes 100 piecemeals, and the row storage file in piecemeal can be with
When storing 100,000 row data, N can be arranged to 1024, that is to say, that make the maximum between the data that are stored in row storage file
Difference is uniformly divided into 1024 range sections, then whether marked has this 1024 scope segment datas in row storage file
Memory space shared by scope segment index is about only the percentage of the memory space shared by all data in storage file
One of.
In specific implementation, for step S41, the number having in 1 flag column storage file in range section can be used
According to the data not having in 0 flag column storage file in range section, 0-249 range sections such as are labeled as into 1, show row storage text
Include 0-249 data in part, 250-499 range sections are labeled as 0, show not including 250-499's in row storage file
Data, 500-749 range section is labeled as 1, shows the data for including 500-749 in row storage file, by that analogy, used
Whether there are the data of different range section in 0 and 1 flag column storage file.
As can be seen here, the date storage method that the present embodiment provides, scope that can be very small by taking memory space
When data query is further reduced in segment index, the range of search of data, the inquiry velocities of data is lifted.
In specific implementation, perhaps while numeric type data is stored, string data can be also stored, in order that this hair
The date storage method of bright offer, the storage of string data is equally applicable to, the present embodiment is on the basis of above-described embodiment
It is further improved, by foundation character position index corresponding with row storage file, the word in row storage file will be stored in
Where some character in symbol string data is being marked so that when user is in polling character string data, passes through character
Row storage file where the string data of position index fast positioning inquiry.
Fig. 5 is the flow chart of another date storage method provided in an embodiment of the present invention.As shown in figure 5, as preferred
Ground embodiment, on the basis of Fig. 4, in addition to：
S50：After current data writes, established in current piecemeal with respectively when the corresponding character of forefront storage file
Position index, and in character position index, mark works as the distribution situation of character type data in the storage file of forefront.
It should be noted that performed after step S50 is arranged in into execution of step S41 in the present embodiment, but specific real
Shi Zhong, as long as step S50 is performed before or after being the either step execution after execution of step S11.Character type number
According to distribution situation refer to some character in the string data that is stored in row storage file where, moreover, working as
During data renewal in row storage file, character position index corresponding with row storage file also real-time update therewith.
, can be with 0 and 1 come in the data that are stored in flag column storage file for step S50 in specific implementation
Some character whether in some position.For example, the position that character A is likely to occur is 64, then 64 positions are compiled since 1
Number, to compile always to 64, i.e. the numbering of M (1≤M≤64) individual position is M, the position mark that then will have A to occur is 1, is not had
The position mark that A occurs is 0, and similarly, the marking convention of the marking convention and character A of other characters duplicates, moreover, being based on the word
It is only the memory space shared by the data in row storage file to accord with the memory space shared by the character position index of marking convention
1 percent.Specifically, when user needs inquiry string ABC, then first judge whether A occurred on 1 position, B is 2
Position on whether occurred, whether C occurred on 3 position, that is to say, that as long as judge retrieval data 1 position on,
Mark whether corresponding to character A on 0,2 positions, to mark whether corresponding to character B on 0 and 3 positions, to be marked corresponding to character C
Whether note is 0, as long as and one of mark is that data in the row storage file for then judging currently to retrieve do not have
There is ABC, in this way, can then determine that inquiring about data whether there is in the row storage file currently retrieved only by character position index
In, looked into without being retrieved to the total data in all row storage files with determining to work as whether to have in the storage file of forefront
Data are ask, further reduce the range of search of data, improve the inquiry velocity of data.
As can be seen here, the date storage method that the present embodiment provides, character that can be very small by taking memory space
When position index further reduces data query, the range of search of data, the inquiry velocities of data is lifted.
Moreover, in specific implementation, when the querying condition of user is multiple, determination can be indexed first with subregion and be come from
The partial data of body storage meets the table subregion of first querying condition, then recycles block index, scope segment index and word
Symbol position index continues retrieval and determined in the table subregion, and all data of itself storage are satisfied by the first of first querying condition
Strong correlation row storage file, and the first related column storage file of partial data first querying condition of satisfaction of itself storage,
May then continue with using block index, scope segment index and character position index continue retrieval determine the first strong correlation row storage
In file or the first related column storage file, all data of itself storage equally meet the last the second of second querying condition
Related column storage file, and the second related column storage file of partial data second querying condition of satisfaction of itself storage, with
This analogizes.So, when only inquiry meets the number of inquiry data of querying condition, then only need to store text to related column
The data stored in part carry out decompression retrieval, and the data for being stored in strong correlation row storage file, then without carrying out
Decompression retrieval, the inquiry data of querying condition can be met with determination according only to index corresponding with the strong correlation row storage file
Number, so as to avoid unnecessary decompression retrieval, reduce range of search, improve the inquiry velocity of data.Moreover, just
Inquiry meets the inquiry data of querying condition at last, is also only often to obtain a certain number of inquiry data, also and need not
Decompression retrieval is carried out to each row storage file, if the inquiry included in the data stored in strong correlation row storage file
Data have met query demand, then can equally avoid carrying out decompression inspection to the data that store in related column storage file
Rope, the range of search of data is further reduced, improve the retrieval rate of data.
In order to further lift the inquiry velocity of data, index is straight corresponding to the data that will can be updated in row storage file
Connect and be loaded onto JAVA virtual machine internal memory, when needing to inquire about the data of renewal, then without being called again from internal memory and more
Indexed corresponding to row storage file where new data, so as to further lift the inquiry velocity of data.As preferred
Ground embodiment, the date storage method that above-described embodiment provides also include：, will be with working as forefront after current data writes
Each index is loaded onto JAVA virtual machine internal memory corresponding to storage file；When receiving inquiry request, if JAVA virtual machine internal memory
In be stored with index corresponding with the row storage file where current queries data, then according to storing in JAVA virtual machine internal memory
Indexed search goes out current queries data.And if there is no storage to be deposited with the row where current queries data in JAVA virtual machine internal memory
Store up and indexed corresponding to file, then go out current queries data according to the indexed search stored in internal memory.Specifically, each deposits row storage
When data in file update, it is virtual index corresponding with the row storage file for updating the data place can be loaded onto JAVA
Machine internal memory, and the index amount that can be loaded in JAVA virtual machine internal memory can be changed by configuration file.In addition, work as JAVA virtual machine
When the index amount loaded in internal memory reaches maximum, it can also be replaced by minimum in the recent period using algorithm (lru algorithm).
In order to further lift the inquiry velocity of data, the data stored in the row storage file of recent queries can be deposited
Storage is into JAVA out-pile internal memories so that when user obtains the data stored in the row storage file that recent queries cross again, directly
Connect from JAVA out-piles internal memory and obtain the data that solution presses through, without transferring this from hardware storage device or internal memory again
Row storage file, and decompression operations are carried out to obtain inquiry data to the data that do not decompress in the row storage file, so as to save
Time of data query, improve the inquiry velocity of data.Preferably embodiment, the data that above-described embodiment provides
Storage method also includes：After inquiry data are obtained, by all decompression data in the row storage file where inquiry data
Store to JAVA out-pile internal memories；When receiving inquiry request, if being stored with current queries data in JAVA out-pile internal memories,
Current queries data are obtained from JAVA out-pile internal memories.If no storage current queries data in JAVA out-pile internal memories, from
The row storage file is obtained in hardware storage device or internal memory, and decompression behaviour is carried out to the data that do not decompress in the row storage file
Make, to obtain current queries data.Storable data volume can also be modified by configuration file in JAVA out-pile internal memories,
Moreover, when the data volume stored in JAVA out-pile internal memories reaches maximum, can be replaced by lru algorithm.In addition, value
It is noted that can be cached according to the total amount of JAVA out-pile internal memories and the amount that has used dynamically to adjust in JAVA out-pile internal memories
The maximum amount of data of data, e.g., one-time detection is carried out every ten seconds service conditions to JAVA out-pile internal memories, when in JAVA out-piles
When the usage amount deposited is more than 80 percent, then maximum amount of data is reduced 10；When the use of JAVA out-pile internal memories
When amount is less than 70 percent, then maximum amount of data is increased by 10.
In specific implementation, because the character string in JAVA is using unicode coding storages, and unicode codings make
With 3 to 4 byte-identifiers, one character, so, if using the character string mode in JAVA by the decompression number in storage file
According to being stored in JAVA out-pile internal memories, very more memory spaces will be wasted.Therefore, preferably embodiment, it is above-mentioned will
All decompression data storages to the JAVA out-pile internal memories in row storage file where inquiry data specifically include：Data will be decompressed
Stored in the form of byte stream to JAVA out-pile internal memories.It is then corresponding, it is specific that current queries data are obtained from JAVA out-pile internal memories
Including：In the form of byte stream current queries data are retrieved in JAVA out-pile internal memories；Current queries data are changed into character
The current queries data of string form, and obtain the current queries data of character string forms.
, can be by recent inquiry data Cun Chudao JAVA virtual machine internal memories in order to further lift the inquiry velocity of data
In, likewise, the data volume of storable inquiry data can be modified by configuration file in JAVA virtual machine internal memory, and
And when the data volume of the inquiry data stored in virtual memory reaches maximum amount of data, it can be replaced by lru algorithm
Change.Moreover, it is noted that when the inquiry data being stored in JAVA virtual machine internal memory are not within a certain period of time by again
In use, the inquiry data dump is then gone out into JAVA virtual machine internal memory.Specifically, preferably embodiment, above-mentioned data
The method of storage also includes：Each inquiry data are write into JAVA virtual machine internal memory, also, if in JAVA virtual machine internal memory
The inquiry data of storage are not used in preset time, then remove the inquiry data stored in JAVA virtual machine internal memory.
It is described in detail above for a kind of embodiment of date storage method provided by the invention, the present invention is also
A kind of data storage device corresponding with this method is provided, due to the embodiment and the embodiment phase of method part of device part
Mutually correlate, therefore the embodiment of device part refers to the description of the embodiment of method part, wouldn't repeat here.
Fig. 6 is a kind of structure chart of data storage device provided in an embodiment of the present invention.As shown in figure 5, data storage fills
Put including：
Module 60 is established, for before current data writes, when without piecemeal is not filled with, establishing and including in table subregion
There is the piecemeal of row storage file；
Writing module 61, for current data to be split by row, and current data is pressed and arranges concurrently write-in when forefront stores
File；
Block index module 62 is created, after being write in current data, is established in current piecemeal and each current
Block index corresponding to row storage file, and record works as maximum data, the minimum number in the storage file of forefront in block index
According to, the average value of canned data total amount, empty data total amount, each canned data sum and canned data；
The index module that creates the division 63, for write-in data all write-in after, in current table subregion establish with
Subregion index corresponding to specified partition section in current table subregion, and the maximum number in subregion index in record specified partition section
According to and minimum data.
As can be seen here, the data storage device that the present embodiment provides, when carrying out data storage, in being indexed due to subregion only
Maximum data, the minimum data in the specified partition section of table subregion are stored, block index is only stored in row storage file
Maximum data, minimum data, canned data total amount, empty data total amount, the average value of each canned data sum and canned data, institute
It is very small with the memory space shared by subregion index and block index, when mass data storage is carried out, index text
Part can also be stored in internal memory substantially, avoid due to index file is huge and caused by internal memory frequently replace.Therefore, when with
When family carries out data query, fast positioning directly can be indexed to the table subregion for meeting querying condition using the subregion in internal memory,
After table subregion is navigated to, the block index fast positioning being stored in internal memory is recycled to store text to the row for meeting querying condition
Part, the data for meeting querying condition are only then retrieved in related column storage file.So number that the present embodiment provides
According to storage device, when carrying out mass data storage, can not only be improved by the way that index file is completely stored in internal memory
The inquiry velocity of data, the range of search of data by the only query and search data in related column storage file, can also be reduced,
Further increase the inquiry velocity of data.
Date storage method provided by the present invention and device are described in detail above.Each implementation in specification
Example is described by the way of progressive, and each embodiment stresses it is difference with other embodiments, each embodiment
Between identical similar portion mutually referring to.
It should be pointed out that for those skilled in the art, under the premise without departing from the principles of the invention,
Some improvement and modification can also be carried out to the present invention, these are improved and modification also falls into the protection domain of the claims in the present invention
It is interior.
It should also be noted that, in this manual, such as first and second etc relational terms are used merely to one
Individual entity either operates to be made a distinction with another entity or operation, and is not necessarily required and either implied these entities or behaviour
Any this actual relation or order between work be present.Moreover, term " comprising ", "comprising" or its any variant are intended to
Cover including for nonexcludability, so that process, method, article or equipment including a series of key element not only include that
A little key elements, but also other key elements including being not expressly set out, either also include for this process, method, article or set
Standby intrinsic key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that
Other identical element in the process including the key element, method, article or equipment also be present.
Claims (10)
- A kind of 1. date storage method, it is characterised in that including：Before current data write-in, when without piecemeal is not filled with, the piecemeal for including row storage file is established in table subregion；The current data is split by row, and by row, concurrently forefront storage file is worked as in write-in by the current data；After the current data writes, established in current piecemeal and described work as the corresponding piecemeal of forefront storage file with each Index, and the record maximum data worked as in the storage file of forefront, minimum data, canned data are total in the block index The average value of amount, empty data total amount, each canned data sum and the canned data；After write-in data all write-in, established in current table subregion and the specified partition section in the current table subregion Corresponding subregion index, and maximum data and minimum data in the specified partition section is recorded during the subregion indexes.
- 2. date storage method according to claim 1, it is characterised in that it is described when without piecemeal is not filled with, in table point Established in area and include the piecemeal of row storage file and specifically include：When without table subregion is not filled with, newly-built current table subregion, and establish in the current table subregion and to include the row and deposit Store up the piecemeal of file.
- 3. date storage method according to claim 1, it is characterised in that also include：After the write-in of said write data finishes, the file data being had been written under currently stored catalogue is scanned；When scanning is stored with uncompressed data to the row storage file being filled with piecemeal, the uncompressed number is compressed According to.
- 4. according to the date storage method described in claim 1-3 any one, it is characterised in that also include：After the current data writes, calculate described when the difference of maximum data and minimum data in the storage file of forefront Value, and the difference is evenly dividing as N number of range section；Established in the current piecemeal with it is each it is described work as the corresponding scope segment index of forefront storage file, and in the range section In index, the mark distribution situation for working as numeric type data in the storage file of forefront；Wherein, N is positive integer.
- 5. date storage method according to claim 4, it is characterised in that also include：After the current data writes, established in the current piecemeal and each described to work as forefront storage file corresponding Character position index, and in the character position index, the mark distribution situation for working as character type data in the storage file of forefront.
- 6. date storage method according to claim 5, it is characterised in that also include：After the current data writes, will with described when the corresponding each index of forefront storage file, to be loaded onto JAVA virtual Machine internal memory；When receiving inquiry request, if be stored with the JAVA virtual machine internal memory and the row where current queries data The index corresponding to storage file, then go out according to the indexed search stored in the JAVA virtual machine internal memory described current Inquire about data.
- 7. date storage method according to claim 3, it is characterised in that also include：After inquiry data are obtained, by all decompression data storages in the row storage file where the inquiry data To JAVA out-pile internal memories；When receiving inquiry request, if being stored with current queries data in the JAVA out-piles internal memory, from the JAVA The current queries data are obtained in out-pile internal memory.
- 8. date storage method according to claim 7, it is characterised in that it is described by it is described inquiry data where described in All decompression data storages to JAVA out-pile internal memories in row storage file specifically include：The decompression data are stored in the form of byte stream to the JAVA out-piles internal memory；It is then corresponding, it is described the current queries data are obtained from the JAVA out-piles internal memory to specifically include：The current queries data are retrieved in the JAVA out-piles internal memory in the form of the byte stream；The current queries data are changed into the current queries data of character string forms, and obtained the character string forms Current queries data.
- 9. date storage method according to claim 5, it is characterised in that also include：After inquiry data are obtained, the inquiry data are write into JAVA virtual machine internal memory, and if the JAVA virtual machine The inquiry data stored in internal memory are not used in preset time, then remove the institute stored in the JAVA virtual machine internal memory State inquiry data；It is then corresponding, when receiving the inquiry request for obtaining the inquiry data in preset time, directly from the JAVA The inquiry data are obtained in out-pile internal memory.
- A kind of 10. data storage device, it is characterised in that including：Module is established, for before current data writes, when without piecemeal is not filled with, is established in table subregion and includes row and deposit Store up the piecemeal of file；Writing module, for the current data to be split by row, and the current data is concurrently write when forefront is deposited by row Store up file；Create block index module, for the current data write after, in current piecemeal establish with it is each it is described work as Block index corresponding to the storage file of forefront, and the maximum number worked as in the storage file of forefront is recorded in the block index According to, minimum data, canned data total amount, empty data total amount, each canned data sum and the canned data average value；Create the division index module, for after write-in data all write-in, establishing in current table subregion and working as with described Subregion corresponding to specified partition section in preceding table subregion indexes, and is recorded in subregion index in the specified partition section Maximum data and minimum data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710842915.0A CN107577436B (en) | 2017-09-18 | 2017-09-18 | Data storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710842915.0A CN107577436B (en) | 2017-09-18 | 2017-09-18 | Data storage method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107577436A true CN107577436A (en) | 2018-01-12 |
| CN107577436B CN107577436B (en) | 2020-07-07 |
Family
ID=61036045
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710842915.0A Active CN107577436B (en) | 2017-09-18 | 2017-09-18 | Data storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107577436B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108256064A (en) * | 2018-01-16 | 2018-07-06 | 北京三快在线科技有限公司 | A kind of data search method and device |
| CN109325031A (en) * | 2018-09-13 | 2019-02-12 | 上海达梦数据库有限公司 | A kind of data statistical approach, device, equipment and storage medium |
| CN109815241A (en) * | 2019-01-31 | 2019-05-28 | 上海达梦数据库有限公司 | Data query method, apparatus, equipment and storage medium |
| CN110442576A (en) * | 2019-07-02 | 2019-11-12 | 北京奇艺世纪科技有限公司 | Data query method, apparatus, server and computer-readable medium |
| CN110555037A (en) * | 2019-09-12 | 2019-12-10 | 苏州百宝箱科技有限公司 | Smart city data sharing system |
| CN110704433A (en) * | 2019-09-23 | 2020-01-17 | 北京优炫软件股份有限公司 | Brin index construction method of columnar storage data, data retrieval method and device |
| CN110755063A (en) * | 2018-10-06 | 2020-02-07 | 江苏创越医疗科技有限公司 | Low-delay electrocardiogram drawing method |
| WO2021179782A1 (en) * | 2020-03-13 | 2021-09-16 | 苏州浪潮智能科技有限公司 | Method, device and apparatus for improving execution efficiency of database appliance, and medium |
| CN113722623A (en) * | 2021-09-03 | 2021-11-30 | 锐掣(杭州)科技有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN117234436A (en) * | 2023-11-14 | 2023-12-15 | 苏州元脑智能科技有限公司 | Method, device, storage system and product for expanding capacity of disk array |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110219020A1 (en) * | 2010-03-08 | 2011-09-08 | Oks Artem A | Columnar storage of a database index |
| CN104376119A (en) * | 2014-12-03 | 2015-02-25 | 天津南大通用数据技术股份有限公司 | Data access method and device adapted to super-large scale column-storage database |
| US20150106382A1 (en) * | 2013-10-11 | 2015-04-16 | Oracle International Corporation | Tables With Unlimited Number Of Sparse Columns And Techniques For An Efficient Implementation |
| CN104834650A (en) * | 2014-02-12 | 2015-08-12 | 清华大学 | Method and system for generating effective query tasks |
| CN105408857A (en) * | 2013-07-29 | 2016-03-16 | 亚马逊科技公司 | Generating a multi-column index for relational databases by interleaving data bits for selectivity |
| CN106844541A (en) * | 2016-12-30 | 2017-06-13 | 晶赞广告（上海）有限公司 | A kind of on-line analytical processing method and device |
-
2017
- 2017-09-18 CN CN201710842915.0A patent/CN107577436B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110219020A1 (en) * | 2010-03-08 | 2011-09-08 | Oks Artem A | Columnar storage of a database index |
| CN105408857A (en) * | 2013-07-29 | 2016-03-16 | 亚马逊科技公司 | Generating a multi-column index for relational databases by interleaving data bits for selectivity |
| US20150106382A1 (en) * | 2013-10-11 | 2015-04-16 | Oracle International Corporation | Tables With Unlimited Number Of Sparse Columns And Techniques For An Efficient Implementation |
| CN104834650A (en) * | 2014-02-12 | 2015-08-12 | 清华大学 | Method and system for generating effective query tasks |
| CN104376119A (en) * | 2014-12-03 | 2015-02-25 | 天津南大通用数据技术股份有限公司 | Data access method and device adapted to super-large scale column-storage database |
| CN106844541A (en) * | 2016-12-30 | 2017-06-13 | 晶赞广告（上海）有限公司 | A kind of on-line analytical processing method and device |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108256064A (en) * | 2018-01-16 | 2018-07-06 | 北京三快在线科技有限公司 | A kind of data search method and device |
| CN109325031B (en) * | 2018-09-13 | 2021-08-03 | 上海达梦数据库有限公司 | Data statistical method, device, equipment and storage medium |
| CN109325031A (en) * | 2018-09-13 | 2019-02-12 | 上海达梦数据库有限公司 | A kind of data statistical approach, device, equipment and storage medium |
| CN110755063B (en) * | 2018-10-06 | 2023-06-02 | 江苏创越医疗科技有限公司 | Low-delay electrocardiogram drawing method |
| CN110755063A (en) * | 2018-10-06 | 2020-02-07 | 江苏创越医疗科技有限公司 | Low-delay electrocardiogram drawing method |
| CN109815241B (en) * | 2019-01-31 | 2021-05-11 | 上海达梦数据库有限公司 | Data query method, device, equipment and storage medium |
| CN109815241A (en) * | 2019-01-31 | 2019-05-28 | 上海达梦数据库有限公司 | Data query method, apparatus, equipment and storage medium |
| CN110442576A (en) * | 2019-07-02 | 2019-11-12 | 北京奇艺世纪科技有限公司 | Data query method, apparatus, server and computer-readable medium |
| CN110555037A (en) * | 2019-09-12 | 2019-12-10 | 苏州百宝箱科技有限公司 | Smart city data sharing system |
| CN110704433A (en) * | 2019-09-23 | 2020-01-17 | 北京优炫软件股份有限公司 | Brin index construction method of columnar storage data, data retrieval method and device |
| CN110704433B (en) * | 2019-09-23 | 2023-03-28 | 北京优炫软件股份有限公司 | Brin index construction method of columnar storage data, data retrieval method and device |
| WO2021179782A1 (en) * | 2020-03-13 | 2021-09-16 | 苏州浪潮智能科技有限公司 | Method, device and apparatus for improving execution efficiency of database appliance, and medium |
| CN113722623A (en) * | 2021-09-03 | 2021-11-30 | 锐掣(杭州)科技有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113722623B (en) * | 2021-09-03 | 2024-07-05 | 锐掣(杭州)科技有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN117234436A (en) * | 2023-11-14 | 2023-12-15 | 苏州元脑智能科技有限公司 | Method, device, storage system and product for expanding capacity of disk array |
| CN117234436B (en) * | 2023-11-14 | 2024-02-20 | 苏州元脑智能科技有限公司 | Method, device, storage system and product for expanding capacity of disk array |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107577436B (en) | 2020-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107577436A (en) | A kind of date storage method and device | |
| US10664497B2 (en) | Hybrid database table stored as both row and column store | |
| US11238098B2 (en) | Heterogenous key-value sets in tree database | |
| US10346383B2 (en) | Hybrid database table stored as both row and column store | |
| CN102521334B (en) | Data storage and query method based on classification characteristics and balanced binary tree | |
| US8768927B2 (en) | Hybrid database table stored as both row and column store | |
| CN103488709B (en) | A kind of index establishing method and system, search method and system | |
| CN102339315B (en) | Index updating method and system of advertisement data | |
| US9047330B2 (en) | Index compression in databases | |
| US9672241B2 (en) | Representing an outlier value in a non-nullable column as null in metadata | |
| CN110383261A (en) | Stream for multithread storage device selects | |
| CN110291518A (en) | Merge tree garbage index | |
| CN110268399A (en) | Merging tree for attended operation is modified | |
| CN104361113B (en) | A kind of OLAP query optimization method under internal memory flash memory mixing memory module | |
| CN102214176B (en) | Method for splitting and join of huge dimension table | |
| KR101400214B1 (en) | Appratus for providing a hybrid c interface | |
| US9977801B2 (en) | Paged column dictionary | |
| CN106599040A (en) | Layered indexing method and search method for cloud storage | |
| CN110309233A (en) | Method, apparatus, server and the storage medium of data storage | |
| US20140222828A1 (en) | Columnwise Storage of Point Data | |
| CN104750727A (en) | Column type memory storage and query device and column type memory storage and query method | |
| CN103530111A (en) | Process definition acquiring method and device | |
| CN106202408B (en) | Data query server based on OLAP, system and method | |
| CN114841806A (en) | Mode matching system under real-time transfer link | |
| CN106776772A (en) | A kind of method and device of data retrieval |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |