CN107506497A - A kind of database data access method and device - Google Patents
A kind of database data access method and device Download PDFInfo
- Publication number
- CN107506497A CN107506497A CN201710898775.9A CN201710898775A CN107506497A CN 107506497 A CN107506497 A CN 107506497A CN 201710898775 A CN201710898775 A CN 201710898775A CN 107506497 A CN107506497 A CN 107506497A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- manipulation sentence
- storehouse
- parsing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of database data access method and device, method to include：The data manipulation sentence issued is parsed, obtains parsing data；According to parsing data and pre-configured burst rule, the data manipulation sentence is routed into corresponding data source in database, the burst rule includes distinguishing key element division storehouse with the first of user data, key element division table is distinguished in storehouse with the second of user data, to user data burst；The data manipulation sentence is performed, the data of acquisition return after having performed.Database data access method of the present invention and device, divide table mechanism using a point storehouse to the burst rule of database, the query rate to data in database can be improved, database data access performance can be improved, lift data access rate.
Description
Technical field
The present invention relates to database storage techniques field, more particularly to a kind of database data access method and device.
Background technology
Relevant database refers to employ the database that relational model carrys out tissue data, and in simple terms, relational model is just
Refer to two-dimensional table model, a relevant database be exactly by bivariate table and its between a data group forming of contact
Knit.
As cloud computing develops, data volume is continuously increased, and mass data and high concurrent business processing are used relationship type
The application system of database proposes new test, although the appearance for having non-relational database provides when solving such problem
Solution, but based on relevant database have stable storage, inquiry flexibly, the characteristic such as compatibility is good, be still most
The preference data storehouse of number application side.And existing relevant database in face of mass data and voice and packet data concurrent service access when, data are deposited
Take performance low.
The content of the invention
It is an object of the invention to provide a kind of database data access method and device.Improve database data accessibility
Can, improve data access rate.
To achieve the above object, the present invention provides following technical scheme：
A kind of database data access method, including：
The data manipulation sentence issued is parsed, obtains parsing data；
According to the parsing data and pre-configured burst rule, the data manipulation sentence is routed in database
Corresponding data source, the burst rule includes distinguishing key element division storehouse with the first of user data, with user data in storehouse
Second distinguish key element division table, to user data burst；
The data manipulation sentence is performed, the data of acquisition return after having performed.
Alternatively, using the distributed data processing system of increasing income handled in real time for big data, the data issued are grasped
Parsed as sentence.
Alternatively, using single table method for routing, contingency table method for routing or cartesian product method for routing, using parsing number
Corresponding True Data source in database is routed to according to by the data manipulation sentence.
Alternatively, the execution data manipulation sentence includes：The data manipulation language is performed using multi-thread concurrent
Sentence.
Alternatively, in addition to：Multiple data manipulation sentences are together loaded, are once sent to database execution.
A kind of database data access mechanism, including：
Parsing module, for being parsed to the data manipulation sentence issued, obtain parsing data；
Routing module, for regular according to the parsing data and pre-configured burst, by the data manipulation sentence
Corresponding data source in database is routed to, the burst rule includes distinguishing key element division storehouse with the first of user data,
Key element division table is distinguished with the second of user data in storehouse, to user data burst；
Execution module, for performing the data manipulation sentence, the data of acquisition return after having performed.
Alternatively, the parsing module, which uses, is used for the distributed data processing system of increasing income that big data is handled in real time, right
The data manipulation sentence issued is parsed.
Alternatively, the routing module uses single table method for routing, contingency table method for routing or cartesian product route side
Method, the data manipulation sentence is routed into corresponding True Data source in database using data are parsed.
Alternatively, the execution module is specifically used for performing the data manipulation sentence using multi-thread concurrent.
As shown from the above technical solution, database data access method provided by the present invention and device, first to issuing
Data manipulation sentence parsed, obtain parsing data, then according to parsing data and pre-configured burst rule, by institute
State data manipulation sentence and route to corresponding data source in database, burst rule is included with the first of user data in this method
Key element division storehouse is distinguished, key element division table is distinguished with the second of user data in storehouse, to user data burst, finally performs number
According to action statement, the data of acquisition return after having performed.Database data access method of the present invention and device, to database
Burst rule divides table mechanism using a point storehouse, can improve the query rate to data in database, can improve database data and deposit
Performance is taken, lifts data access rate.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is a kind of flow chart of database data access method provided in an embodiment of the present invention；
Fig. 2 is a kind of schematic diagram of database data access mechanism provided in an embodiment of the present invention.
Embodiment
In order that those skilled in the art more fully understand the technical scheme in the present invention, below in conjunction with of the invention real
The accompanying drawing in example is applied, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described implementation
Example only part of the embodiment of the present invention, rather than whole embodiments.It is common based on the embodiment in the present invention, this area
The every other embodiment that technical staff is obtained under the premise of creative work is not made, should all belong to protection of the present invention
Scope.
Fig. 1 is refer to, a kind of database data access method provided in an embodiment of the present invention, is comprised the following steps：
S10：The data manipulation sentence issued is parsed, obtains parsing data.
The data manipulation sentence SQL issued is parsed, obtained parsing data include the data manipulation sentence SQL
The information for the data to be accessed.
In the specific implementation, corresponding to user data in database are carried out with inquiry operation, insertion operation or is deleted to grasp
Make etc., data manipulation sentence can be that data query instruction, data inserting instruction or data delete instruction etc..
Preferably, used in this method and be used for the distributed data processing system of increasing income that big data is handled in real time, to issuing
Data manipulation sentence parsed.It is weighing apparatus because the parsing to data action statement SQL is the core that database data accesses
An application system performance and the most important index of compatibility are measured, distributed data processing system conduct of increasing income is used in this method
Resolver, ensure the performance of application system, for example Druid database connection pools can be used.
S11：According to the parsing data and pre-configured burst rule, the data manipulation sentence is routed into data
Corresponding data source in storehouse, the burst rule includes distinguishing key element division storehouse with the first of user data, with data in storehouse
Second distinguish key element division table, to user data burst.
In this method, the burst rule of use includes：Key element division storehouse is distinguished with the first of user data, with number in storehouse
According to second distinguish key element division table, to user data burst.Key element is distinguished for user data, first based on user data
First divide storehouse, in the storehouse of division, second based on user data distinguishes key element division table.In the specific implementation, point storehouse point is carried out
First differentiation key element of table and the second differentiation key element can flexibly be set according to actual conditions.
Such as the order data for user, storehouse can be divided according to ID first, the order numbers of same user will be belonged to
According to being divided in a storehouse；In the storehouse of corresponding ID, table can be divided according to order ID, by corresponding to same order ID phase
Data are closed to be recorded in same table.
Again for example, for the billing data of user, storehouse can be divided according to the time first, the billing data in same time is drawn
Divide in same storehouse；Then the subscriber's account data of same January can be recorded in by same table with month+ID division table
In preserved.
According to the data for parsing to obtain to data action statement SQL, according to burst rule by the data manipulation sentence SQL
Route to corresponding True Data source in database.
In the specific implementation, single table method for routing, contingency table method for routing or cartesian product method for routing, root can be used
Data manipulation sentence is routed into corresponding True Data source in database according to obtained parsing data.Single table route is the simplest
It is single, but route results not necessarily fall into unique storehouse or table, because supporting that the operator according to as between and in is divided
Piece, so final result still may be fallen into multiple storehouses or table.Contingency table can be regarded as a point storehouse and divide table rule completely the same
Master-salve table, illustrate：Order table and order detail table are all homogeneous as burst key, any time burst logic according to order ID
Together.Such correlation inquiry and single table inquiry difficulty and performance are suitable.Cartesian product inquiry is the most complicated, because can not be according to pass
The uniformity of connection relation positioning burst rule, is performed so the correlation inquiry of dereferenced table needs to disassemble to combine for cartesian product.
Query performance is relatively low, and database connection number is higher, needs to use with caution when in use.
S12：The data manipulation sentence is performed, the data obtained after the completion of execution are returned.
In the specific implementation, data manipulation sentence SQL is routed in database behind True Data source, multithreading can be used
Data manipulation sentence SQL is concurrently performed, sentence execution efficiency can be improved, improve the access rate to database.
It is further preferred that can together load multiple data manipulation sentence SQL, database execution is once sent to,
Realize batch processing and execution, it is possible to increase sentence performs speed, improves database data access performance.
After data manipulation sentence has performed, Query Result is subjected to merger processing, and return to the data of user's needs.
The present embodiment database data access method, database burst rule divide table mechanism using a point storehouse.It can improve pair
The query rate of data in database, database data access performance is improved, lift data access rate；Divide by using storehouse is divided
Table mechanism, the data of magnanimity are subjected to burst storage, avoid in a table that data volume is excessive to be caused to access the problem of slow, carried
Application system performance is risen.
In the specific implementation, this method can use database connection (Java Data Base Connectivity, java,
JDBC) framework of open source projects combination database itself is realized.
Accordingly, Fig. 2 is refer to, the embodiment of the present invention also provides a kind of database data access mechanism, including：
Parsing module 20, for being parsed to the data manipulation sentence issued, obtain parsing data；
Routing module 21, for regular according to the parsing data and pre-configured burst, by the data manipulation language
Sentence routes to corresponding data source in database, and the burst rule includes distinguishing key element division storehouse with the first of user data,
Key element division table is distinguished with the second of user data in storehouse, to user data burst；
Execution module 22, for performing the data manipulation sentence, the data of acquisition return after having performed.
As can be seen that the present embodiment database data access mechanism, is parsed to the data manipulation sentence issued first,
Parsing data are obtained, then according to parsing data and pre-configured burst rule, the data manipulation sentence is routed into number
According to corresponding data source in storehouse, burst rule includes distinguishing key element division storehouse with the first of user data in this method, in storehouse
Key element division table is distinguished with the second of user data, to user data burst, data manipulation sentence is finally performed, after having performed
The data of acquisition return.The present embodiment database data access mechanism, divide table mechanism using a point storehouse to the burst rule of database,
The query rate to data in database can be improved, database data access performance can be improved, lifts data access rate.It is logical
Cross and divide table mechanism using a point storehouse, the data of magnanimity are subjected to burst storage, avoid in a table that data volume is excessive to be caused to access
The problem of slow, improve application system performance.
Preferably, in the present embodiment database data access mechanism, the parsing module 20 is real-time using big data is used for
The distributed data processing system of increasing income of processing, is parsed to the data manipulation sentence issued.
Optionally, the routing module 21 can use single table method for routing, contingency table method for routing or cartesian product road
By method, the data manipulation sentence is routed into corresponding True Data source in database using data are parsed.
It is further preferred that the execution module 22 is specifically used for performing the data manipulation language using multi-thread concurrent
Sentence.Sentence execution efficiency can be improved, lifts the access rate to database.Further, can be by multiple data manipulation sentences
SQL is together loaded, and is once sent to database execution, is realized batch processing and execution, it is possible to increase sentence performs speed, improves number
According to storehouse data access performance.
The present embodiment database data access method and device, burst rule divide table mechanism using a point storehouse, can be effectively
Lift the storage capacity of database mass data, increase handling capacity, lift the access rate of user, improve platform stability and
Robustness, improve product competitiveness.
A kind of database data access method provided by the present invention and device are described in detail above.Herein
Apply specific case to be set forth the principle and embodiment of the present invention, the explanation of above example is only intended to help
Understand the method and its core concept of the present invention.It should be pointed out that for those skilled in the art, do not taking off
On the premise of from the principle of the invention, some improvement and modification can also be carried out to the present invention, these are improved and modification also falls into this
In invention scope of the claims.
Claims (9)
- A kind of 1. database data access method, it is characterised in that including：The data manipulation sentence issued is parsed, obtains parsing data；According to the parsing data and pre-configured burst rule, the data manipulation sentence is routed to corresponding in database Data source, burst rule includes distinguishing key element division storehouse with the first of user data, with the of user data in storehouse Two distinguish key element division table, to user data burst；The data manipulation sentence is performed, the data of acquisition return after having performed.
- 2. database data access method according to claim 1, it is characterised in that use and handled in real time for big data Distributed data processing system of increasing income, the data manipulation sentence issued is parsed.
- 3. database data access method according to claim 1, it is characterised in that using single table method for routing, association Table method for routing or cartesian product method for routing, it is right in database using parsing data by the data manipulation sentence and routing to The True Data source answered.
- 4. according to the database data access method described in claim any one of 1-3, it is characterised in that described to perform the number Include according to action statement：The data manipulation sentence is performed using multi-thread concurrent.
- 5. database data access method according to claim 4, it is characterised in that also include：By multiple data Action statement together loads, and is once sent to database execution.
- A kind of 6. database data access mechanism, it is characterised in that including：Parsing module, for being parsed to the data manipulation sentence issued, obtain parsing data；Routing module, for according to the parsing data and pre-configured burst rule, the data manipulation sentence to be route The corresponding data source into database, the burst rule includes distinguishing key element division storehouse with the first of user data, in storehouse Key element division table is distinguished with the second of user data, to user data burst；Execution module, for performing the data manipulation sentence, the data of acquisition return after having performed.
- 7. database data access mechanism according to claim 6, it is characterised in that the parsing module, which uses, to be used for greatly The distributed data processing system of increasing income of generating date, the data manipulation sentence issued is parsed.
- 8. database data access mechanism according to claim 6, it is characterised in that the routing module uses Dan Biao roads By method, contingency table method for routing or cartesian product method for routing, the data manipulation sentence is route using data are parsed The corresponding True Data source into database.
- 9. according to the database data access mechanism described in claim any one of 6-8, it is characterised in that the execution module tool Body is used to perform the data manipulation sentence using multi-thread concurrent.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710898775.9A CN107506497A (en) | 2017-09-28 | 2017-09-28 | A kind of database data access method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710898775.9A CN107506497A (en) | 2017-09-28 | 2017-09-28 | A kind of database data access method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107506497A true CN107506497A (en) | 2017-12-22 |
Family
ID=60699005
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710898775.9A Pending CN107506497A (en) | 2017-09-28 | 2017-09-28 | A kind of database data access method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107506497A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110851474A (en) * | 2018-07-26 | 2020-02-28 | 深圳市优必选科技有限公司 | Data query method, database middleware, data query device and storage medium |
| CN110955666A (en) * | 2019-12-17 | 2020-04-03 | 用友网络科技股份有限公司 | Data processing method and data processing device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140201192A1 (en) * | 2013-01-15 | 2014-07-17 | Syscom Computer Engineering Co. | Automatic data index establishment method |
| CN105930387A (en) * | 2016-04-14 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Data operation system and method based on data routing and sharding |
| CN106528579A (en) * | 2015-09-15 | 2017-03-22 | 阿里巴巴集团控股有限公司 | Search method, device and system based on sharding structure databases |
| CN107122365A (en) * | 2016-02-25 | 2017-09-01 | 阿里巴巴集团控股有限公司 | The access method and device of heterogeneous database |
| CN107180081A (en) * | 2017-04-28 | 2017-09-19 | 广东亿迅科技有限公司 | The call method and device of massive relation type database |
-
2017
- 2017-09-28 CN CN201710898775.9A patent/CN107506497A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140201192A1 (en) * | 2013-01-15 | 2014-07-17 | Syscom Computer Engineering Co. | Automatic data index establishment method |
| CN106528579A (en) * | 2015-09-15 | 2017-03-22 | 阿里巴巴集团控股有限公司 | Search method, device and system based on sharding structure databases |
| CN107122365A (en) * | 2016-02-25 | 2017-09-01 | 阿里巴巴集团控股有限公司 | The access method and device of heterogeneous database |
| CN105930387A (en) * | 2016-04-14 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Data operation system and method based on data routing and sharding |
| CN107180081A (en) * | 2017-04-28 | 2017-09-19 | 广东亿迅科技有限公司 | The call method and device of massive relation type database |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110851474A (en) * | 2018-07-26 | 2020-02-28 | 深圳市优必选科技有限公司 | Data query method, database middleware, data query device and storage medium |
| CN110955666A (en) * | 2019-12-17 | 2020-04-03 | 用友网络科技股份有限公司 | Data processing method and data processing device |
| CN110955666B (en) * | 2019-12-17 | 2023-10-03 | 用友网络科技股份有限公司 | Data processing method and data processing device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103902653B (en) | A kind of method and apparatus for building data warehouse table genetic connection figure | |
| CN107402988A (en) | A kind of distributed NewSQL Database Systems and Query semi-structured for data method | |
| CN103177056B (en) | It is stored as row storage and row stores the hybrid database table of the two | |
| CN103366015B (en) | A kind of OLAP data based on Hadoop stores and querying method | |
| CN103902702B (en) | A kind of data-storage system and storage method | |
| CN104112011B (en) | The method and device that a kind of mass data is extracted | |
| CN110362572A (en) | A kind of time series database system based on column storage | |
| CN104090962B (en) | Towards the nested query method of magnanimity distributed data base | |
| CN109344153A (en) | The processing method and terminal device of business datum | |
| KR101400214B1 (en) | Appratus for providing a hybrid c interface | |
| CN110222029A (en) | A kind of big data multidimensional analysis computational efficiency method for improving and system | |
| CN106201537A (en) | A kind of data processing method and device | |
| CN107943952A (en) | A kind of implementation method that full-text search is carried out based on Spark frames | |
| CN106095863B (en) | A kind of multidimensional data query and storage system and method | |
| CN106055621A (en) | Log retrieval method and device | |
| CN108875042A (en) | A kind of mixing on-line analysing processing system and data query method | |
| CN108536692A (en) | A kind of generation method of executive plan, device and database server | |
| CN108388575A (en) | A kind of appraisal procedure, device, terminal device and the storage medium of settlement of insurance claim expense | |
| CN107577436A (en) | A kind of date storage method and device | |
| CN107506497A (en) | A kind of database data access method and device | |
| CN107193898A (en) | The inquiry sharing method and system of log data stream based on stepped multiplexing | |
| CN104809246B (en) | The processing method and processing device of charge data | |
| CN106325756A (en) | Data storage and data computation methods and devices | |
| CN102915344B (en) | SQL (structured query language) statement processing method and device | |
| CN107784030A (en) | A kind of method and device for handling Connection inquiring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171222 |