CN109992617B - Data access method and system supporting multiple programming languages and multiple data sources - Google Patents
Data access method and system supporting multiple programming languages and multiple data sources Download PDFInfo
- Publication number
- CN109992617B CN109992617B CN201910215253.3A CN201910215253A CN109992617B CN 109992617 B CN109992617 B CN 109992617B CN 201910215253 A CN201910215253 A CN 201910215253A CN 109992617 B CN109992617 B CN 109992617B
- Authority
- CN
- China
- Prior art keywords
- data
- access
- connection
- request
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 3
- 238000007726 management method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a data access method and a data access system supporting multiple programming languages and multiple data sources. The method converts the input request parameters into the access request based on the preset rules, thereby solving the problem of cross-programming language; the access request is analyzed based on the codes, so that the problem of multi-platform compatibility is solved; due to the multi-source connection pool, the problem that management of connection handles of multiple data sources is complex is solved, and multiple data sources are accessed.
Description
Technical Field
The invention relates to a data access method and a data access system supporting multiple programming languages and multiple data sources, and belongs to the technical field of automatic power grid model management of power systems.
Background
With the increasing complexity of power grid data storage, the calling modes of system service data are more and more diversified, and the requirements for standardizing and simplifying the access modes of each service data are increasingly highlighted. There is therefore a need for a uniform data access across languages, across data sources.
Disclosure of Invention
The invention provides a data access method and a data access system supporting multiple programming languages and multiple data sources, and solves the problems of the existing data access.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a method of data access, comprising,
converting the input request parameters into access requests according to preset rules, and sending the access requests to a data server; the access request comprises all request parameters, and the request parameters comprise data source connectors, access statements, return formats and codes;
and receiving data feedback of the data server.
When the access request is sent, if the data server is available, the access request is directly sent, if the data server is unavailable, the input request parameters are written into a buffer area, and after the data server is available, the buffer area request parameters are preferentially converted into the access request and sent.
The encoding is located in the first byte of the access request and the subsequent bytes of the access request are the encoded content.
A method of data access, comprising,
receiving an access request sent by a request terminal; the access request comprises all request parameters, and the request parameters comprise data source connectors, access statements, return formats and codes;
reading the code in the access request, and analyzing the access request according to the code;
and connecting the data source, accessing the data source according to the access statement, and returning data to the request terminal according to the return format.
The specific process of connecting with the data source and accessing the data source to acquire data is,
s1, judging whether a connection pool corresponding to the data source in the access request exists, if so, turning to S3, and if not, turning to S2;
s2, creating a connection pool of the data source and creating a connection, and turning to S6;
s3, judging whether idle connection exists in the connection pool, if yes, obtaining connection, and turning to S6; if not, go to S4;
s4, judging whether the number of active connections in the connection pool reaches the upper limit, if not, creating a new connection, and turning to S6; if yes, go to S5;
s5, judging whether connection overtime exists, if yes, removing the overtime connection, creating a new connection, and turning to S6; if not, waiting until a connection is available, and going to S6;
and S6, accessing the data source according to the access statement and returning the data to the request end according to the return format.
And coding the first byte of the access request, reading the first byte of the access request, and analyzing the subsequent bytes of the access request according to the coding.
The data access system comprises a request end and a data service end which are communicated with each other;
the request end comprises a conversion module and a data receiving module;
a conversion module: converting the input request parameters into access requests according to preset rules, and sending the access requests to a data server; the access request comprises all request parameters, and the request parameters comprise data source connectors, access statements, return formats and codes;
a data receiving module: receiving data feedback of a data server;
the data server comprises an access request receiving module, an analysis module and an access module;
an access request receiving module: receiving an access request sent by a request terminal;
an analysis module: reading the code in the access request, and analyzing the access request according to the code;
an access module: and connecting the data source, accessing the data source according to the access statement, and returning data to the request terminal according to the return format.
The request end also comprises a cache module,
a caching module: when the access request is sent, if the data server is unavailable, the input request parameters are written into a buffer area, and after the data server is available, the buffer area request parameters are preferentially converted into the access request and sent.
The access module comprises a connection pool existence judging module, a connection pool establishing module, an idle connection judging module, an active connection number upper limit judging module, an overtime connection judging module and a data acquiring module;
a connection pool existence judgment module: judging whether a connection pool of the data source in the access request exists, if so, switching to an idle connection judging module, and if not, switching to a connection pool establishing module;
a connection pool creation module: creating a connection pool of the data source, creating a connection, and transferring to a data acquisition module;
an idle connection judgment module: judging whether idle connection exists in the connection pool, if so, acquiring the connection, and switching to the data acquisition module; if not, switching to an upper limit judgment module of the number of active connections;
an upper limit judgment module of the number of active connections: judging whether the number of active connections in the connection pool reaches an upper limit, if not, creating a new connection, and turning to a data acquisition module; if yes, switching to an overtime connection judging module;
the overtime connection judging module: judging whether connection overtime exists or not, if yes, removing the overtime connection, creating a new connection, and transferring to a data acquisition module; if not, waiting until connection is available, and switching to the data acquisition module;
a data acquisition module: and accessing the data source according to the access statement, and returning the data to the request terminal according to the return format.
The code is located in the first byte of the access request, the subsequent byte of the access request is the coded content, the parsing module reads the first byte of the access request and parses the subsequent byte of the access request according to the code.
The invention achieves the following beneficial effects: the method converts the input request parameters into the access request based on the preset rules, thereby solving the problem of cross-programming language; the access request is analyzed based on the codes, so that the problem of multi-platform compatibility is solved; due to the multi-source connection pool, the problem that the management of the multi-data source connection handle is complex is solved, and multi-data source access is realized.
Drawings
FIG. 1 is a flow chart of a requestor;
FIG. 2 is a flow chart of translating access requests;
FIG. 3 is a flow chart of a data server;
FIG. 4 is a flow chart of the resolution;
FIG. 5 is a flow chart of data assembly;
FIG. 6 is a flow chart of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
The data access method supports multiple programming languages and multiple data sources, and in the method, message interaction mainly comprises two ends, namely a request end and a data service end, wherein the request end is mainly used for sending an access request and receiving feedback data, and the data service end is mainly used for receiving the access request, the access data source and the feedback data.
The main flow of the request end is shown in fig. 1, and specifically as follows:
A1) and converting the input request parameters into access requests according to preset rules, and sending the access requests to a data server through a TCP protocol.
When the access request is sent, if the data server is available, the access request is directly sent, if the data server is unavailable, the input request parameters are written into a buffer area, and after the data server is available, the buffer area request parameters are preferentially converted into the access request and sent.
Because different development languages have different access modes to different data sources, considering the compatibility with the elements of uniformity and the like, the access requests need to be unified, as shown in fig. 2, it is considered that a rule is set, and a manually input request parameter is converted into an access request with a uniform format, where the access request includes all request parameters, and the request parameters include a data source connector, an access statement, a return format and a code, where the code is a coding mode of the entire access request, such as UTF-8 code, GBK code, and the like, and for convenience of subsequent parsing, the code is located in a first byte of the access request, and a subsequent byte of the access request is a content after being coded.
For example, the access request is in the form of a string, such as: the data source connector 'mysql:// 10.25.222.31/myDBuser ═ soft & password ═ xxxx', the access statement (query statement) is 'select id from Persons name ═ xxx' ″, the return format is returned by row, the code is UTF-8 code, and the final generated character string style is: 0x01{ "source": rdb "," connect ": mysql://10.25.222.31/myDBuser ═ soft & password ═ xxxx", "sql": select id from Persons name ═ xxx', "dataformat": x query "}; where 0x01 represents UTF-8 encoding.
A2) And receiving data feedback of the data server.
The main flow of the data server is shown in fig. 3, which specifically includes the following steps:
B1) and receiving an access request sent by a request end.
B2) And reading the code in the access request, and analyzing the access request according to the code.
As shown in fig. 4, the first byte of the access request is read, the code is identified, if the code cannot be identified, an error is reported, the illegal content is returned to the requesting end, and if the identification is successful, the subsequent byte of the access request is analyzed according to the code.
B3) And connecting the data source, accessing the data source according to the access statement, and returning data to the request terminal according to the return format.
The specific process of step B3 is as follows:
s1, judging whether a connection pool corresponding to the data source in the access request exists, if so, turning to S3, and if not, turning to S2;
the connection pool is a connection pool of multiple data sources, is not a single data source, adopts keyword mapping, and each keyword maps a connection pool of one data source, and is similar to { "rdb": a relational library connection pool "," rtdb ": a real-time library connection pool", "file": a file source connection pool }
S2, creating a connection pool of the data source and creating a connection, and turning to S6;
s3, judging whether idle connection exists in the connection pool, if so, acquiring connection, and turning to S6; if not, go to S4;
s4, judging whether the number of active connections in the connection pool reaches the upper limit, if not, creating a new connection, and turning to S6; if yes, go to S5;
s5, judging whether connection overtime exists, if yes, removing the overtime connection, creating a new connection, and turning to S6; if not, waiting until a connection is available, and going to S6;
and S6, accessing the data source according to the access statement, and returning the data to the request terminal according to the return format.
Since the data obtained by access is unordered, it needs to be assembled according to a return format, as shown in fig. 5 for example: if the dataformat is xquery (return by line), the data is arranged by line in the format: { "query result" [ { "table _ id":1, "table _ name _ chn" \ u8868\ u4fe1\ u606f \ u8868"}, {" table _ id ":2," table _ name _ chn ": \ u57df \ u4fe1\ u606f \ u8868" }; if dataformat is yuquery, then the data is arranged in columns in the format: { "query result" [ { "table _ id": [1,2] }, { "table _ name _ chn": "[" \\ u8868\ u4fe1\ u606f \ u8868"," \\ u57df \ u4fe1\ u606f \ u8868"] } ] }. Wherein "table _ id" and "table _ name _ chn" are names of query columns, and "1", "2", etc. are specific data contents.
In summary, as shown in fig. 6, the specific flow of the data access method is as follows:
step 1, the request end converts the input request parameters into an access request according to a preset rule and sends the access request to the data server end.
And 2, the data server receives the access request sent by the request end.
And 3, reading the code in the access request by the data server, and analyzing the access request according to the code.
And 4, connecting the data server side with the data source, accessing the data source according to the access statement, and returning data to the request side according to the return format.
And 5, the request end receives the data feedback of the data server end.
The input request parameters are converted into the access requests based on the preset rules, and the data are transmitted by using the standard network protocol, so that the data requests do not depend on a specific programming language, and the problems in multiple aspects such as cross-programming language problems, communication efficiency, communication decoupling and the like are solved; the access request is analyzed based on the codes, various coding formats are supported, and the problem of multi-platform compatibility is solved; due to the multi-source connection pool, the problem that management of connection handles of multiple data sources is complex is solved, and multiple data sources are accessed.
The system corresponding to the data access method comprises a request end and a data service end which are communicated with each other.
The request end comprises a conversion module and a data receiving module.
A conversion module: converting the input request parameters into access requests according to preset rules, and sending the access requests to a data server; the access request comprises all request parameters, and the request parameters comprise a data source connector, an access statement, a return format and a code.
A data receiving module: and receiving data feedback of the data server.
The data server comprises an access request receiving module, an analysis module and an access module.
An access request receiving module: and receiving an access request sent by a request end.
An analysis module: and reading the code in the access request, and analyzing the access request according to the code.
The code is located in the first byte of the access request, the subsequent byte of the access request is the coded content, the parsing module reads the first byte of the access request and parses the subsequent byte of the access request according to the code.
An access module: and connecting the data source, accessing the data source according to the access statement, and returning data to the request terminal according to the return format.
The access module comprises a connection pool existence judging module, a connection pool establishing module, an idle connection judging module, an active connection number upper limit judging module, an overtime connection judging module and a data acquisition module;
a connection pool existence judgment module: judging whether a connection pool corresponding to the data source in the access request exists, if so, switching to an idle connection judgment module, and if not, switching to a connection pool creation module;
a connection pool creation module: creating a connection pool of the data source, creating a connection, and transferring to a data acquisition module;
the idle connection judging module: judging whether idle connection exists in the connection pool, if so, acquiring the connection, and switching to the data acquisition module; if not, switching to an upper limit judgment module of the active connection number;
an upper limit judgment module of the number of active connections: judging whether the number of active connections in the connection pool reaches an upper limit, if not, creating a new connection, and turning to a data acquisition module; if yes, switching to an overtime connection judging module;
the overtime connection judging module: judging whether connection overtime exists or not, if yes, removing the overtime connection, creating a new connection, and transferring to a data acquisition module; if not, waiting until connection is available, and switching to the data acquisition module;
a data acquisition module: and accessing the data source according to the access statement, and returning the data to the request terminal according to the return format.
A cache module: when the access request is sent, if the data server is unavailable, the input request parameters are written into the buffer area, and after the data server is available, the buffer area request parameters are preferentially converted into the access request and sent.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (9)
1. A data access method, characterized by: including a request side method and a data server side method,
the request side method comprises the following steps:
converting the input request parameters into access requests according to preset rules, and sending the access requests to a data server; the access request comprises all request parameters, and the request parameters comprise data source connectors, access statements, return formats and codes;
receiving data feedback of a data server;
the data server side method comprises the following steps:
receiving an access request sent by a request terminal;
reading the code in the access request, and analyzing the access request according to the code;
connecting a data source through a connection pool, accessing the data source according to the access statement, and returning data to the request terminal according to the return format; the connection pool is a connection pool of multiple data sources, is not a single data source, and adopts keyword mapping, and each keyword maps a connection pool of one data source.
2. The data access method of claim 1, wherein: when the access request is sent, if the data server is available, the access request is directly sent, if the data server is unavailable, the input request parameters are written into a buffer area, and after the data server is available, the buffer area request parameters are preferentially converted into the access request and sent.
3. The data access method of claim 1, wherein: the encoding is located in the first byte of the access request and the subsequent bytes of the access request are the encoded content.
4. The data access method of claim 1, wherein: the connection pool is connected with the data source, the specific process of accessing the data source to acquire data is,
s1, judging whether a connection pool corresponding to the data source in the access request exists, if so, turning to S3, and if not, turning to S2;
s2, creating a connection pool of the data source and creating a connection, and turning to S6;
s3, judging whether idle connection exists in the connection pool, if so, acquiring connection, and turning to S6; if not, go to S4;
s4, judging whether the number of active connections in the connection pool reaches the upper limit, if not, creating a new connection, and turning to S6; if yes, go to S5;
s5, judging whether connection overtime exists, if yes, removing the overtime connection, creating a new connection, and turning to S6; if not, waiting until a connection is available, and going to S6;
and S6, accessing the data source according to the access statement and returning the data to the request end according to the return format.
5. The data access method of claim 1, wherein: the code is located in the first byte of the access request, the first byte of the access request is read, and the subsequent bytes of the access request are analyzed according to the code.
6. A data access system, characterized by: the system comprises a request end and a data service end which are communicated with each other;
the request end comprises a conversion module and a data receiving module;
a conversion module: converting the input request parameters into access requests according to preset rules, and sending the access requests to a data server; the access request comprises all request parameters, and the request parameters comprise a data source connector, an access statement, a return format and a code;
a data receiving module: receiving data feedback of a data server;
the data server comprises an access request receiving module, an analysis module and an access module;
an access request receiving module: receiving an access request sent by a request terminal;
an analysis module: reading a code in the access request, and analyzing the access request according to the code;
an access module: connecting a data source through a connection pool, accessing the data source according to the access statement, and returning data to the request terminal according to the return format; the connection pool is a connection pool of multiple data sources, is not a single data source, and adopts keyword mapping, and each keyword maps a connection pool of one data source.
7. The data access system of claim 6, wherein: the request end also comprises a cache module,
a cache module: when the access request is sent, if the data server is unavailable, the input request parameters are written into a buffer area, and after the data server is available, the buffer area request parameters are preferentially converted into the access request and sent.
8. The data access system of claim 6, wherein: the access module comprises a connection pool existence judging module, a connection pool establishing module, an idle connection judging module, an active connection number upper limit judging module, an overtime connection judging module and a data acquiring module;
a connection pool existence judgment module: judging whether a connection pool of the data source in the access request exists, if so, switching to an idle connection judging module, and if not, switching to a connection pool establishing module;
a connection pool creation module: creating a connection pool of the data source, creating a connection, and transferring to a data acquisition module;
an idle connection judgment module: judging whether idle connection exists in the connection pool, if so, acquiring the connection, and switching to the data acquisition module; if not, switching to an upper limit judgment module of the number of active connections;
an active connection number upper limit judging module: judging whether the number of active connections in the connection pool reaches an upper limit, if not, creating a new connection, and turning to a data acquisition module; if yes, switching to an overtime connection judging module;
the overtime connection judging module: judging whether connection overtime exists or not, if yes, removing the overtime connection, creating a new connection, and transferring to a data acquisition module; if not, waiting until connection is available, and switching to a data acquisition module;
a data acquisition module: and accessing the data source according to the access statement, and returning the data to the request terminal according to the return format.
9. The data access system of claim 6, wherein: the code is located in the first byte of the access request, the subsequent byte of the access request is the coded content, the parsing module reads the first byte of the access request and parses the subsequent byte of the access request according to the code.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910215253.3A CN109992617B (en) | 2019-03-21 | 2019-03-21 | Data access method and system supporting multiple programming languages and multiple data sources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910215253.3A CN109992617B (en) | 2019-03-21 | 2019-03-21 | Data access method and system supporting multiple programming languages and multiple data sources |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109992617A CN109992617A (en) | 2019-07-09 |
| CN109992617B true CN109992617B (en) | 2022-08-26 |
Family
ID=67129145
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910215253.3A Active CN109992617B (en) | 2019-03-21 | 2019-03-21 | Data access method and system supporting multiple programming languages and multiple data sources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109992617B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259037B (en) * | 2020-01-14 | 2023-09-19 | 重庆特斯联智慧科技股份有限公司 | Data query method and device based on rule configuration, storage medium and terminal |
| CN114064684B (en) * | 2021-10-13 | 2024-07-19 | 中国电子科技集团公司第十五研究所 | Unified access method for heterogeneous data sources |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107818127A (en) * | 2017-09-09 | 2018-03-20 | 国网浙江省电力公司 | A kind of querying method and system for multi-source data |
-
2019
- 2019-03-21 CN CN201910215253.3A patent/CN109992617B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107818127A (en) * | 2017-09-09 | 2018-03-20 | 国网浙江省电力公司 | A kind of querying method and system for multi-source data |
Non-Patent Citations (3)
| Title |
|---|
| 基于OCI和对象池技术的跨平台Oracle数据源管理中间件的实现;章邢等;《信息化研究》;20180820(第04期);全文 * |
| 基于跨平台的Oracle数据库连接研究;金弟等;《杭州电子科技大学学报》;20060225(第01期);全文 * |
| 连接池访问数据库的研究与应用;戴长秀;《计算机时代》;20171115(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109992617A (en) | 2019-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110933146B (en) | Data conversion method and device and server | |
| JP2680201B2 (en) | Request processing method in database | |
| CN109274773B (en) | Method, device and system for realizing remote service calling | |
| CN111198901B (en) | Unified data service opening method and system based on multiple data sources | |
| CN105786998A (en) | Database middleware system and method for processing data through database middleware system | |
| CN109992617B (en) | Data access method and system supporting multiple programming languages and multiple data sources | |
| CN101609415B (en) | Universal service calling system and method based on middleware | |
| CN109670081B (en) | Method and device for processing service request | |
| CN101340448A (en) | Medical data integration scheme oriented to digital family | |
| CN111046100B (en) | Method and system for synchronizing relational database to non-relational database | |
| EA019680B1 (en) | Service access method and system | |
| CN108038213A (en) | A kind of method of data processing, client, server and system | |
| CN103530425A (en) | Android terminal remote data base access code generation system and method based on object dependence mapping | |
| CN112579610A (en) | Multi-data source structure analysis method, system, terminal device and storage medium | |
| CN102833015B (en) | Method for determining public software interface of satellite automatic testing system | |
| CN107222461A (en) | Communication processing method and device | |
| CN114301988A (en) | Distributed calling method and device, storage medium and electronic equipment | |
| CN117172214A (en) | API file conversion method, system and computer storage medium | |
| CN110069566A (en) | Heterogeneous database synchronization method in a kind of one-way import system | |
| CN117076491A (en) | Data processing method, storage medium and equipment | |
| KR100479333B1 (en) | Registry system and management method for by using uddi web service based on the ebxml registry | |
| CN101655873A (en) | Single sign-on system as well as method and device for inputting and outputting data thereof | |
| CN102314406A (en) | The method and system that is used for the lazy data sequenceization of compunication | |
| CN101986664A (en) | Communication network-based three-level remote monitoring system of medicinal equipment | |
| CN111125207B (en) | Data acquisition method and device, connector and presto engine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |