CN116541455B - Interface data integration method, device, computer equipment and storage medium - Google Patents

Interface data integration method, device, computer equipment and storage medium Download PDF

Info

Publication number
CN116541455B
CN116541455B CN202310825064.4A CN202310825064A CN116541455B CN 116541455 B CN116541455 B CN 116541455B CN 202310825064 A CN202310825064 A CN 202310825064A CN 116541455 B CN116541455 B CN 116541455B
Authority
CN
China
Prior art keywords
data
external
api interface
interface
postgresql
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310825064.4A
Other languages
Chinese (zh)
Other versions
CN116541455A (en
Inventor
柳遵梁
陈波
王利强
闻建霞
汤国胜
张�浩
王伟峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Meichuang Technology Co ltd
Original Assignee
Hangzhou Meichuang Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Meichuang Technology Co ltd filed Critical Hangzhou Meichuang Technology Co ltd
Priority to CN202310825064.4A priority Critical patent/CN116541455B/en
Publication of CN116541455A publication Critical patent/CN116541455A/en
Application granted granted Critical
Publication of CN116541455B publication Critical patent/CN116541455B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation
    • G06F16/2433Query languages
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/248Presentation of query results
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Stored Programmes (AREA)

Abstract

The embodiment of the application discloses an interface data integration method, an interface data integration device, computer equipment and a storage medium. The method comprises the following steps: setting an external API interface for acquiring data; creating FDW of PostgreSQL database, and defining external table structure matched with returned data of external API interface; querying or operating the external table through SQL sentences to acquire the data of the external API interface so as to obtain a query result; and storing the query result. By implementing the method of the embodiment of the application, the data from the API interface can be effectively integrated, so that the data can be uniformly managed and operated in the PostgreSQL database.

Description

Interface data integration method, device, computer equipment and storage medium
Technical Field
The present application relates to a data processing method, and more particularly, to an interface data integration method, apparatus, computer device, and storage medium.
Background
In the current information-based society, data has become one of the most important resources. With the rapid growth of the internet, huge amounts of data are generated, transmitted and processed every day. In this environment, database management systems have become a key tool for data storage and management.
Currently, many applications require access to and integration of data from multiple sources, e.g., data from different databases, data in a file system, and data obtained through an API interface. However, for this situation, existing database management systems often encounter many challenges, such as data source heterogeneity, data real-time and consistency issues, and the like.
In particular, although the PostgreSQL-based database system provides the function of FDW (external data wrapper, foreign Data Wrapper) to enable it to access and operate external data sources, how to efficiently integrate data from API interfaces remains a problem to be solved. In practical applications, the data types, formats, access modes and the like of the API interfaces may have great differences, so that it is difficult to directly use the FDW for data integration.
Therefore, there is a need to devise a new method to efficiently integrate data from API interfaces so that it can be uniformly managed and manipulated in the PostgreSQL database.
Disclosure of Invention
The application aims to overcome the defects of the prior art and provide an interface data integration method, an interface data integration device, computer equipment and a storage medium.
In order to achieve the above purpose, the present application adopts the following technical scheme: an interface data integration method, comprising:
setting an external API interface for acquiring data;
creating FDW of PostgreSQL database, and defining external table structure matched with returned data of external API interface;
querying or operating the external table through SQL sentences to acquire the data of the external API interface so as to obtain a query result;
and storing the query result.
The further technical scheme is as follows: the setting of the external API interface for acquiring data comprises the following steps:
determining an external API interface;
setting parameters of the external API interface, wherein the parameters comprise the URL of the API interface, required parameters and expected return data format;
validating the external API interface.
The further technical scheme is as follows: the creating the FDW of the PostgreSQL database and defining an external table structure matched with the external API interface return data comprises:
defining the structure of an external table according to the data format returned by the external API interface, and setting column names and data types;
writing a JavaScript program to write the FDW;
setting a data acquisition mode;
registering the FDW with a PostgreSQL database.
The further technical scheme is as follows: the querying or operating the external table through the SQL statement to obtain the data of the external API interface to obtain the query result includes:
querying the external table through SQL sentences to obtain the data of the external API interface;
operating the external table to convert the external table into corresponding operation on the external API interface through FDW so as to obtain an operation result;
and integrating the data of the external API interface and the operation result to obtain a query result.
The further technical scheme is as follows: the operating the external table to convert to a corresponding operation on the external API interface through FDW to obtain an operation result includes:
and performing insertion, updating and deleting operations on the external table to convert the external table into corresponding operations on the external API interface through FDW so as to obtain an operation result.
The application also provides an interface data integration device, which comprises:
the interface setting unit is used for setting an external API interface for acquiring data;
the creation unit is used for creating FDW of the PostgreSQL database and defining an external table structure matched with the returned data of the external API interface;
the external table processing unit is used for inquiring or operating the external table through an SQL statement so as to acquire the data of the external API interface and obtain an inquiring result;
and the storage unit is used for storing the query result.
The further technical scheme is as follows: the interface setting unit includes:
an interface determination subunit, configured to determine an external API interface;
a parameter setting subunit, configured to set parameters of the external API interface, where the parameters include a URL of the API interface, a required parameter, and an expected return data format;
and the verification subunit is used for verifying the external API interface.
The further technical scheme is as follows: the creation unit includes:
a definition subunit, configured to define a structure of an external table according to a data format returned by the external API interface, and set a column name and a data type;
the writing subunit is used for writing a JavaScript program to write the FDW;
a mode setting subunit, configured to set a data acquisition mode;
and the registration subunit is used for registering the FDW to the PostgreSQL database.
The application also provides a computer device which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the method when executing the computer program.
The present application also provides a storage medium storing a computer program which, when executed by a processor, implements the above method.
Compared with the prior art, the application has the beneficial effects that: the application establishes the FDW and the external table structure matched with the returned data of the external API interface by setting the external API interface, inquires or operates the external table, acquires the corresponding data, saves the inquired result, and realizes the effective integration of the data from the API interface, so that the data can be uniformly managed and operated in the PostgreSQL database.
The application is further described below with reference to the drawings and specific embodiments.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an application scenario of an interface data integration method provided by an embodiment of the present application;
fig. 2 is a flow chart of an interface data integration method according to an embodiment of the present application;
FIG. 3 is a schematic sub-flowchart of an interface data integration method according to an embodiment of the present application;
fig. 4 is a schematic sub-flowchart of an interface data integration method according to an embodiment of the present application;
FIG. 5 is a schematic sub-flowchart of an interface data integration method according to an embodiment of the present application;
FIG. 6 is a schematic block diagram of an interface data integration apparatus provided by an embodiment of the present application;
fig. 7 is a schematic block diagram of an interface setting unit of the interface data integration device according to the embodiment of the application;
FIG. 8 is a schematic block diagram of a creation unit of an interface data integration apparatus provided by an embodiment of the application;
FIG. 9 is a schematic block diagram of an external table processing unit of the interface data integration device provided by an embodiment of the application;
fig. 10 is a schematic block diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic application scenario diagram of an interface data integration method according to an embodiment of the present application. Fig. 2 is a schematic flow chart of an interface data integration method according to an embodiment of the present application. The interface data integration method is applied to the server. The server performs data interaction with the terminal, sets an external API interface of the docking terminal, including the URL, parameters and expected return data format of the API, creates FDW by using JavaScript, defines an external table structure matched with the return data of the API, and queries or operates the external table through SQL sentences to effectively integrate the data from the API interface so that the data can be uniformly managed and operated in a PostgreSQL database.
Fig. 2 is a flow chart of an interface data integration method according to an embodiment of the present application. As shown in fig. 2, the method includes the following steps S110 to S140.
S110, setting an external API interface required to acquire data.
In this embodiment, the external API interface refers to an interface that needs to perform acquisition data of a data set.
In one embodiment, referring to fig. 3, the step S110 may include steps S111 to S113.
S111, determining an external API interface.
In this embodiment, an appropriate external API interface is selected according to the requirements of data integration. The choice of external API interface should be based on such factors as the relevance, accuracy and reliability of its data.
For example: an external API interface for weather data may be selected that is capable of providing real-time weather information for each location.
S112, setting parameters of the external API interface, wherein the parameters comprise the URL of the API interface, required parameters and expected return data format.
In this embodiment, setting parameters of the external API interface may be implemented by programming or configuration files.
For the above example, the API interface for weather data requires a 'key' parameter and a 'location' parameter. According to the actual situation, the two parameters are set. Information of the API interface (including URL and parameters) is saved in a configuration file.
S113, verifying the external API interface.
In this embodiment, the correctness of the external API interface is verified by sending a test request. If the data returned by the API interface is as expected, then step S120 may be entered.
Specifically, an HTTP GET request is sent to the API interface to verify the validity and correctness of the API interface. If the data format and content returned by the API interface meets the expectations, step S120 may be entered.
S120, creating FDW of the PostgreSQL database, and defining an external table structure matched with the external API interface return data.
In this embodiment, the external table structure refers to a table structure that can precisely locate data returned to the API.
In one embodiment, referring to fig. 4, the step S120 may include steps S121 to S124.
S121, defining the structure of an external table according to the data format returned by the external API interface, and setting column names and data types.
Specifically, the structure of the external table is defined according to the data format returned by the external API interface. This includes setting column names, data types, and the like. These definitions should exactly match the data returned by the external API to ensure that the data is properly imported into the PostgreSQL database.
Interface data type and PostgreSQL type mapping relationship:
intelger: the sms alint, integer or big type is selected for use in PostgreSQL.
Number: the real, double precision, numeric, float or Decimal type is selected for use in PostgreSQL.
String: the char, varchar, text type is selected for use in PostgreSQL.
Boolean: the choice of the bootgresql type is used.
Date/DateTime: the date, time or timestamp type is selected for use in PostgreSQL.
Binary: the byte type is selected for use in PostgreSQL.
Uuid: the uuid type is selected for use in PostgreSQL.
Array: the integer or text type is selected for use in PostgreSQL.
Object: the json, jsonb or hstore type is selected for use in PostgreSQL.
For example:
for weather data, the interface data contains 'location_name', 'temp_c' and 'hub' fields. The type of 'location_name' is string, the type of 'temp_c' is number, and the type of 'location' is number.
The name of the definition table is the interface name or custom. For example: weather_data
Columns of the definition table are named 'location_name', 'temp_c' and 'sensitivity'.
The field data type of a column of the table is defined. The PostgreSQL data type of 'location_name' is 'text', the PostgreSQL data type of 'temp_c' is 'float', and the PostgreSQL data type of 'hub' is 'integer'.
S122, writing a JavaScript program to write the FDW.
In this embodiment, a program is written using JavaScript, which will act as an FDW for accessing the API interface and acquiring data. This program should be able to send HTTP requests to the API interface and process the returned data. It should be able to handle various possible error conditions, such as network errors, API errors, etc.
Specifically, a program is written using JavaScript, and this program will be used to acquire the data of the API interface and import it into the external table. In this procedure, the HTTP module of JavaScript is used to send HTTP requests, and the postgres-FDW module is used to create the FDW.
S123, setting a data acquisition mode.
In the JavaScript program, a method is set, which will be used to acquire data of the external API interface. This method will send an HTTP GET request to the external API interface, GET the returned data and convert the data into JSON format.
The following is the code for this program:
const http = require('http');
const postgres = require('postgres-fdw');
let fdw = new postgres.FDW({
tableName: 'weather_data',
columns: {
'location_name': 'text',
'temp_c': 'float',
'humidity': 'integer',
// ...
},
});
fdw.setFetchMethod((callback) =>{
const url = 'http://api.weatherapi.com/v1/current.jsonkey=YOUR_KEY&location=Beijing';
http.get(url, (response) =>{
let data = '';
response.on('data', (chunk) =>{
data += chunk;
});
response.on('end', () =>{
const jsonData = JSON.parse(data);
callback(null, jsonData);
});
}).on('error', (error) =>{
callback(error);
});
});
fdw.register()。
s124, registering the FDW into a PostgreSQL database.
In this embodiment, this JavaScript program is registered as an FDW in the PostgreSQL database. Thus, postgreSQL is able to access the API interface and obtain data through this FDW.
S130, inquiring or operating the external table through SQL sentences to acquire the data of the external API interface so as to obtain an inquiring result.
In this embodiment, the query result refers to the data of the external API interface and the set of results of the corresponding operation on the external API interface.
In one embodiment, referring to fig. 5, the step S130 may include steps S131 to S133.
S131, inquiring the external table through SQL sentences to acquire the data of the external API interface.
Specifically, the external table is queried by SQL statements, which may be performed like querying a local table, to obtain the data of the API interface.
S132, operating the external table to convert the operation into corresponding operation of the external API interface through FDW so as to obtain an operation result.
Specifically, the external table is subjected to insertion, updating and deleting operations, so that the external table is converted into corresponding operations on the external API interface through FDW, and an operation result is obtained.
S133, integrating the data of the external API interface and the operation result to obtain a query result.
For example:
writing SQL sentences: the external table is queried or operated on using SQL statements. An SQL statement is written to query the current temperature of a place.
For example, when inquiring the current temperature of Beijing, the SQL statement is: SELECT temp_c FROM weather_ data WHERE location _name= 'beijin';
when querying all places with humidity higher than 70%, the SQL statement is: SELECT location_ name FROM weather _ data WHERE humidity >70;
and performing insertion, update or deletion operations as required. Sending the SQL statement to the PostgreSQL database, executing the statement by the database and returning the result; after receiving the query results, the results are processed. For example, the results may be displayed on a user interface or saved to a database.
S140, storing the query result.
And saving the query result to the designated position.
According to the interface data integration method, the FDW and the external table structure matched with the returned data of the external API interface are created by setting the external API interface, the external table is queried or operated, the corresponding data is obtained, and the queried result is stored, so that the data from the API interface can be effectively integrated, and unified management and operation can be performed in the PostgreSQL database.
Fig. 6 is a schematic block diagram of an interface data integration apparatus 300 according to an embodiment of the application. As shown in fig. 6, the present application also provides an interface data integration apparatus 300 corresponding to the above interface data integration method. The interface data integration apparatus 300 includes a unit for performing the interface data integration method described above, and may be configured in a server. Specifically, referring to fig. 6, the interface data integration apparatus 300 includes an interface setting unit 301, a creating unit 302, an external table processing unit 303, and a saving unit 304.
An interface setting unit 301, configured to set an external API interface where data needs to be acquired; a creating unit 302, configured to create an FDW of the PostgreSQL database, and define an external table structure that matches the external API interface return data; an external table processing unit 303, configured to query or operate the external table through an SQL statement, so as to obtain data of the external API interface, so as to obtain a query result; a saving unit 304, configured to save the query result.
In one embodiment, as shown in fig. 7, the interface setting unit 301 includes an interface determination subunit 3011, a parameter setting subunit 3012, and a verification subunit 3013.
An interface determination subunit 3011, configured to determine an external API interface; a parameter setting subunit 3012, configured to set parameters of the external API interface, where the parameters include a URL of the API interface, a required parameter, and an expected return data format; and a verification subunit 3013, configured to verify the external API interface.
In one embodiment, as shown in fig. 8, the creating unit 302 includes a defining subunit 3021, a writing subunit 3022, a mode setting subunit 3023, and a registering subunit 3024.
A defining subunit 3021, configured to define the structure of the external table according to the data format returned by the external API interface, and set a column name and a data type; a writing subunit 3022, configured to write a JavaScript program to write the FDW; a mode setting subunit 3023 for setting a data acquisition mode; a registration subunit 3024 for registering the FDW with the PostgreSQL database.
In one embodiment, as shown in fig. 9, the external table processing unit 303 includes a query subunit 3031, an operation subunit 3032, and an integration subunit 3033.
A query subunit 3031, configured to query the external table through an SQL statement, and obtain data of the external API interface; an operation subunit 3032, configured to operate the external table, so as to convert the external table into a corresponding operation on the external API interface through FDW, so as to obtain an operation result; and the integrating subunit 3033 is configured to integrate the data of the external API interface and the operation result to obtain a query result.
In an embodiment, the operation subunit 3032 is configured to perform insert, update, and delete operations on the external table, so as to convert the external table into a corresponding operation on the external API interface through FDW, so as to obtain an operation result.
It should be noted that, as will be clearly understood by those skilled in the art, the specific implementation process of the interface data integration apparatus 300 and each unit may refer to the corresponding description in the foregoing method embodiments, and for convenience and brevity of description, the description is omitted here.
The interface data integration apparatus 300 described above may be implemented in the form of a computer program that can be run on a computer device as shown in fig. 10.
Referring to fig. 10, fig. 10 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 may be a server, where the server may be a stand-alone server or may be a server cluster formed by a plurality of servers.
With reference to FIG. 10, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform an interface data integration method.
The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the execution of a computer program 5032 in the non-volatile storage medium 503, which computer program 5032, when executed by the processor 502, causes the processor 502 to perform an interface data integration method.
The network interface 505 is used for network communication with other devices. It will be appreciated by those skilled in the art that the structure shown in FIG. 10 is merely a block diagram of some of the structures associated with the present inventive arrangements and does not constitute a limitation of the computer device 500 to which the present inventive arrangements may be applied, and that a particular computer device 500 may include more or fewer components than shown, or may combine certain components, or may have a different arrangement of components.
Wherein the processor 502 is configured to execute a computer program 5032 stored in a memory to implement the steps of:
setting an external API interface for acquiring data; creating FDW of PostgreSQL database, and defining external table structure matched with returned data of external API interface; querying or operating the external table through SQL sentences to acquire the data of the external API interface so as to obtain a query result; and storing the query result.
In one embodiment, when the step of setting the external API interface to acquire data is implemented by the processor 502, the following steps are specifically implemented:
determining an external API interface; setting parameters of the external API interface, wherein the parameters comprise the URL of the API interface, required parameters and expected return data format; validating the external API interface.
In one embodiment, when the processor 502 implements the FDW for creating the PostgreSQL database and defines an external table structure step that matches the external API interface return data, the following steps are specifically implemented:
defining the structure of an external table according to the data format returned by the external API interface, and setting column names and data types; writing a JavaScript program to write the FDW; setting a data acquisition mode; registering the FDW with a PostgreSQL database.
In an embodiment, when the processor 502 queries or operates the external table through the SQL statement to obtain the data of the external API interface to obtain the query result, the following steps are specifically implemented:
querying the external table through SQL sentences to obtain the data of the external API interface; operating the external table to convert the external table into corresponding operation on the external API interface through FDW so as to obtain an operation result; and integrating the data of the external API interface and the operation result to obtain a query result.
In an embodiment, when the processor 502 performs the operation on the external table to convert the operation to a corresponding operation on the external API interface through FDW to obtain an operation result, the following steps are specifically implemented:
and performing insertion, updating and deleting operations on the external table to convert the external table into corresponding operations on the external API interface through FDW so as to obtain an operation result.
It should be appreciated that in an embodiment of the application, the processor 502 may be a central processing unit (Central Processing Unit, CPU), the processor 502 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Those skilled in the art will appreciate that all or part of the flow in a method embodying the above described embodiments may be accomplished by computer programs instructing the relevant hardware. The computer program comprises program instructions, and the computer program can be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.
Accordingly, the present application also provides a storage medium. The storage medium may be a computer readable storage medium. The storage medium stores a computer program which, when executed by a processor, causes the processor to perform the steps of:
setting an external API interface for acquiring data; creating FDW of PostgreSQL database, and defining external table structure matched with returned data of external API interface; querying or operating the external table through SQL sentences to acquire the data of the external API interface so as to obtain a query result; and storing the query result.
In one embodiment, when the processor executes the computer program to implement the step of setting the external API interface for acquiring data, the steps are specifically implemented as follows:
determining an external API interface; setting parameters of the external API interface, wherein the parameters comprise the URL of the API interface, required parameters and expected return data format; validating the external API interface.
In one embodiment, when the processor executes the computer program to implement the FDW for creating the PostgreSQL database and define an external table structure step matched with the external API interface return data, the steps are specifically implemented as follows:
defining the structure of an external table according to the data format returned by the external API interface, and setting column names and data types; writing a JavaScript program to write the FDW; setting a data acquisition mode; registering the FDW with a PostgreSQL database.
In one embodiment, when the processor executes the computer program to implement the step of querying or operating the external table through an SQL statement to obtain the data of the external API interface, the following steps are specifically implemented:
querying the external table through SQL sentences to obtain the data of the external API interface; operating the external table to convert the external table into corresponding operation on the external API interface through FDW so as to obtain an operation result; and integrating the data of the external API interface and the operation result to obtain a query result.
In an embodiment, when the processor executes the computer program to implement the step of operating the external table to translate the operation to a corresponding operation on the external API interface through FDW to obtain an operation result, the following steps are specifically implemented:
and performing insertion, updating and deleting operations on the external table to convert the external table into corresponding operations on the external API interface through FDW so as to obtain an operation result.
The storage medium may be a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, or other various computer-readable storage media that can store program codes.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed.
The steps in the method of the embodiment of the application can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the application can be combined, divided and deleted according to actual needs. In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The integrated unit may be stored in a storage medium if implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present application is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a terminal, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application.
While the application has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the application. Therefore, the protection scope of the application is subject to the protection scope of the claims.

Claims (8)

1. The interface data integration method is characterized by comprising the following steps:
setting an external API interface for acquiring data;
creating FDW of PostgreSQL database, and defining external table structure matched with returned data of external API interface;
querying or operating the external table through SQL sentences to acquire the data of the external API interface so as to obtain a query result;
storing the query result;
the creating the FDW of the PostgreSQL database and defining an external table structure matched with the external API interface return data comprises:
defining the structure of an external table according to the data format returned by the external API interface, and setting column names and data types; specifically, the mapping relation between the interface data type and the postgreSQL type:
integer selects the sms line, integer or bigint type in PostgreSQL;
number selects the real, double precision, number, float or decmal type in postgreSQL;
string selects the char, varchar, text type in PostgreSQL;
boolean selects the Boolean type for use in PostgreSQL;
Date/DateTime selects the Date, time or timestamp type to use in PostgreSQL;
the Binary selects a byte type to use in PostgreSQL;
uuid selects the use of Uuid type in PostgreSQL;
array selects the use of integer [ ] or text [ ] types in PostgreSQL;
object selects json, jsob or hstore type for use in PostgreSQL;
writing a JavaScript program to write the FDW;
setting a data acquisition mode;
registering the FDW with a PostgreSQL database.
2. The interface data integration method according to claim 1, wherein the setting of the external API interface required to acquire data includes:
determining an external API interface;
setting parameters of the external API interface, wherein the parameters comprise the URL of the API interface, required parameters and expected return data format;
validating the external API interface.
3. The interface data integration method according to claim 1, wherein the querying or operating the external table through an SQL statement to obtain the data of the external API interface to obtain a query result includes:
querying the external table through SQL sentences to obtain the data of the external API interface;
operating the external table to convert the external table into corresponding operation on the external API interface through FDW so as to obtain an operation result;
and integrating the data of the external API interface and the operation result to obtain a query result.
4. The interface data integration method of claim 3, wherein the operating the external table to translate through FDW to a corresponding operation on the external API interface to obtain an operation result comprises:
and performing insertion, updating and deleting operations on the external table to convert the external table into corresponding operations on the external API interface through FDW so as to obtain an operation result.
5. An interface data integration apparatus, comprising:
the interface setting unit is used for setting an external API interface for acquiring data;
the creation unit is used for creating FDW of the PostgreSQL database and defining an external table structure matched with the returned data of the external API interface;
the external table processing unit is used for inquiring or operating the external table through an SQL statement so as to acquire the data of the external API interface and obtain an inquiring result;
a storage unit, configured to store the query result;
the creation unit includes:
a definition subunit, configured to define a structure of an external table according to a data format returned by the external API interface, and set a column name and a data type; specifically, the mapping relation between the interface data type and the postgreSQL type:
integer selects the sms line, integer or bigint type in PostgreSQL;
number selects the real, double precision, number, float or decmal type in postgreSQL;
string selects the char, varchar, text type in PostgreSQL;
boolean selects the Boolean type for use in PostgreSQL;
Date/DateTime selects the Date, time or timestamp type to use in PostgreSQL;
the Binary selects a byte type to use in PostgreSQL;
uuid selects the use of Uuid type in PostgreSQL;
array selects the use of integer [ ] or text [ ] types in PostgreSQL;
object selects json, jsob or hstore type for use in PostgreSQL;
the writing subunit is used for writing a JavaScript program to write the FDW;
a mode setting subunit, configured to set a data acquisition mode;
and the registration subunit is used for registering the FDW to the PostgreSQL database.
6. The interface data integration apparatus according to claim 5, wherein the interface setting unit includes:
an interface determination subunit, configured to determine an external API interface;
a parameter setting subunit, configured to set parameters of the external API interface, where the parameters include a URL of the API interface, a required parameter, and an expected return data format;
and the verification subunit is used for verifying the external API interface.
7. A computer device, characterized in that it comprises a memory on which a computer program is stored and a processor which, when executing the computer program, implements the method according to any of claims 1-4.
8. A storage medium storing a computer program which, when executed by a processor, implements the method of any one of claims 1 to 4.
CN202310825064.4A 2023-07-06 2023-07-06 Interface data integration method, device, computer equipment and storage medium Active CN116541455B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310825064.4A CN116541455B (en) 2023-07-06 2023-07-06 Interface data integration method, device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310825064.4A CN116541455B (en) 2023-07-06 2023-07-06 Interface data integration method, device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN116541455A CN116541455A (en) 2023-08-04
CN116541455B true CN116541455B (en) 2023-09-22

Family

ID=87444004

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310825064.4A Active CN116541455B (en) 2023-07-06 2023-07-06 Interface data integration method, device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116541455B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106844545A (en) * 2016-12-30 2017-06-13 江苏瑞中数据股份有限公司 A kind of implementation method of the Database Systems with double engines based on stsndard SQL
US9760571B1 (en) * 2013-07-23 2017-09-12 jSonar Inc. Tabular DB interface for unstructured data
CN111126623A (en) * 2019-12-17 2020-05-08 支付宝(杭州)信息技术有限公司 Model updating method, device and equipment
CN111381820A (en) * 2018-12-29 2020-07-07 北京数聚鑫云信息技术有限公司 Method and device for automatically generating API based on GUI

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9760571B1 (en) * 2013-07-23 2017-09-12 jSonar Inc. Tabular DB interface for unstructured data
CN106844545A (en) * 2016-12-30 2017-06-13 江苏瑞中数据股份有限公司 A kind of implementation method of the Database Systems with double engines based on stsndard SQL
CN111381820A (en) * 2018-12-29 2020-07-07 北京数聚鑫云信息技术有限公司 Method and device for automatically generating API based on GUI
CN111126623A (en) * 2019-12-17 2020-05-08 支付宝(杭州)信息技术有限公司 Model updating method, device and equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于PostgreSQL的大规模分布式异构数据库统一访问研究;汪雪飞;中国优秀硕士学位论文全文数据库 (基础科学辑);第I138-928页 *

Also Published As

Publication number Publication date
CN116541455A (en) 2023-08-04

Similar Documents

Publication Publication Date Title
US20200183932A1 (en) Optimizing write operations in object schema-based application programming interfaces (apis)
CA3025493C (en) Optimizing read and write operations in object schema-based application programming interfaces (apis)
WO2017100544A1 (en) Query processor
US10296499B2 (en) Dynamic database mapping
CN110795455A (en) Dependency relationship analysis method, electronic device, computer device and readable storage medium
CN108647357B (en) Data query method and device
KR20140093947A (en) Systems and methods for dynamic service integration
US11263542B2 (en) Technologies for auto discover and connect to a rest interface
CN111966747A (en) Data synchronization method, system, terminal device and storage medium
US20130290253A1 (en) Data index using a linked data standard
US10234295B2 (en) Address remediation using geo-coordinates
US11693859B2 (en) Systems and methods for data retrieval from a database indexed by an external search engine
CN116541455B (en) Interface data integration method, device, computer equipment and storage medium
CN111581212B (en) Data storage method, system, server and storage medium of relational database
US20230289331A1 (en) Model generation service for data retrieval
US9489652B1 (en) Obtaining and running a local query on a computing device
US9037752B1 (en) Remote materialization of low velocity data
US11093492B1 (en) System and method of fetching data from an external program
US11003690B1 (en) Aggregator systems for storage of data segments
CN115202711A (en) Data publishing method and system
US10749766B1 (en) Archival datastore for aggregated metrics
US9916154B2 (en) File-based transport of table content
CN114036158A (en) Elasticissearch and MySQL combined query method and device
CN112988798A (en) Log processing method, device, equipment and medium
CN115545006B (en) Rule script generation method, device, computer equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant