CN111767303A - Data query method and device, server and readable storage medium - Google Patents
Data query method and device, server and readable storage medium Download PDFInfo
- Publication number
- CN111767303A CN111767303A CN202010737086.1A CN202010737086A CN111767303A CN 111767303 A CN111767303 A CN 111767303A CN 202010737086 A CN202010737086 A CN 202010737086A CN 111767303 A CN111767303 A CN 111767303A
- Authority
- CN
- China
- Prior art keywords
- full
- query
- text search
- search engine
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2428—Query predicate definition using graphical user interfaces, including menus and forms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a data query method, a data query device, a server and a readable storage medium, wherein the method comprises the following steps: acquiring a data query request submitted by a user through a query interface, wherein the data query request comprises query conditions and target fields related to the query conditions, and the target fields correspond to a plurality of list tables of a relational database; acquiring indexes of full-text search engines corresponding to the multiple single tables, wherein the indexes of the full-text search engines are generated according to the multiple single tables, and the indexes of the full-text search engines comprise all fields of the multiple single tables; and querying data matched with the query condition from the index of the full-text search engine by using the retrieval grammar of the full-text search engine, so that full-text retrieval of stored data can be supported, data query across multiple tables is realized, and the efficiency of data query is improved.
Description
Technical Field
The present invention relates to the field of database technologies, and in particular, to a data query method, apparatus, server, and readable storage medium.
Background
At present, a single form engine based on a relational database is simple in implementation mode, but in daily application, few scenarios can be achieved by only using a single form. In most cases, the query for a certain data still depends on multi-table (cross-table) query, for example, now the multi-table query generates a view by associating multi-tables through view, but because the association result of the multi-table may be very large, the view is not suitable for full-text retrieval, and the efficiency of data query is low.
Disclosure of Invention
The embodiment of the invention provides a data query method, a data query device, a server and a readable storage medium, which can support full-text retrieval of stored data, realize data query across multiple tables and improve the efficiency of data query.
A first aspect of an embodiment of the present invention provides a data query method, including:
acquiring a data query request submitted by a user through a query interface, wherein the data query request comprises query conditions and target fields related to the query conditions, and the target fields correspond to a plurality of list tables of a relational database;
acquiring indexes of full-text search engines corresponding to the multiple single tables, wherein the indexes of the full-text search engines are generated according to the multiple single tables, and the indexes of the full-text search engines comprise all fields of the multiple single tables;
and querying data matched with the query condition from the index of the full-text search engine by utilizing the retrieval grammar of the full-text search engine.
A second aspect of the embodiments of the present invention provides a data query apparatus, including:
the system comprises an acquisition module, a query module and a query module, wherein the acquisition module is used for acquiring a data query request submitted by a user through a query interface, the data query request comprises query conditions and target fields related to the query conditions, and the target fields correspond to a plurality of list tables of a relational database;
the obtaining module is further configured to obtain indexes of a full-text search engine corresponding to the multiple single tables, where the indexes of the full-text search engine are generated according to the multiple single tables, and the indexes of the full-text search engine include all fields of the multiple single tables;
and the processing module is used for inquiring the data matched with the inquiry condition from the index of the full-text search engine by utilizing the retrieval grammar of the full-text search engine.
A third aspect of embodiments of the present invention provides a server, including a processor, a network interface, and a storage device, where the processor, the network interface, and the storage device are connected to each other, where the network interface is controlled by the processor to send and receive data, and the storage device is used to store a computer program, where the computer program includes program instructions, and the processor is configured to call the program instructions to execute the data query method according to the first aspect.
A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, where program instructions are stored in the computer-readable storage medium, and when the program instructions are executed, the computer-readable storage medium is configured to implement the data query method according to the first aspect.
A fifth aspect of embodiments of the present invention provides a computer program product or a computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to make the computer device execute the data query method according to the first aspect.
In the embodiment of the invention, a server acquires a data query request submitted by a user through a query interface, the data query request comprises a query condition and a target field related to the query condition, the target field corresponds to a plurality of single tables of a relational database, and acquires an index of a full-text search engine corresponding to the plurality of single tables, the index of the full-text search engine is generated according to the plurality of single tables, the index of the full-text search engine comprises all the fields of the plurality of single tables, and further, the data matched with the query condition is queried from the index of the full-text search engine by utilizing a retrieval grammar of the full-text search engine, so that full-text retrieval of stored data can be supported, data query across the plurality of tables is realized, and the efficiency of data query is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a data query method according to an embodiment of the present invention;
FIG. 2 is a flow chart of another data query method provided by the embodiment of the invention;
FIG. 3a is a form engine configuration interface provided by an embodiment of the present invention;
FIG. 3b is an interface for configuring query criteria provided by an embodiment of the present invention;
FIG. 3c is a diagram of an edit style provided by an embodiment of the invention;
FIG. 3d is a field of setup results and presentation provided by an embodiment of the present invention;
FIG. 3e is a query interface provided by an embodiment of the present invention;
FIG. 4 is a diagram of a multi-table generated list table provided by an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of a data query device according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Cloud computing (cloud computing) refers to a delivery and use mode of an IT infrastructure, and refers to obtaining required resources in an on-demand and easily-extensible manner through a network; the generalized cloud computing refers to a delivery and use mode of a service, and refers to obtaining a required service in an on-demand and easily-extensible manner through a network. Such services may be IT and software, internet related, or other services. Cloud Computing is a product of development and fusion of traditional computers and network technologies, such as Grid Computing (Grid Computing), distributed Computing (distributed Computing), Parallel Computing (Parallel Computing), Utility Computing (Utility Computing), network storage (network storage technologies), Virtualization (Virtualization), Load balancing (Load Balance), and the like.
With the development of diversification of internet, real-time data stream and connecting equipment and the promotion of demands of search service, social network, mobile commerce, open collaboration and the like, cloud computing is rapidly developed. Different from the prior parallel distributed computing, the generation of cloud computing can promote the revolutionary change of the whole internet mode and the enterprise management mode in concept.
A distributed cloud storage system (hereinafter, referred to as a storage system) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of different types in a network through application software or application interfaces to cooperatively work by using functions such as cluster application, grid technology, and a distributed storage file system, and provides a data storage function and a service access function to the outside.
At present, a storage method of a storage system is as follows: logical volumes are created, and when created, each logical volume is allocated physical storage space, which may be the disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as data Identification (ID), the file system writes each object into a physical storage space of the logical volume, and the file system records storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.
The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided in advance into stripes according to a group of capacity measures of objects stored in a logical volume (the measures often have a large margin with respect to the capacity of the actual objects to be stored) and Redundant Array of Independent Disks (RAID), and one logical volume can be understood as one stripe, thereby allocating physical storage space to the logical volume.
Database (Database), which can be regarded as an electronic file cabinet in short, a place for storing electronic files, a user can add, query, update, delete, etc. to data in files. A "database" is a collection of data that is stored together in a manner that can be shared by multiple users, has as little redundancy as possible, and is independent of the application.
A Database Management System (DBMS) is a computer software System designed for managing a Database, and generally has basic functions of storage, interception, security assurance, backup, and the like. The database management system can classify according to the database model supported by it, such as relational expression, XML (Extensible markup language), and full text search engine elastic search; or classified according to the type of computer supported, e.g., server cluster, mobile phone; or sorted according to the Query Language used, such as SQL (Structured Query Language), XQuery, or sorted according to performance impulse emphasis, such as max size, maximum operating speed, or other sorting.
At present, for data query of multiple tables (cross-table), view is mainly utilized to associate the multiple tables to generate one view, so as to implement data query before the multiple tables, which can implement cross-table query, but the association result of the multiple tables may be very large, the query efficiency may be very low, and the method is not suitable for full-text retrieval.
Therefore, the embodiment of the invention changes the associated multiple single tables into one single table by using the form engine thought of the single table, so that the query of the multiple tables becomes the same as the query of the single table, code development is not needed, the development efficiency is greatly improved, trillion-level record retrieval can be realized by utilizing the natural distributed architecture of the Elasticisearch, and the problem of low query efficiency can be solved by utilizing the full-text retrieval supported by the Elasticisearch which is very good. The Elasticisearch is a nosql database, provides a full-text search engine with distributed multi-user capability, and provides a RESTful web interface based on Lucene implementation. The Elasticsearch was developed in the Java language and published as open source under the Apache licensing terms, a popular enterprise level search engine. In cloud computing, real-time searching can be achieved, and the method is stable, reliable, rapid and convenient to install and use.
In a feasible embodiment, the server can obtain a data query request submitted by a user through a query interface, the data query request comprises a query condition and a target field related to the query condition, the target field corresponds to a plurality of single tables of a relational database, and obtains indexes of a full-text search engine corresponding to the plurality of single tables, the indexes of the full-text search engine are generated according to the plurality of single tables, the indexes of the full-text search engine comprise all the fields of the plurality of single tables, and then, the data matched with the query condition is queried from the indexes of the full-text search engine by using the retrieval grammar of the full-text search engine, so that full-text retrieval of stored data can be supported, data query across the plurality of tables is realized, and the efficiency of data query is improved.
Referring to fig. 1, fig. 1 is a schematic flow chart of a data query method according to an embodiment of the present invention. The method may be executed by a server, and the data query method described in this embodiment includes the following steps:
101. the method comprises the steps of obtaining a data query request submitted by a user through a query interface, wherein the data query request comprises query conditions and target fields related to the query conditions, and the target fields correspond to a plurality of list tables of a relational database.
The target field comprises at least two fields related to the query condition, the query condition can be gender, time, identity card, etc., and the query condition can be associated with at least two fields. For example, it is assumed that a data query request is used for querying which roles a certain user has, the data query request includes the query condition and a target field associated with the query condition, the query condition is the user and the roles, and the target field associated with the query condition includes a field corresponding to the role and a field corresponding to the user.
The multiple single tables comprise an association relation table among the single tables, the association relation table comprises a main foreign key relation of the single tables, and the association relation among the multiple tables can be determined according to the main foreign key relation.
Specifically, the server obtains a data query data request submitted by a user through a query interface, and the server can determine at least two sheet tables in the database corresponding to the fields related to the query conditions according to the at least two fields related to the query conditions. For example, in the above example, the target field associated with the query condition includes a field corresponding to a role and a field corresponding to a user, and a user table A, user _ role user and role association table and a role table may be determined according to the field corresponding to the role and the field corresponding to the user.
For example, according to the above example, the target field associated with the query condition includes a field corresponding to a role and a field corresponding to a user, three single tables (multiple tables) can be determined according to the field corresponding to the role and the field corresponding to the user, which are respectively a user table a in table 1, a user _ role user and role association table in table 2, and a role table in table 3, where table 2 includes the main foreign key relationships of the remaining two single tables, and the relationship between table 1 and table 2 can be established through the user _ role user and role association table in table 2, so that the subsequent multiple table query is performed according to the corresponding main foreign key relationships.
TABLE 1
| Field(s) | Type (B) | Length of | Description of the invention |
| user_id | bigint | 20 | Main key |
| user_name | char | 50 | Name(s) |
| create_time | datetime | 10 | Creation |
| channel | tinyint | ||
| 4 | Registering a channel source |
TABLE 2
| Field(s) | Type (B) | Length of | Description of the invention |
| role_id | bigint | 20 | Main key, external key role table |
| user_id | bigint | 20 | Main key, external key user table |
TABLE 3
| Field(s) | Type (B) | Length of | Description of the invention |
| role_id | bigint | 20 | Main key |
| role_name | char | 20 | Role names |
It should be noted that at least two list tables are queried according to at least two fields related to query conditions, and there is no one-to-one correspondence, where the list tables related to at least two fields related to query conditions may also be queried, for example, the above example, 3 list tables are queried.
102. And acquiring indexes of full-text search engines corresponding to the multiple single tables, wherein the indexes of the full-text search engines are generated according to the multiple single tables, and the indexes of the full-text search engines comprise all fields of the multiple single tables.
The full-text search engine may be an Elasticsearch, and if the full-text search engine is an Elasticsearch, the index of the full-text search engine is an Elasticsearch index, and the Elasticsearch index is a table of a similar relational database.
The index of the full text search engine may be understood as a single table that includes all of the fields of the various tables.
Specifically, after acquiring a data query request submitted by a user through a query interface, the server acquires a plurality of single tables corresponding to target fields according to the target fields, and since the index of the full-text search engine is generated by the plurality of single tables and includes all the fields of the plurality of single tables, the index of the full-text search engine generated according to the plurality of single tables can be acquired according to the fields included in the plurality of single tables.
103. And querying data matched with the query condition from the index of the full-text search engine by utilizing the retrieval grammar of the full-text search engine.
There is a fixed retrieval syntax for the full-text search engine, for example, the retrieval syntax of a general full-text search engine may be restapi or dsl.
In one possible embodiment, the server may construct a search statement using the search syntax and the query condition of the full-text search engine, and query the index of the full-text search engine for data matching the query condition using the search statement. The retrieval sentence is constructed by utilizing the retrieval grammar and the query condition of the full-text search engine, so that the retrieval transformation of the index of the full-text search engine can be realized, and the Structured Query Language (SQL) query mode in the relational database is replaced.
For example, the search statement of a structured query language SQL query in a relational database is:
and the retrieval statement (such as the elastic search dsl grammar) of the full-text search engine is:
through retrieval modification, the retrieval sentences utilizing the full-text search engine are relatively simple.
In the embodiment of the invention, the server can obtain the data query request submitted by the user through the query interface, the data query request comprises the query condition and the target field relevant to the query condition, the target field corresponds to the multiple single tables of the relational database, the indexes of the full-text search engine generated according to the multiple single tables corresponding to the multiple single tables are obtained, and then the data matched with the query condition is queried from the indexes of the full-text search engine by utilizing the retrieval grammar of the full-text search engine, so that the full-text retrieval of the stored data can be supported, the data query across the multiple tables is realized, and the efficiency of the data query is improved.
Referring to fig. 2, fig. 2 is a schematic flow chart illustrating another data query method according to an embodiment of the present invention. The method may be executed by a server, and the data query method described in this embodiment includes:
201. obtaining a plurality of fields in fields included in the plurality of sheets, the plurality of fields being selected by an administrator from the fields included in the plurality of sheets through a form engine configuration interface.
Specifically, the fields are formed by selecting at least one field in each of the multiple sheets through a form engine configuration interface in advance by an administrator (e.g., a developer), and the server may obtain the fields of the at least one field in the fields included in each of the multiple sheets.
In a possible embodiment, the form engine configuration interface may provide a form engine configuration interface for configuring a certain form, the server responds to a form engine configuration request for the form, and outputs the form engine configuration interface through the client, a data source, a query condition, and the like need to be selected on the form engine configuration interface, when the data source is selected, it needs to be determined which fields may be fields corresponding to the query condition, and in 3a, it can be seen that the fields sfzh, sbsj _ date, and generator may be fields corresponding to the query condition. After completing the data source selection, the server outputs an interface for configuring query conditions, such as an identity card, a filing time, a gender, and the like, and determines a field sfzh corresponding to the identity card, a field generator corresponding to the gender, and a field sbsj _ date corresponding to the filing time, as shown in fig. 3 b. Corresponding styles can be edited according to the identity card, the application time, the gender and the like, and a gender style editing interface is shown in fig. 3 c; after the query condition is configured, entering fig. 3d, setting the result and the field to be displayed, that is, setting the display sequence of the result and the field to be displayed; after configuration is successful, the whole query interface can be displayed through the client, and is shown in fig. 3e, wherein a date field can be searched according to date, gender can be generated in a dictionary form, pull-down selection is performed, and data query is performed through date, an identity card and gender.
The plurality of single tables are arranged by the single table flow.
202. And acquiring the plurality of list tables from the relational database according to the plurality of fields.
The multiple single tables comprise an incidence relation table among the single tables, and the incidence relation table comprises a main foreign key relation of the single tables.
Specifically, the fields are matched with the single tables in the relational database, and if the fields are matched with the fields in the single tables, the single tables are obtained from the relational database.
203. And carrying out field mapping processing on the plurality of single tables by using a structured query language and the field mapping relation so as to generate an index of the full-text search engine.
Wherein the structured query language may be SQL.
Because the relational databases are more and the type definitions are different, a field mapping relation is set, and the field types of the relational databases and the field types in the full-text search engine are mapped so as to facilitate the subsequent form engine processing.
Specifically, the server processes the multiple single tables by using the structured query language to obtain a result index of the full-text search engine, the field type included in the result index is the same as the field type in the multiple tables, and then the field type included in the result index is mapped according to the field mapping relation in the full-text search engine to realize field type unification, so that the index of the full-text search engine is obtained.
For example, in the embodiment of fig. 1, referring to the user table A, user _ role user and role association table and the role table, a single table (i.e. an index of a full text search engine) as shown in fig. 4 can be generated according to the structured query language SQL and the field mapping relationship, and it can be seen that the single table includes all the fields of the three single tables, i.e. the user table A, user _ role user and role association table and the role table. Wherein, this fig. 4 can be implemented by the following sql:
select user.user_id,user.user_name,user.create_time,user.channel,
role.role_id,role.role_name
from user,user_role,role
where user.user_id＝user_role.user_id and user_role.role_id＝role.role_id
and obtaining the fields of the result index according to the sql, and uniformly mapping the fields of the result index through a field mapping relation, thereby constructing the index shown in fig. 4.
It should be noted that if the fields are the same in different tables, the table name + field can be used as the field of the result index to distinguish different tables when naming.
In a possible embodiment, before the server performs the field mapping process on the multiple single tables by using the structured query language and the field mapping relationship to generate the index of the full-text search engine, the server may determine the field mapping relationship between the field type of each relational database and the field type in the full-text search engine, and store the field mapping relationship in the full-text search engine, so that the field mapping relationship is directly used in the subsequent field mapping. For example, in Table 4, the database field types may be mapped to the types in the elastic search index, such as int, integer, etc. may be mapped to long, date, datetime, timemap may be mapped to date, etc.
TABLE 4
In a feasible embodiment, after the field mapping processing is performed on the plurality of single tables by using the structured query language and the field mapping relationship to generate the index of the full-text search engine, the data included in each of the plurality of single tables is imported into the index of the full-text search engine through full synchronization, and the specific implementation manner is as follows: the server queries data included in each of the plurality of list tables by using a structured query language, and imports the data included in each list table into an index of a full-text search engine. If the data included in each single table is excessive, the data can be imported into the index of the full-text search engine in batches, and the data included in each single table in the multiple single tables can be synchronized with the data in the index of the full-text search engine.
Further, when data included in any one of the plurality of single tables changes, the changed data is updated to an index of the full-text search engine through incremental synchronization, and the specific implementation manner is as follows: when the data included in any one of the multiple single tables changes, the server updates the changed data to the index of the full-text search engine according to the field corresponding to the changed data of any one single table. When the data included in any one of the plurality of list tables changes, the server puts the changed data into a queue, and when a code responsible for updating the changed data is acquired, determining and updating data corresponding to the index of the full-text search engine according to the field corresponding to the changed data and the field mapping relation in the full-text search engine, namely, the index of the full-text search engine is found according to the field corresponding to the changed data and the field mapping relation, the changed data is then updated into the index of the full text search engine, for example, when the user table A, user _ role user and role association table, the role color table in the role color table are modified as described above (i.e. the data included in the role color table is changed), and determining and updating data corresponding to the index of the full-text search engine according to the field corresponding to the modified data in the role table and the field mapping relation in the full-text search engine.
204. The method comprises the steps of obtaining a data query request submitted by a user through a query interface, wherein the data query request comprises query conditions and target fields related to the query conditions, and the target fields correspond to a plurality of list tables of a relational database.
205. And acquiring indexes of full-text search engines corresponding to the multiple single tables, wherein the indexes of the full-text search engines are generated according to the multiple single tables, and the indexes of the full-text search engines comprise all fields of the multiple single tables.
206. And querying data matched with the query condition from the index of the full-text search engine by utilizing the retrieval grammar of the full-text search engine.
The specific implementation manner of steps 204 to 206 may refer to the related description in steps 101 to 103 in the data query method, and is not described herein again.
In the embodiment of the invention, a server acquires a plurality of fields in fields included in a plurality of single tables, the fields are selected from the fields included in the plurality of single tables through a form engine configuration interface by an administrator, acquires the plurality of single tables from a relational database according to the fields, performs field mapping processing on the plurality of single tables by using a structured query language and a field mapping relation to generate an index of a full-text search engine, acquires a data query request submitted by a user through the query interface, the data query request includes a query condition and a target field related to the query condition, the target field corresponds to the plurality of single tables in the relational database, acquires an index of the full-text search engine corresponding to the plurality of single tables, the index of the full-text search engine is generated according to the plurality of single tables, and the index of the full-text search engine includes all the fields of the plurality of single tables, the retrieval grammar of the full-text search engine is utilized to inquire the data matched with the inquiry condition from the index of the full-text search engine, so that the multi-table is converted into the single table, the form engine for inquiring the multi-table (cross-table) is indirectly realized, and the efficiency of data inquiry is improved.
Fig. 5 is a schematic structural diagram of a data query apparatus according to an embodiment of the present invention. The data query apparatus described in this embodiment, which is executable by a server, includes:
an obtaining module 501, configured to obtain a data query request submitted by a user through a query interface, where the data query request includes a query condition and a target field related to the query condition, and the target field corresponds to multiple list tables of a relational database;
the obtaining module 501 is further configured to obtain indexes of a full-text search engine corresponding to the multiple single tables, where the indexes of the full-text search engine are generated according to the multiple single tables, and the indexes of the full-text search engine include all fields of the multiple single tables;
a processing module 502, configured to query, using the retrieval syntax of the full-text search engine, data matching the query condition from the index of the full-text search engine.
Optionally, the obtaining module 501 is further configured to:
acquiring a plurality of fields in fields included in the plurality of sheets, wherein the fields are selected from the fields included in the plurality of sheets by an administrator through a form engine configuration interface;
acquiring the multiple list tables from the relational database according to the multiple fields;
and carrying out field mapping processing on the plurality of single tables by using a structured query language and the field mapping relation so as to generate an index of the full-text search engine.
Optionally, the processing module 502 is further configured to:
determining a field mapping relation between the field type of each relational database and the field type in the full-text search engine;
storing the field mapping in the full-text search engine.
Optionally, the multiple single tables include an association relationship table between the single tables, where the association relationship table includes a primary foreign key relationship of the single table.
Optionally, the processing module 502 performs field mapping processing on the multiple sheets by using a structured query language and a field mapping relationship to generate an index of a full-text search engine, and then is further configured to:
querying data included in each of the plurality of sheet tables by using the structured query language;
and importing the data included in each sheet table into the index of the full-text search engine.
Optionally, the processing module 502 is further configured to:
and when the data included in any one of the multiple single tables changes, updating the changed data into the index of the full-text search engine according to the field corresponding to the changed data of any one single table.
Optionally, the processing module 502 is specifically configured to:
constructing a retrieval statement by using the retrieval grammar of the full-text search engine and the query condition;
and querying data matched with the query condition from the index of the full-text search engine by using the retrieval statement.
It can be understood that the functions of each functional module of the data query apparatus in this embodiment may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the relevant description in fig. 1 or fig. 2 of the foregoing method embodiment, which is not described herein again.
In the embodiment of the present invention, the obtaining module 501 may obtain a data query request submitted by a user through a query interface, where the data query request includes a query condition and a target field related to the query condition, the target field corresponds to multiple single tables of a relational database, and obtains an index of a full-text search engine generated according to the multiple single tables, where the multiple single tables correspond to the multiple single tables, and further the processing module 502 queries data matched with the query condition from the index of the full-text search engine by using a retrieval syntax of the full-text search engine, so that full-text retrieval of stored data may be supported, data query across multiple tables may be implemented, and data query efficiency is improved.
Fig. 6 is a schematic structural diagram of a server according to an embodiment of the present invention. The server described in this embodiment includes: a processor 601, a network interface 602, and a memory 603. The processor 601, the network interface 602, and the memory 603 may be connected by a bus or other means, and the embodiment of the present invention is exemplified by being connected by a bus.
The processor 601 (or Central Processing Unit, CPU) is a computing core and a control core of the server. The network interface 602 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI, mobile communication interface, etc.), controlled by the processor 601 for transceiving data. The Memory 603(Memory) is a Memory device of the server, and stores programs and data. It is understood that the memory 603 may be a high-speed RAM memory, or a non-volatile memory (e.g., at least one disk memory); optionally, at least one memory device may be located remotely from the processor 601. Memory 603 provides storage space that stores the operating system and executable program code of the server, which may include, but is not limited to: windows system (an operating system), Linux system (an operating system), etc., which are not limited in this regard.
In the embodiment of the present invention, the processor 601 executes the executable program code in the memory 603 to perform the following operations:
acquiring a data query request submitted by a user through a query interface, wherein the data query request comprises query conditions and target fields related to the query conditions, and the target fields correspond to a plurality of list tables of a relational database;
acquiring indexes of full-text search engines corresponding to the multiple single tables, wherein the indexes of the full-text search engines are generated according to the multiple single tables, and the indexes of the full-text search engines comprise all fields of the multiple single tables;
and querying data matched with the query condition from the index of the full-text search engine by utilizing the retrieval grammar of the full-text search engine.
Optionally, before the processor 601 obtains the data query request submitted by the user through the query interface, the processor is further configured to:
acquiring a plurality of fields in fields included in the plurality of sheets, wherein the fields are selected from the fields included in the plurality of sheets by an administrator through a form engine configuration interface;
acquiring the multiple list tables from the relational database according to the multiple fields;
and carrying out field mapping processing on the plurality of single tables by using a structured query language and the field mapping relation so as to generate an index of the full-text search engine.
Optionally, the processor 601, before performing field mapping processing on the multiple sheets by using the structured query language and the field mapping relationship to generate the index of the full-text search engine, is further configured to: :
determining a field mapping relation between the field type of each relational database and the field type in the full-text search engine;
storing the field mapping in the full-text search engine.
Optionally, the multiple single tables include an association relationship table between the single tables, where the association relationship table includes a primary foreign key relationship of the single table.
Optionally, the processor 601 performs field mapping processing on the multiple sheets by using a structured query language and field mapping relationships to generate an index of a full-text search engine, and then further:
querying data included in each of the plurality of sheet tables by using the structured query language;
and importing the data included in each sheet table into the index of the full-text search engine.
Optionally, the processor 601 is further configured to:
and when the data included in any one of the multiple single tables changes, updating the changed data into the index of the full-text search engine according to the field corresponding to the changed data of any one single table.
Optionally, the processor 601 is specifically configured to:
constructing a retrieval statement by using the retrieval grammar of the full-text search engine and the query condition;
and querying data matched with the query condition from the index of the full-text search engine by using the retrieval statement.
In the embodiment of the present invention, the server 601 may obtain a data query request submitted by a user through a query interface, where the data query request includes a query condition and a target field related to the query condition, the target field corresponds to multiple single tables of a relational database, and obtains an index of a full-text search engine generated according to the multiple single tables corresponding to the multiple single tables, and then the server 601 queries data matched with the query condition from the index of the full-text search engine by using a retrieval syntax of the full-text search engine, so that full-text retrieval of stored data may be supported, data query across multiple tables is implemented, and efficiency of data query is improved.
In a specific implementation, the processor 601, the network interface 602, and the memory 603 described in the embodiment of the present invention may execute the implementation described in the flow of the data query method provided in the embodiment of the present invention, and may also execute the implementation described in the data query apparatus provided in the embodiment of the present invention, which is not described herein again.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, where the computer program includes program instructions, and when the program instructions are executed by a processor, the steps performed in the data query embodiment described above may be performed.
Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the steps performed in the data query embodiment described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A method for data query, the method comprising:
acquiring a data query request submitted by a user through a query interface, wherein the data query request comprises query conditions and target fields related to the query conditions, and the target fields correspond to a plurality of list tables of a relational database;
acquiring indexes of full-text search engines corresponding to the multiple single tables, wherein the indexes of the full-text search engines are generated according to the multiple single tables, and the indexes of the full-text search engines comprise all fields of the multiple single tables;
and querying data matched with the query condition from the index of the full-text search engine by utilizing the retrieval grammar of the full-text search engine.
2. The method of claim 1, wherein before the obtaining of the data query request submitted by the user through the query interface, the method further comprises:
acquiring a plurality of fields in fields included in the plurality of sheets, wherein the fields are selected from the fields included in the plurality of sheets by an administrator through a form engine configuration interface;
acquiring the multiple list tables from the relational database according to the multiple fields;
and carrying out field mapping processing on the plurality of single tables by using a structured query language and the field mapping relation so as to generate an index of the full-text search engine.
3. The method of claim 2, wherein prior to performing the field mapping process on the plurality of sheets using the structured query language and the field mapping relationship to generate the index of the full text search engine, the method further comprises:
determining a field mapping relation between the field type of each relational database and the field type in the full-text search engine;
storing the field mapping in the full-text search engine.
4. The method of claim 2 or 3, wherein the plurality of sheets comprise an association table between sheets, the association table comprising primary foreign key relationships of sheets.
5. The method of claim 2, wherein after the field mapping processing of the plurality of sheets using the structured query language and the field mapping relationship to generate the index of the full text search engine, the method further comprises:
querying data included in each of the plurality of sheet tables by using the structured query language;
and importing the data included in each sheet table into the index of the full-text search engine.
6. The method of claim 5, further comprising:
and when the data included in any one of the multiple single tables changes, updating the changed data into the index of the full-text search engine according to the field corresponding to the changed data of any one single table.
7. The method of claim 1, wherein the querying the index of the full-text search engine for data matching the query condition using the retrieval syntax of the full-text search engine comprises:
constructing a retrieval statement by using the retrieval grammar of the full-text search engine and the query condition;
and querying data matched with the query condition from the index of the full-text search engine by using the retrieval statement.
8. A data query apparatus, characterized in that the apparatus comprises:
the system comprises an acquisition module, a query module and a query module, wherein the acquisition module is used for acquiring a data query request submitted by a user through a query interface, the data query request comprises query conditions and target fields related to the query conditions, and the target fields correspond to a plurality of list tables of a relational database;
the obtaining module is further configured to obtain indexes of a full-text search engine corresponding to the multiple single tables, where the indexes of the full-text search engine are generated according to the multiple single tables, and the indexes of the full-text search engine include all fields of the multiple single tables;
and the processing module is used for inquiring the data matched with the inquiry condition from the index of the full-text search engine by utilizing the retrieval grammar of the full-text search engine.
9. A server, comprising a processor, a network interface and a storage device, wherein the processor, the network interface and the storage device are connected with each other, wherein the network interface is controlled by the processor for transceiving data, the storage device is used for storing a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the data query method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program comprising program instructions that, when executed by a processor, cause the processor to perform the data query method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010737086.1A CN111767303A (en) | 2020-07-28 | 2020-07-28 | Data query method and device, server and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010737086.1A CN111767303A (en) | 2020-07-28 | 2020-07-28 | Data query method and device, server and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111767303A true CN111767303A (en) | 2020-10-13 |
Family
ID=72727586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010737086.1A Pending CN111767303A (en) | 2020-07-28 | 2020-07-28 | Data query method and device, server and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111767303A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112269806A (en) * | 2020-12-11 | 2021-01-26 | 深圳市房多多网络科技有限公司 | Data query method, device, equipment and computer storage medium |
| CN112463827A (en) * | 2020-11-16 | 2021-03-09 | 北京达佳互联信息技术有限公司 | Query method and device, electronic equipment and storage medium |
| CN112507200A (en) * | 2020-12-28 | 2021-03-16 | 浪潮云信息技术股份公司 | Method and apparatus for synchronizing data into search engine |
| CN112685445A (en) * | 2020-12-29 | 2021-04-20 | 杭州旷云金智科技有限公司 | Data query method and device, storage medium and electronic equipment |
| CN112711649A (en) * | 2020-12-25 | 2021-04-27 | 政采云有限公司 | Database multi-field matching method, device, equipment and storage medium |
| CN112905600A (en) * | 2021-03-19 | 2021-06-04 | 腾讯科技（深圳）有限公司 | Data query method and device, storage medium and electronic equipment |
| CN113111641A (en) * | 2021-04-20 | 2021-07-13 | 上海渠杰信息科技有限公司 | Data operation method and equipment based on full-text search engine |
| CN113157754A (en) * | 2021-03-22 | 2021-07-23 | 招银云创信息技术有限公司 | Method, device and equipment for displaying flow form data and storage medium |
| CN114925118A (en) * | 2022-06-09 | 2022-08-19 | 北京百度网讯科技有限公司 | Cross-table search method, device, equipment and storage medium |
| CN115617859A (en) * | 2022-12-16 | 2023-01-17 | 安徽华云安科技有限公司 | Data query method and device based on knowledge graph cluster |
| CN116204556A (en) * | 2022-12-29 | 2023-06-02 | 上海云砺信息科技有限公司 | Real-time object storage query system based on search engine and relational database |
| WO2024000896A1 (en) * | 2022-06-27 | 2024-01-04 | 深圳前海微众银行股份有限公司 | Sql script optimization method and device |
-
2020
- 2020-07-28 CN CN202010737086.1A patent/CN111767303A/en active Pending
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463827A (en) * | 2020-11-16 | 2021-03-09 | 北京达佳互联信息技术有限公司 | Query method and device, electronic equipment and storage medium |
| CN112463827B (en) * | 2020-11-16 | 2024-03-12 | 北京达佳互联信息技术有限公司 | Query method, query device, electronic equipment and storage medium |
| CN112269806B (en) * | 2020-12-11 | 2021-03-19 | 深圳市房多多网络科技有限公司 | Data query method, device, equipment and computer storage medium |
| CN112269806A (en) * | 2020-12-11 | 2021-01-26 | 深圳市房多多网络科技有限公司 | Data query method, device, equipment and computer storage medium |
| CN112711649A (en) * | 2020-12-25 | 2021-04-27 | 政采云有限公司 | Database multi-field matching method, device, equipment and storage medium |
| CN112507200A (en) * | 2020-12-28 | 2021-03-16 | 浪潮云信息技术股份公司 | Method and apparatus for synchronizing data into search engine |
| CN112685445A (en) * | 2020-12-29 | 2021-04-20 | 杭州旷云金智科技有限公司 | Data query method and device, storage medium and electronic equipment |
| CN112905600B (en) * | 2021-03-19 | 2023-09-26 | 腾讯科技（深圳）有限公司 | Data query method and device, storage medium and electronic equipment |
| CN112905600A (en) * | 2021-03-19 | 2021-06-04 | 腾讯科技（深圳）有限公司 | Data query method and device, storage medium and electronic equipment |
| CN113157754A (en) * | 2021-03-22 | 2021-07-23 | 招银云创信息技术有限公司 | Method, device and equipment for displaying flow form data and storage medium |
| CN113157754B (en) * | 2021-03-22 | 2023-01-31 | 招银云创信息技术有限公司 | Method, device and equipment for displaying flow form data and storage medium |
| CN113111641A (en) * | 2021-04-20 | 2021-07-13 | 上海渠杰信息科技有限公司 | Data operation method and equipment based on full-text search engine |
| CN114925118A (en) * | 2022-06-09 | 2022-08-19 | 北京百度网讯科技有限公司 | Cross-table search method, device, equipment and storage medium |
| WO2024000896A1 (en) * | 2022-06-27 | 2024-01-04 | 深圳前海微众银行股份有限公司 | Sql script optimization method and device |
| CN115617859A (en) * | 2022-12-16 | 2023-01-17 | 安徽华云安科技有限公司 | Data query method and device based on knowledge graph cluster |
| CN116204556A (en) * | 2022-12-29 | 2023-06-02 | 上海云砺信息科技有限公司 | Real-time object storage query system based on search engine and relational database |
| CN116204556B (en) * | 2022-12-29 | 2023-11-28 | 上海云砺信息科技有限公司 | Real-time object storage query system based on search engine and relational database |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111767303A (en) | Data query method and device, server and readable storage medium | |
| US11580070B2 (en) | Utilizing metadata to prune a data set | |
| CN109299102B (en) | HBase secondary index system and method based on Elastcissearch | |
| US11468103B2 (en) | Relational modeler and renderer for non-relational data | |
| CN108304444B (en) | Information query method and device | |
| EP2605158B1 (en) | Mixed join of row and column database tables in native orientation | |
| TW202032386A (en) | Data storage apparatus, translation apparatus, and database access method | |
| US20220083618A1 (en) | Method And System For Scalable Search Using MicroService And Cloud Based Search With Records Indexes | |
| US11294861B2 (en) | Data pruning based on metadata | |
| US20100235344A1 (en) | Mechanism for utilizing partitioning pruning techniques for xml indexes | |
| CN106294695A (en) | A kind of implementation method towards the biggest data search engine | |
| CN103778133A (en) | Database object changing method and device | |
| JP4483034B2 (en) | Heterogeneous data source integrated access method | |
| CN110837520A (en) | Data processing method, platform and system | |
| US10445370B2 (en) | Compound indexes for graph databases | |
| Caldarola et al. | Big data: A survey-the new paradigms, methodologies and tools | |
| CN110781183A (en) | Method and device for processing incremental data in Hive database and computer equipment | |
| US11567957B2 (en) | Incremental addition of data to partitions in database tables | |
| CN111125045B (en) | Lightweight ETL processing platform | |
| CN101719162A (en) | Multi-version open geographic information service access method and system based on fragment pattern matching | |
| CN106339432A (en) | System and method for balancing load according to content to be inquired | |
| Gašpar et al. | Integrating Two Worlds: Relational and NoSQL | |
| US11714790B2 (en) | Data unification | |
| CN112948441B (en) | Multi-dimensional data collection method and equipment for financial data | |
| CN113742346A (en) | Asset big data platform architecture optimization method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230920 Address after: 100190 Beijing Haidian District Zhichun Road 49 No. 3 West 309 Applicant after: TENCENT CLOUD COMPUTING (BEIJING) Co.,Ltd. Address before: 518057 Tencent Building, No. 1 High-tech Zone, Nanshan District, Shenzhen City, Guangdong Province, 35 floors Applicant before: TENCENT TECHNOLOGY (SHENZHEN) Co.,Ltd. |