CN116628034A - Cross-microservice and cross-library joint query method based on wine industry - Google Patents
Cross-microservice and cross-library joint query method based on wine industry Download PDFInfo
- Publication number
- CN116628034A CN116628034A CN202310630340.1A CN202310630340A CN116628034A CN 116628034 A CN116628034 A CN 116628034A CN 202310630340 A CN202310630340 A CN 202310630340A CN 116628034 A CN116628034 A CN 116628034A
- Authority
- CN
- China
- Prior art keywords
- data
- cross
- database
- query
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 230000008859 change Effects 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000012423 maintenance Methods 0.000 description 3
- 238000004220 aggregation Methods 0.000 description 2
- 241001178520 Stomatepia mongo Species 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000000178 monomer Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24558—Binary matching operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of software, in particular to a cross-microservice and cross-library joint query method based on the wine industry, which comprises the following steps: the workflow data and the business data are associated through a field; storing the associated data in addition to a document-type database; the data synchronization is carried out on the basis of a Canal and Flink CDC tool realized by the database log; acquiring field information of a related table, constructing a database query statement, and performing conditional query; the beneficial effects are as follows: the cross-micro service and cross-library joint query method based on the wine industry uses a non-relational database to store wide-table data, and uses a data synchronization tool to synchronize the data; the method provides a solution to the problem of cross-service joint query commonly found in software development.
Description
Technical Field
The invention relates to the technical field of software, in particular to a cross-microservice and cross-library joint query method based on the wine industry.
Background
Modern software functions are more and more complex and implementation behind the software is more and more difficult. Traditional software architecture is difficult to meet the requirement of the present-day software development, both development and maintenance are carried out, bottlenecks are met, and software architecture upgrading is urgent.
In the prior art, most applications are accumulated from simple to complex, and to a certain extent, the applications become more and more huge and are more difficult to maintain, besides the applications themselves, middleware (such as a database, a cache and the like) used by the applications may have performance bottlenecks, and splitting a single service into a plurality of micro services tends to be a trend. Also, some middleware needs to be split with the service. The micro-service can solve the problems of huge and bulky application of the monomer, difficult maintenance, complex structure, difficult operation and maintenance and the like, but other problems can be introduced.
Disclosure of Invention
The invention aims to provide a cross-micro service and cross-library combined query method based on the wine industry, so as to solve the problems in the background technology.
In order to achieve the above purpose, the present invention provides the following technical solutions: a cross-micro service and cross-library combined query method based on wine industry comprises the following steps:
the workflow data and the business data are associated through a field;
storing the associated data in addition to a document-type database;
the data synchronization is carried out on the basis of a Canal and Flink CDC tool realized by the database log;
and acquiring field information of the related table, constructing a database query statement, and carrying out conditional query.
Preferably, when the workflow data and the business data are associated through the fields, the workflow data and the business data are in the same database, and the association inquiry is carried out when the workflow data and the business data are inquired; after splitting the service into workflow service and business service, the database is correspondingly split into workflow library and business library, each service only accesses the corresponding database, and the service can not be associated with the query any more, and the data association is carried out by calling between the services.
Preferably, when the associated data is additionally stored in the document type database, the data format is converted into a stereoscopic object document from the original flat data, and the object nest represents the association relationship between the data.
Preferably, when the Canal and Flink CDC tools realized based on the database log perform data synchronization, the data synchronization tool performs data complementation after monitoring the data change, and stores the completed data in the document database.
Preferably, when the field information of the related table is acquired, the field, the name corresponding to the field, the type of the field and the length of the field are acquired from the database schema.
Compared with the prior art, the invention has the beneficial effects that:
the cross-micro service and cross-library joint query method based on the wine industry uses a non-relational database to store wide-table data, and uses a data synchronization tool to synchronize the data; the method provides a solution to the problem of cross-service joint query commonly found in software development.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a flow chart of the data query of the present invention;
FIG. 3 is a schematic diagram of the object nested representation data association relationship of the present invention.
Detailed Description
In order to make the objects, technical solutions, and advantages of the present invention more apparent, the embodiments of the present invention will be further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are some, but not all, embodiments of the present invention, are intended to be illustrative only and not limiting of the embodiments of the present invention, and that all other embodiments obtained by persons of ordinary skill in the art without making any inventive effort are within the scope of the present invention.
Example 1
Referring to fig. 1 to 2, the present invention provides a technical solution: a cross-micro service and cross-library combined query method based on wine industry comprises the following steps:
the workflow data and the business data are associated through a field;
storing the associated data in addition to a document-type database;
the data synchronization is carried out on the basis of a Canal and Flink CDC tool realized by the database log;
and acquiring field information of the related table, constructing a database query statement, and carrying out conditional query.
Example two
Based on the first embodiment, the specific implementation manner is as follows:
one flow system contains at least two parts of data, one part is workflow related data, as shown in table 1, and is counted as a table workflow_def, and the other part is business data related to the flow, as shown in table 2, and is counted as a table business.
| Fields | Type(s) | Description of the invention |
| id | char | Main key |
| name | varchar | Flow definition name |
| proc_def_id | char | Flow definition id |
| owner | varchar | Flow owner |
| create_time | datetime | Flow creation time |
TABLE 1 workflow part field
| Fields | Type(s) | Description of the invention |
| id | char | Main key |
| title | varchar | Title of the book |
| proc_def_id | char | Flow definition id |
| use_date | datetime | Time of occurrence |
Table 2 service part field
The workflow data is associated with the business data through one or more fields. When the workflow data and the business data are in the same database, the association inquiry can be carried out during inquiry. After splitting the service into workflow service and business service, the database is correspondingly split into workflow library and business library, each service only accesses the corresponding database, and the related query can not be performed any more during the query, and the data association can only be performed by calling between the services. Taking the above two tables as an example, according to 10 flows with the creation time ordering query flow owners of Zhang San, the workflow service queries data and then takes out all flow definition ids, then uses the ids as parameters to call business service, the business service queries the data of the appointed flow definition ids and returns, the flow is shown in figure 1,
if both libraries now have query conditions, this approach cannot be implemented, and the workflow service queried data does not necessarily match the flow definition id of the business service queried data, as in fig. 2.
For this problem, the present patent provides a solution to store the associated data in a document-type database, such as Elasticsearch, mongoDB. The data format is converted into a stereoscopic object document from the original flat data, and the object nest represents the association relationship between the data, as shown in fig. 3:
after the introduction of the additional database, the problem of data synchronization between the relational database and the document database needs to be solved. The data synchronization can be performed by using tools such as Canal and Flink CDC realized based on database logs, and the data synchronization tool performs data completion after monitoring the data change and saves the completed data into a document database.
The addition, deletion and examination are the natural behavior of the data, and most of the data can be modified and inquired. For most business systems, querying data is a frequent operation. Most queries are also queries specifying fields, such as looking up the flow definition name xx data in the workflow table workflow _ def,
SELECT*FROM workflow_def WHERE name＝‘xx’
if a plurality of conditions exist, the conditions are needed to be spliced, if the conditions change frequently, the program is needed to be changed frequently, and the user-defined query conditions are difficult to realize for common query.
To realize the general query, firstly, field information of a related table needs to be acquired, at least fields, names corresponding to the fields, types of the fields, lengths of the fields and the like need to be acquired, and the fields can be acquired from a database schema, but the acquisition of the fields directly from the schema does not necessarily meet the requirements: using notes as field names, the notes may contain some content for the developer that is not suitable for the user to show; not all fields are provided to the user, some are not necessarily provided to the user, such as id, etc.; the original table fields may need to be processed to be returned to the user. For the above reasons, the present patent provides a solution that uses a Langchaws cloud data lake product or other related tools, through which database table fields are extracted, and provides the ability to maintain these fields, through which usable fields are specified by tags, or through which only usable fields are extracted when extracting fields. The dictionary is maintained by a dictionary to maintain the value objects of some fields, the code represents the value stored in the database, the label represents the name shown by the value object, such as a gender field representing gender in the database, the dictionary is configured to be [ { code: '0', label: 'men' }, { code: '1', label: 'women' }, and the gender value is 1.
The search boxes presented by the different field types are different, the fields of the number type can present a range of number input boxes, the fields of the date type can present a range of date and time input boxes, and the rest of the general fields can present text input boxes. When the user side displays the search box, the type of the field needs to be judged, and when searching is carried out, the user input content and the field information are transmitted to the back end, and the back end constructs the query condition. The backend needs to judge the type of field again when building the query condition, which is in fact unnecessary, since the user has already judged once. The user end and the back end can agree with the common query conditions, such as date, number type, data matching EQUAL, fuzzy matching LIKE, other possible IN, NOT_IN and the LIKE, so that the back end can construct the query only according to the fields, conditions, search values and the LIKE. Taking Java as an example, the backend defines query condition enumeration,
the query condition is used to receive the query condition transmitted by the user terminal,
query conditions need to be built into database query sentences to query, each query condition is built according to the type of query, the eQUAL is converted into field_a= 'xx', the LIKE is converted into field_b LIKE 'yy', the IN is converted into field_cIN ('oo', 'pp'), and the LIKE. In mongo db, the query conditions may be constructed as criterion objects, the transformation process is, in each query condition,
MongoDB uses $match aggregation for conditional queries, since there are multiple query conditions, and the conditions are "and" in relation to each other, it is necessary to place the conditions in $and sub-aggregations,
although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (5)
1. A cross-micro service and cross-library joint query method based on wine industry is characterized in that: the joint query method comprises the following steps:
the workflow data and the business data are associated through a field;
storing the associated data in addition to a document-type database;
the data synchronization is carried out on the basis of a Canal and Flink CDC tool realized by the database log;
and acquiring field information of the related table, constructing a database query statement, and carrying out conditional query.
2. The cross-microservice and cross-library joint query method based on the wine industry as claimed in claim 1, wherein the method is characterized by comprising the following steps: when the workflow data and the business data are associated through the fields, the workflow data and the business data are in the same database, and the association inquiry is carried out when the workflow data and the business data are inquired; after splitting the service into workflow service and business service, the database is correspondingly split into workflow library and business library, each service only accesses the corresponding database, and the service can not be associated with the query any more, and the data association is carried out by calling between the services.
3. The cross-microservice and cross-library joint query method based on the wine industry as claimed in claim 1, wherein the method is characterized by comprising the following steps: when the associated data is additionally stored in the document type database, the data format is converted into a three-dimensional object document from the original flat data, and the object nest represents the association relation between the data.
4. The cross-microservice and cross-library joint query method based on the wine industry as claimed in claim 1, wherein the method is characterized by comprising the following steps: when the Canal and Flink CDC tools realized based on the database logs perform data synchronization, the data synchronization tool performs data complementation after monitoring the data change, and stores the completed data into the document database.
5. The cross-microservice and cross-library joint query method based on the wine industry as claimed in claim 1, wherein the method is characterized by comprising the following steps: when the field information of the related table is acquired, the field, the name corresponding to the field, the type of the field and the length of the field are acquired from the database schema.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310630340.1A CN116628034A (en) | 2023-05-31 | 2023-05-31 | Cross-microservice and cross-library joint query method based on wine industry |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310630340.1A CN116628034A (en) | 2023-05-31 | 2023-05-31 | Cross-microservice and cross-library joint query method based on wine industry |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116628034A true CN116628034A (en) | 2023-08-22 |
Family
ID=87637907
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310630340.1A Pending CN116628034A (en) | 2023-05-31 | 2023-05-31 | Cross-microservice and cross-library joint query method based on wine industry |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116628034A (en) |
-
2023
- 2023-05-31 CN CN202310630340.1A patent/CN116628034A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8375044B2 (en) | Query processing pipelines with single-item and multiple-item query operators | |
| CA2484009C (en) | Managing expressions in a database system | |
| CN111506621B (en) | Data statistical method and device | |
| US9665607B2 (en) | Methods and apparatus for organizing data in a database | |
| US9251212B2 (en) | Profiling in a massive parallel processing environment | |
| US20120246154A1 (en) | Aggregating search results based on associating data instances with knowledge base entities | |
| US20040128276A1 (en) | System and method for accessing data in disparate information sources | |
| US8880463B2 (en) | Standardized framework for reporting archived legacy system data | |
| CN104731945B (en) | A kind of text searching method and device based on HBase | |
| CN106326429A (en) | Hbase second-level query scheme based on solr | |
| CN102262640A (en) | Method and device for full-text retrieval of document database | |
| CN111506559A (en) | Data storage method and device, electronic equipment and storage medium | |
| CN107291964A (en) | A kind of method that fuzzy query is realized based on HBase | |
| CN112579610A (en) | Multi-data source structure analysis method, system, terminal device and storage medium | |
| US9053207B2 (en) | Adaptive query expression builder for an on-demand data service | |
| CN113779349A (en) | Data retrieval system, apparatus, electronic device, and readable storage medium | |
| CN107169003B (en) | Data association method and device | |
| CN114328981A (en) | Knowledge graph establishing and data obtaining method and device based on mode mapping | |
| CN114064660A (en) | Data structured analysis method based on ElasticSearch | |
| CN116842142B (en) | Intelligent retrieval system for medical instrument | |
| CN116628034A (en) | Cross-microservice and cross-library joint query method based on wine industry | |
| CN106874498B (en) | Financial data access method and access device | |
| CN111143329B (en) | Data processing method and device | |
| CN110609926A (en) | Data tag storage management method and device | |
| CN110704421A (en) | Data processing method, device, equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |