CN113438304A - Data query method, device, server and medium based on database cluster - Google Patents
Data query method, device, server and medium based on database cluster Download PDFInfo
- Publication number
- CN113438304A CN113438304A CN202110696748.XA CN202110696748A CN113438304A CN 113438304 A CN113438304 A CN 113438304A CN 202110696748 A CN202110696748 A CN 202110696748A CN 113438304 A CN113438304 A CN 113438304A
- Authority
- CN
- China
- Prior art keywords
- query
- data query
- proxy server
- data
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 239000012634 fragment Substances 0.000 claims abstract description 70
- 230000005540 biological transmission Effects 0.000 claims description 19
- 238000013467 fragmentation Methods 0.000 claims description 16
- 238000006062 fragmentation reaction Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 12
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000007726 management method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 238000000926 separation method Methods 0.000 description 3
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the field of data processing, and discloses a data query method based on a database cluster, which comprises the following steps: associating at least one proxy server with each database in the database cluster, and defining a master proxy server and a slave proxy server in the proxy servers; receiving a data query task transmitted by a client, and fragmenting the data query task to obtain a plurality of fragmented query tasks; transmitting a plurality of fragment query tasks to a master proxy server, and receiving a distribution result of a slave proxy server of each fragment query task returned by the master proxy server; and according to the distribution result, sending a data query instruction of the fragment query task to the slave proxy server, receiving a data query result of the slave proxy server for performing data query on the database of the database cluster according to the data query instruction, and returning the data query result to the client. In addition, the invention also relates to a block chain technology, and the data query result can be stored in the block chain. The invention can improve the efficiency of data query.
Description
Technical Field
The invention relates to the field of data processing, in particular to a data query method and device based on a database cluster, a fragment server and a computer readable storage medium.
Background
The database cluster is characterized in that at least two or more database servers are utilized to form a virtual single database logical image to provide transparent data services for clients, such as a non-relational database nosql cluster and a relational database mysql cluster, and the database cluster can realize serial processing of data, namely, a database cluster in the database cluster is utilized to solve a data query task, so that the time of the data query task is shortened, and the efficiency of the data query task is improved.
At present, a database cluster usually adopts a middleware form to realize fragmentation management of a database, that is, corresponding middleware is selected based on different service data to distribute a data query task to the database cluster, for example, for web data, the web middleware is selected to distribute the web data query task to the database cluster, and for message data, the message middleware is selected to distribute the message data query task to the database cluster, so that the database cluster is easily limited by the type of query data during data query, and the efficiency of data query is low.
Disclosure of Invention
The invention provides a data query method and device based on a database cluster, a fragment server and a computer readable storage medium, and mainly aims to improve the efficiency of data query.
In order to achieve the above object, the present invention provides a data query method based on a database cluster, where the method is applied to a sharded server, and includes:
associating at least one proxy server for each database in a database cluster, and defining a master proxy server and a slave proxy server in the proxy servers;
receiving a data query task transmitted by a client, and fragmenting the data query task to obtain a plurality of fragmented query tasks;
transmitting the plurality of fragment query tasks to the master proxy server, and receiving a distribution result of each slave proxy server of each fragment query task returned by the master proxy server;
and sending a data query instruction corresponding to the fragment query task to the slave proxy server according to the distribution result, receiving a data query result obtained by the slave proxy server performing data query on a database in the database cluster according to the data query instruction, and returning the data query result to the client.
Optionally, the associating at least one proxy server for each database in the database cluster includes:
and acquiring the IP address and the port of the proxy server, and establishing connection between each database in the database cluster and at least one proxy server according to the IP address and the port.
Optionally, the fragmenting the data query task to obtain a plurality of fragmented query tasks includes:
acquiring a data query field in the data query task, and identifying the data query type of the data query field;
and according to the data query type, fragmenting the data query task to generate a plurality of fragmented query tasks.
Optionally, the transmitting the plurality of fragmentation query tasks to the master proxy server includes:
acquiring a first interface parameter of a fragment server corresponding to a client of each fragment query task and a second interface parameter of the main proxy server;
configuring a data transmission channel of each fragment query task and the main proxy server according to the first interface parameter and the second interface parameter;
and transmitting each fragment query task to the main proxy server according to the data transmission channel.
Optionally, before the receiving the distribution result of the slave proxy server of each of the fragmented query tasks returned by the master proxy server, the method further includes:
creating a data encryption transmission channel of the master proxy server and the slave proxy server;
and distributing at least one slave proxy server to each fragment query task by using the master proxy server according to the data encryption transmission channel to obtain the distribution result of the slave proxy server of each fragment query task.
Optionally, before receiving a data query result of the data query from the proxy server to the database in the database cluster according to the data query instruction, the method further includes:
acquiring a query field of the data query instruction corresponding to the distribution query task;
converting the fragment query task into a corresponding data query statement according to the query field;
and according to the data query statement, performing data query on a database in the database cluster to obtain a data query result.
Optionally, the converting the slicing query task into a corresponding data query statement according to the query field includes:
generating a query statement identifier of each segmented query task according to the query field;
converting the fragment query task into a query statement parameter;
and generating a data query statement of the slicing query task according to the query statement identification and the query statement parameters.
In order to solve the above problem, the present invention further provides a data query apparatus based on a database cluster, where the apparatus is applied to a sharding server, and includes:
the server association module is used for associating at least one proxy server with each database in the database cluster and defining a master proxy server and a slave proxy server in the proxy servers;
the task fragmentation module is used for receiving a data query task transmitted by a client and fragmenting the data query task to obtain a plurality of fragmentation query tasks;
the task transmission module is used for transmitting the plurality of fragment query tasks to the master proxy server and receiving the distribution result of the slave proxy server of each fragment query task returned by the master proxy server;
and the task query module is used for sending a data query instruction corresponding to the fragment query task to the slave proxy server according to the distribution result, receiving a data query result obtained by the slave proxy server performing data query on a database in the database cluster according to the data query instruction, and returning the data query result to the client.
In order to solve the above problem, the present invention further provides a server for sharding, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to implement the database cluster-based data query method described above.
In order to solve the above problem, the present invention further provides a computer-readable storage medium, in which at least one computer program is stored, where the at least one computer program is executed by a processor in a sharded server to implement the database cluster-based data query method described above.
The embodiment of the invention firstly associates the database of the database cluster with the proxy server, responds to the data query task of the client through the proxy server, realizes the data query of the database cluster through the proxy server, and supports the expansibility of the data query; secondly, the embodiment of the invention divides the data query task further by performing the fragmentation task on the data query task, thereby improving the query speed of the data query; furthermore, the embodiment of the invention writes the fragment query task based on the predefined master server, distributes the written fragment query task to the corresponding slave proxy server, and realizes the read-write separation of the data query task by reading the data of the fragment query task from the slave proxy server, thereby further improving the efficiency of data query. Therefore, the data query method, the data query device, the fragment server and the storage medium based on the database cluster can improve the efficiency of data query.
Drawings
Fig. 1 is a system architecture diagram of a database cluster-based data query method according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a database cluster-based data query method according to an embodiment of the present invention;
FIG. 3 is a schematic flowchart illustrating a detailed process of one step of the database cluster-based data query method provided in FIG. 2 according to an embodiment of the present invention;
FIG. 4 is a block diagram of a database cluster-based data query apparatus according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an internal structure of a sharding server for implementing a database cluster-based data query method according to an embodiment of the present invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the invention provides a data query method based on a database cluster. The execution subject of the data query method based on the database cluster includes, but is not limited to, at least one of the sharding servers of the server, the terminal and the like that can be configured to execute the method provided by the embodiment of the present invention. In other words, the database cluster-based data query method may be performed by software or hardware installed in a terminal device or a server device, and the software may be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
Referring to fig. 1, a system architecture diagram of a data query method based on a database cluster according to an embodiment of the present invention is shown. The data query method based on the database cluster can be applied to a system architecture comprising a client 1, a sharding server 2, a main proxy server 3, a plurality of slave proxy servers 4, a database cluster 5 and a plurality of databases 6.
In the embodiment of the invention, the fragment server 4 receives the data query task transmitted by the client 1, and transmits the data query task to the master proxy server 3 after being fragmented, and receives the distribution result of the slave proxy server 4 of the fragmented data query task returned by the master proxy server 3, according to the distribution result, the fragment server 4 sends a data query instruction corresponding to the fragment query task to the slave proxy server 4, and receives the data result of the data query from the proxy server 4 to the database 6 in the database cluster 5 according to the data query instruction, returns the data result to the client 1, wherein, the sharding server 4 associates each database 6 in the database cluster 5 with at least one slave proxy server 4 before receiving the data query task transmitted by the client 1.
Fig. 2 is a schematic flowchart of a data query method based on a database cluster according to an embodiment of the present invention. In an embodiment of the present invention, the data query method based on a database cluster includes:
and S1, associating at least one proxy server for each database in the database cluster, and defining a master proxy server and a slave proxy server in the proxy servers.
In the embodiment of the invention, the database cluster comprises a MySQL database cluster and is used for solving the problem of I/O capacity limitation of a single-node database server caused by the sudden increase of the service data volume and improving the expansibility of the database. It should be understood that the data processing in the database is implemented based on a server, such as an add-drop-modify-check operation of data, where the server is similar to a computing unit to determine a logical structure and a computing manner of each data in the database, and therefore, in the embodiment of the present invention, at least one proxy server is associated with each data in the database cluster to implement functions of subsequent database connection and data query, so as to ensure data throughput in the database and improve data processing performance of the database.
In an alternative embodiment of the invention, the proxy server comprises a ttablet server, which belongs to a component in the cloud native database system, is a proxy server located in front of the database for maximizing throughput of database data executed while protecting the database from harmful queries. The cloud native database system can be a vitess, is a tool set for database cluster management, can automatically store the data table of the database in a fragmentation mode, and rewrites a single query task into a distributed mode to be sent to a plurality of database services.
Further, in one embodiment of the present invention, the associating at least one proxy server for each database in the database cluster includes: and acquiring the IP address and the port of the proxy server, and establishing connection between each database in the database cluster and at least one proxy server in a database connection pool mode according to the IP address and the port so as to avoid the phenomenon that the server is blocked due to overlarge data volume in the subsequent data query task and ensure the reaction of the subsequent data query.
The IP address refers to server IP information of a corresponding proxy server, the port refers to a user name of a port of the logging proxy server, and specific information of the server connected with the database can be determined based on the IP address and the port, so that the phenomenon that the same server is connected with different databases is avoided.
Furthermore, the embodiment of the present invention defines the master proxy server and the slave proxy server in the proxy server to implement read-write separation of subsequent data queries, that is, the master proxy server processes the write transaction of the data query, and the slave database proxy server processes the read transaction of the data query, thereby improving the system performance of the database. The writing transaction refers to a data query event transmitted in the client, and the reading transaction refers to an event for performing data query in the database cluster.
In an optional embodiment, the present invention selects any one proxy server from the proxy servers as a master proxy server by a random method, and uses the remaining proxy servers as slave proxy servers.
And S2, receiving the data query task transmitted by the client, and fragmenting the data query task to obtain a plurality of fragmented query tasks.
In the embodiment of the present invention, the data query task is generated based on different user requirements, for example, the user a needs to query the financial data of a company in the last year, and the user B queries the insurance claim data in the last year, where the client is called a user side and refers to a program corresponding to a server and providing a local service to a client. The sharded server may be a vtgate server of the cloud native database system in S1, where the vtgate is a light proxy server, and is used to route traffic of the data query task to a correct proxy server, so as to perform data query on the database.
Further, in an optional embodiment of the present invention, the fragmenting the data query task to obtain a plurality of fragmented query tasks includes: acquiring a data query field in the data query task, and identifying the data query type of the data query field; and according to the data query type, fragmenting the data query task to generate a plurality of fragmented query tasks.
Wherein the data query field generally comprises: id. And data, name and the like, wherein the data query type is identified based on a corresponding id, for example, if the id is a vehicle insurance policy, the corresponding data query type can be identified as the vehicle insurance policy type, and if the id is an accident insurance policy, the corresponding data query type is identified as the accident insurance policy type.
S3, transmitting the plurality of fragment query tasks to the master proxy server, and receiving the distribution result of the slave proxy server returned by the master proxy server for each fragment query task.
In one embodiment of the present invention, referring to fig. 3, the transmitting the plurality of fragmentation inquiry tasks to the master proxy server includes:
s30, acquiring a first interface parameter of a fragment server corresponding to each fragment query task and a second interface parameter of the main proxy server;
s31, configuring each fragment query task and a data transmission channel of the main proxy server according to the first interface parameter and the second interface parameter;
and S32, transmitting each fragment query task to the main proxy server according to the data transmission channel.
The interface parameters refer to window parameters for data interaction between the server and the third-party platform, and include port parameters, address parameters and operation parameters.
In one embodiment of the present invention, before the receiving that the master proxy server returns the distribution result of the slave proxy server of each of the fragmented query tasks, the method further includes: and distributing at least one slave proxy server to each fragment query task by using the master proxy server to obtain the distribution result of the slave proxy server of each fragment query task.
Further, in another optional implementation of the present invention, the allocating, by using the master proxy server, each of the fragmented query tasks to at least one slave proxy server to obtain an allocation result of the slave proxy server of each of the fragmented query tasks, includes: creating a data encryption transmission channel of the master proxy server and the slave proxy server; and distributing at least one slave proxy server to each fragment query task by using the master proxy server in a load balancing mode according to the data encryption transmission channel to obtain the distribution result of the slave proxy server of each fragment query task.
The data encryption transmission channel is created through the identity authentication of the master node server and the slave proxy server, namely, the data encryption transmission channel is created when the master proxy server successfully authenticates the slave proxy server and the slave proxy server successfully authenticates the master proxy server. Optionally, the identity authentication between the master proxy server and the slave proxy server is implemented by a handshake protocol, which may be an ssl (secure Socket layer) handshake protocol. In an optional embodiment of the present invention, the load balancing of the data query tasks is implemented by using a routing manner, where the routing manner is that traffic bandwidths consumed by executing the data query tasks via the proxy server are consistent.
S4, sending a data query instruction corresponding to the fragment query task to the slave proxy server according to the distribution result, receiving a data query result of the slave proxy server for performing data query to the database in the database cluster according to the data query instruction, and returning the data query result to the client.
In the embodiment of the invention, a data query instruction corresponding to the fragment query task is sent to the slave proxy server according to the distribution result so as to obtain the data query result of each fragment query task.
Further, in this embodiment of the present invention, before receiving a data query result of the data query from the proxy server to the database in the database cluster according to the data query instruction, the method further includes: acquiring a query field of the data query instruction corresponding to the distribution query task; converting the fragment query task into a corresponding data query statement according to the query field; and according to the data query statement, performing data query on a database in the database cluster to obtain a data query result.
In an optional embodiment, the converting the fragmentation query task into a corresponding data query statement according to the query field includes: generating a query statement identifier of each segmented query task according to the query field; converting the fragment query task into a query statement parameter; and generating a data query statement of the slicing query task according to the query statement identification and the query statement parameters.
In an alternative embodiment, the data query statement comprises an sql query statement, such as a select statement, the query statement identifying identity information for characterizing the data query statement, and the query statement parameters comprise a query object and a query attribute.
Further, the embodiment of the present invention returns the data query result to the client.
Further, in order to ensure the security and privacy of the result of the data query, the result of the data query may also be stored in a blockchain node.
The embodiment of the invention firstly associates the database with the proxy server, responds to the data query task of the client through the proxy server, can realize the data query of the database cluster through the proxy server, and supports the expansibility of the data query; secondly, the embodiment of the invention divides the data query task further by performing the fragmentation task on the data query task, thereby improving the query speed of the data query; furthermore, the embodiment of the invention writes the fragment query task based on the predefined master server, distributes the written fragment query task to the corresponding slave proxy server, and realizes the read-write separation of the data query task by reading the data of the fragment query task from the slave proxy server, thereby further improving the efficiency of data query. Therefore, the data query method, the data query device, the fragment server and the storage medium based on the database cluster can improve the efficiency of data query.
Fig. 4 is a functional block diagram of the data query apparatus based on database cluster according to the present invention.
The database cluster-based data query device 100 of the present invention may be installed in a sharding server. According to the implemented functions, the database cluster-based data query device may include a server association module 101, a task fragmentation module 102, a task transmission module 103, and a task query module 104. The module described in this invention can also be called as a unit, and refers to a series of computer program segments that can be executed by the processor of the slice server and can complete fixed functions, and they are stored in the memory of the slice server.
In the present embodiment, the functions regarding the respective modules/units are as follows:
the server association module 101 is configured to associate at least one proxy server with each database in the database cluster, and define a master proxy server and a slave proxy server in the proxy servers;
the task fragmentation module 102 is configured to receive a data query task transmitted by a client, fragment the data query task, and obtain a plurality of fragment query tasks;
the task transmission module 103 is configured to transmit the multiple fragment query tasks to the master proxy server, and receive an allocation result of the slave proxy server of each fragment query task returned by the master proxy server;
the task query module 104 is configured to send a data query instruction corresponding to the fragment query task to the slave proxy server according to the allocation result, receive a data query result obtained by the slave proxy server performing data query on the database in the database cluster according to the data query instruction, and return the data query result to the client.
In detail, when the modules in the data query apparatus 100 based on a database cluster in the embodiment of the present invention are used, the same technical means as the data query method based on a database cluster described in fig. 2 and fig. 3 are adopted, and the same technical effect can be produced, which is not described herein again.
Fig. 5 is a schematic structural diagram of a sharded server for implementing a database cluster-based data query method according to the present invention.
The sharding server 100 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and may further include a computer program stored in the memory 11 and operable on the processor 10, such as a database cluster-based data query program.
In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same function or different functions, and includes one or more Central Processing Units (CPUs), a microprocessor, a digital Processing chip, a graphics processor, a combination of various control chips, and the like. The processor 10 is a Control Unit of the sharding server 100, connects various components of the whole sharding server 100 by using various interfaces and lines, and executes various functions and processes data of the sharding server 100 by running or executing programs or modules stored in the memory 11 (for example, executing a database cluster-based data query program, etc.), and calling data stored in the memory 11.
The memory 11 includes at least one type of readable storage medium including flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory 11 may be an internal storage unit of the sharding server 100 in some embodiments, for example, a mobile hard disk of the sharding server 100. The memory 11 may also be an external storage device of the slice server 100 in other embodiments, for example, a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the slice server 100. Further, the memory 11 may also include both an internal storage unit and an external storage device of the sharding server 100. The memory 11 may be used to store not only the application software installed in the sharding server 100 and various types of data, such as codes of a database cluster-based data query program, but also temporarily store data that has been output or will be output.
The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
The communication interface 13 is used for communication between the fragmentation server 100 and other devices, and includes a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), which is generally used to establish a communication connection between the sharding server 100 and other sharding servers 100. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. Wherein the display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the slicing server 100 and for displaying a visualized user interface.
Fig. 5 shows only the sharded server 100 with components, and those skilled in the art will appreciate that the structure shown in fig. 5 does not constitute a limitation of the sharded server 100, and may include fewer or more components than those shown, or some components in combination, or a different arrangement of components.
For example, although not shown, the sharded server 100 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The fragmentation server 100 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The database cluster-based data query program stored in the memory 11 of the sharding server 100 is a combination of a plurality of computer programs, and when running in the processor 10, can realize:
associating at least one proxy server for each database in a database cluster, and defining a master proxy server and a slave proxy server in the proxy servers;
receiving a data query task transmitted by a client, and fragmenting the data query task to obtain a plurality of fragmented query tasks;
transmitting the plurality of fragment query tasks to the master proxy server, and receiving a distribution result of each slave proxy server of each fragment query task returned by the master proxy server;
and sending a data query instruction corresponding to the fragment query task to the slave proxy server according to the distribution result, receiving a data query result obtained by the slave proxy server performing data query on a database in the database cluster according to the data query instruction, and returning the data query result to the client.
Specifically, the processor 10 may refer to the description of the relevant steps in the embodiment corresponding to fig. 2 for a specific implementation method of the computer program, which is not described herein again.
Further, the integrated modules/units of the sharding server 100, if implemented in the form of software functional units and sold or used as independent products, may be stored in a non-volatile computer-readable storage medium. The computer readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The present invention also provides a computer-readable storage medium, which stores a computer program, which when executed by a processor of the fragmentation server 100, can implement:
associating at least one proxy server for each database in a database cluster, and defining a master proxy server and a slave proxy server in the proxy servers;
receiving a data query task transmitted by a client, and fragmenting the data query task to obtain a plurality of fragmented query tasks;
transmitting the plurality of fragment query tasks to the master proxy server, and receiving a distribution result of each slave proxy server of each fragment query task returned by the master proxy server;
and sending a data query instruction corresponding to the fragment query task to the slave proxy server according to the distribution result, receiving a data query result obtained by the slave proxy server performing data query on a database in the database cluster according to the data query instruction, and returning the data query result to the client.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A data query method based on database cluster is applied to a sharding server, and comprises the following steps:
associating at least one proxy server for each database in a database cluster, and defining a master proxy server and a slave proxy server in the proxy servers;
receiving a data query task transmitted by a client, and fragmenting the data query task to obtain a plurality of fragmented query tasks;
transmitting the plurality of fragment query tasks to the master proxy server, and receiving a distribution result of each slave proxy server of each fragment query task returned by the master proxy server;
and sending a data query instruction corresponding to the fragment query task to the slave proxy server according to the distribution result, receiving a data query result obtained by the slave proxy server performing data query on a database in the database cluster according to the data query instruction, and returning the data query result to the client.
2. The database cluster-based data query method of claim 1, wherein said associating at least one proxy server for each database in the database cluster comprises:
and acquiring the IP address and the port of the proxy server, and establishing connection between each database in the database cluster and at least one proxy server according to the IP address and the port.
3. The database cluster-based data query method of claim 1, wherein the fragmenting the data query task to obtain a plurality of fragmented query tasks comprises:
acquiring a data query field in the data query task, and identifying the data query type of the data query field;
and according to the data query type, fragmenting the data query task to generate a plurality of fragmented query tasks.
4. The database cluster-based data query method of claim 1, wherein the transmitting the plurality of sharded query tasks to the master proxy server comprises:
acquiring a first interface parameter of a fragment server corresponding to a client of each fragment query task and a second interface parameter of the main proxy server;
configuring a data transmission channel of each fragment query task and the main proxy server according to the first interface parameter and the second interface parameter;
and transmitting each fragment query task to the main proxy server according to the data transmission channel.
5. The database cluster-based data query method of claim 1, wherein before receiving the assignment result of the slave proxy server of each of the fragmented query tasks returned by the master proxy server, further comprising:
creating a data encryption transmission channel of the master proxy server and the slave proxy server;
and distributing at least one slave proxy server to each fragment query task by using the master proxy server according to the data encryption transmission channel to obtain the distribution result of the slave proxy server of each fragment query task.
6. The database cluster-based data query method according to any one of claims 1 to 5, wherein before receiving the data query result of the data query from the proxy server to the database in the database cluster according to the data query instruction, the method further comprises:
acquiring a query field of the data query instruction corresponding to the distribution query task;
converting the fragment query task into a corresponding data query statement according to the query field;
and according to the data query statement, performing data query on a database in the database cluster to obtain a data query result.
7. The database cluster-based data query method of claim 6, wherein said converting the sharded query task into corresponding data query statements according to the query field comprises:
generating a query statement identifier of each segmented query task according to the query field;
converting the fragment query task into a query statement parameter;
and generating a data query statement of the slicing query task according to the query statement identification and the query statement parameters.
8. A data query device based on database cluster, which is applied to a sharding server, comprises:
the server association module is used for associating at least one proxy server with each database in the database cluster and defining a master proxy server and a slave proxy server in the proxy servers;
the task fragmentation module is used for receiving a data query task transmitted by a client and fragmenting the data query task to obtain a plurality of fragmentation query tasks;
the task transmission module is used for transmitting the plurality of fragment query tasks to the master proxy server and receiving the distribution result of the slave proxy server of each fragment query task returned by the master proxy server;
and the task query module is used for sending a data query instruction corresponding to the fragment query task to the slave proxy server according to the distribution result, receiving a data query result obtained by the slave proxy server performing data query on a database in the database cluster according to the data query instruction, and returning the data query result to the client.
9. A sharding server, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the database cluster-based data query method of any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, implements a database cluster-based data query method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110696748.XA CN113438304B (en) | 2021-06-23 | 2021-06-23 | Data query method, device, server and medium based on database cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110696748.XA CN113438304B (en) | 2021-06-23 | 2021-06-23 | Data query method, device, server and medium based on database cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113438304A true CN113438304A (en) | 2021-09-24 |
| CN113438304B CN113438304B (en) | 2023-04-07 |
Family
ID=77753488
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110696748.XA Active CN113438304B (en) | 2021-06-23 | 2021-06-23 | Data query method, device, server and medium based on database cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113438304B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114327261A (en) * | 2021-12-06 | 2022-04-12 | 神州融安数字科技(北京)有限公司 | Data file storage method and data security agent |
| CN114840268A (en) * | 2022-04-25 | 2022-08-02 | 平安普惠企业管理有限公司 | Service code execution method and device, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102497454A (en) * | 2011-12-31 | 2012-06-13 | 北京新媒传信科技有限公司 | Method for realizing gray publication of application in application service platform system |
| CN103116655A (en) * | 2013-03-06 | 2013-05-22 | 亿赞普(北京)科技有限公司 | Clustered data query method, client side and system |
| CN103942098A (en) * | 2014-04-29 | 2014-07-23 | 国家电网公司 | System and method for task processing |
| CN107273540A (en) * | 2017-07-05 | 2017-10-20 | 北京三快在线科技有限公司 | Distributed search and index updating method, system, server and computer equipment |
| CN110990400A (en) * | 2019-11-20 | 2020-04-10 | 浙江大搜车软件技术有限公司 | Database query method and device, computer equipment and storage medium |
| CN111930770A (en) * | 2020-07-15 | 2020-11-13 | 北京金山云网络技术有限公司 | Data query method and device and electronic equipment |
| US20210132987A1 (en) * | 2019-10-31 | 2021-05-06 | TmaxData Co., Ltd. | Computer program for asynchronous data processing in a database management system |
-
2021
- 2021-06-23 CN CN202110696748.XA patent/CN113438304B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102497454A (en) * | 2011-12-31 | 2012-06-13 | 北京新媒传信科技有限公司 | Method for realizing gray publication of application in application service platform system |
| CN103116655A (en) * | 2013-03-06 | 2013-05-22 | 亿赞普(北京)科技有限公司 | Clustered data query method, client side and system |
| CN103942098A (en) * | 2014-04-29 | 2014-07-23 | 国家电网公司 | System and method for task processing |
| CN107273540A (en) * | 2017-07-05 | 2017-10-20 | 北京三快在线科技有限公司 | Distributed search and index updating method, system, server and computer equipment |
| US20210132987A1 (en) * | 2019-10-31 | 2021-05-06 | TmaxData Co., Ltd. | Computer program for asynchronous data processing in a database management system |
| CN110990400A (en) * | 2019-11-20 | 2020-04-10 | 浙江大搜车软件技术有限公司 | Database query method and device, computer equipment and storage medium |
| CN111930770A (en) * | 2020-07-15 | 2020-11-13 | 北京金山云网络技术有限公司 | Data query method and device and electronic equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114327261A (en) * | 2021-12-06 | 2022-04-12 | 神州融安数字科技(北京)有限公司 | Data file storage method and data security agent |
| CN114840268A (en) * | 2022-04-25 | 2022-08-02 | 平安普惠企业管理有限公司 | Service code execution method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113438304B (en) | 2023-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113438304B (en) | Data query method, device, server and medium based on database cluster | |
| CN111538594B (en) | Order ID generation method, device, equipment and medium based on distributed system | |
| CN111767268B (en) | Database table partitioning method and device, electronic equipment and storage medium | |
| CN112417503A (en) | Method and device for authorizing post authority, electronic equipment and storage medium | |
| CN112115145A (en) | Data acquisition method and device, electronic equipment and storage medium | |
| CN112580079A (en) | Authority configuration method and device, electronic equipment and readable storage medium | |
| CN112084486A (en) | User information verification method and device, electronic equipment and storage medium | |
| CN114185895A (en) | Data import and export method and device, electronic equipment and storage medium | |
| CN107562521B (en) | Resource management method and device | |
| WO2022048362A1 (en) | Data storage method and apparatus, electronic device, and storage medium | |
| CN113297180A (en) | Data migration method and device, electronic equipment and storage medium | |
| CN112685384A (en) | Data migration method and device, electronic equipment and storage medium | |
| CN112085611A (en) | Asynchronous data verification method and device, electronic equipment and storage medium | |
| CN113542387B (en) | System release method and device, electronic equipment and storage medium | |
| CN112257078B (en) | Block chain encryption and decryption service security trusted system based on TEE technology | |
| CN112328656B (en) | Service query method, device, equipment and storage medium based on middle platform architecture | |
| CN115174555A (en) | File transmission method and device, electronic equipment and storage medium | |
| CN115033605A (en) | Data query method and device, electronic equipment and storage medium | |
| CN112925753B (en) | File additional writing method and device, electronic equipment and storage medium | |
| CN113364848B (en) | File caching method and device, electronic equipment and storage medium | |
| CN114897624A (en) | Policy data merging method, device, equipment and storage medium | |
| CN113268514A (en) | Multidimensional data statistical method and device, electronic equipment and storage medium | |
| CN112487400A (en) | Single sign-on method and device based on multiple pages, electronic equipment and storage medium | |
| CN112596675A (en) | Multi-storage method and device of data, electronic equipment and computer storage medium | |
| CN113434359B (en) | Data traceability system construction method and device, electronic device and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |