CN106371932B - Zookeeper-based information configuration method and device - Google Patents
Zookeeper-based information configuration method and device Download PDFInfo
- Publication number
- CN106371932B CN106371932B CN201610755159.3A CN201610755159A CN106371932B CN 106371932 B CN106371932 B CN 106371932B CN 201610755159 A CN201610755159 A CN 201610755159A CN 106371932 B CN106371932 B CN 106371932B
- Authority
- CN
- China
- Prior art keywords
- configuration information
- read
- local disk
- zookeeper
- monitoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 399
- 238000012544 monitoring process Methods 0.000 claims abstract description 66
- 238000012545 processing Methods 0.000 claims description 11
- 238000004891 communication Methods 0.000 claims description 5
- 230000000977 initiatory effect Effects 0.000 claims 2
- 238000010586 diagram Methods 0.000 description 7
- 230000009286 beneficial effect Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
- G06F9/526—Mutual exclusion algorithms
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Telephonic Communication Services (AREA)
Abstract
The invention provides a zookeeper-based information configuration method and a zookeeper-based information configuration device, wherein the method comprises the steps of starting a process to occupy a lock between processes, establishing long connection between the process and a zookeeper server, monitoring all configuration information in a local disk through the process, and acquiring corresponding configuration information from the zookeeper server and updating the configuration information into the local disk by the process according to a received configuration information updating notice, so that the process of the configuration information to be read only needs to search the corresponding configuration information from the local disk. It can be seen that the present invention can reduce the load of zookeeper servers.
Description
The patent application of the invention is a divisional application of a Chinese invention patent application with the application date of 2013, 10 and 11 months and the application number of 201310472913.9 and the name of 'a zookeeper-based information configuration method and device'.
Technical Field
The invention relates to the technical field of computers, in particular to a zookeeper-based information configuration method and device.
Background
With the continuous development of computer technology, a distributed system is generally adopted in network applications, a plurality of servers are used for providing services together, each server has hundreds of processes, the process configuration of the servers is generally the same, and when the process configuration needs to be modified, the process configuration in all the servers needs to be modified.
Zookeeper is currently used to centrally manage all process configuration information in multiple servers. zoo managers are thought by zoo, and zoo is a reliable coordination system aiming at a large-scale distributed system in the technical field of computers, and the provided functions comprise: configuration maintenance, name service, distributed synchronization, group service, etc. The existing information configuration scheme based on zookeeper is as follows: each process in each server needs to establish long connection with the zookeeper server, each process monitors the configuration information of the process by calling an interface of the zookeeper server, when the configuration information of one process needs to be modified, an operator only needs to modify the corresponding configuration information on the zookeeper server, and once the configuration information of the zookeeper server is found to be changed, the corresponding server process updates the configuration information of the process in time.
Since often hundreds of servers are commonly served in a real application, each server typically runs at least hundreds of processes, such as: if 100 servers provide services together, and when 200 processes run on each server, each process in each server establishes a long connection with a zookeeper server, and 20000 connections are required to be established in total, it can be seen that this configuration mode brings a large load to the zookeeper server, and the processing performance of the zookeeper server is seriously affected.
Disclosure of Invention
The information configuration method and device based on the zookeeper are used for effectively reducing the load of the zookeeper server while ensuring that all process configuration information in the server is monitored.
Therefore, the invention provides the following technical scheme:
in a first aspect, the present invention provides a zookeeper-based information configuration method, including: starting a process to occupy a lock between processes, establishing long connection between the process and a zookeeper server, and monitoring all configuration information in a local disk through the process; and the process acquires corresponding configuration information from the zookeeper server according to the received configuration information updating notification and updates the configuration information to a local disk.
In another preferred embodiment, the updated configuration information notification is sent by the zookeeper server to the process when a change in configuration information occurs.
In another preferred embodiment, the configuration information updated to the local disk is used for a process of the configuration information to be read to search, so as to read the configuration information corresponding to the process of the configuration information to be read.
In another preferred embodiment, if the corresponding configuration information is found, the process to be read reads the configuration information corresponding to the process to be read from the local disk, and if the corresponding configuration information is not found, the process to be read establishes a short connection with the zookeeper server, and acquires the configuration information corresponding to the process to be read from the zookeeper server and stores the configuration information in the local disk.
In another preferred embodiment, after updating the corresponding configuration information acquired from the zookeeper server to the local disk, the method further includes:
and updating the corresponding configuration information acquired from the zookeeper server into a shared memory.
In another preferred embodiment, the configuration information updated to the local disk and the shared memory is used for the process of the configuration information to be read to search, so as to read the configuration information corresponding to the process of the configuration information to be read.
In another preferred embodiment, the process of the configuration information to be read firstly searches in the shared memory, and if the corresponding configuration information is found, the process of the configuration information to be read reads the corresponding configuration information from the shared memory; if the corresponding configuration information is not found, the process of the configuration information to be read searches the local disk, if the corresponding configuration information is found, the process of the configuration information to be read reads the corresponding configuration information from the local disk, if the corresponding configuration information is not found, the process of the configuration information to be read establishes short connection with the zookeeper server, and the corresponding configuration information is obtained from the zookeeper server and stored in the shared memory and the local disk.
In another preferred embodiment, the starting a process occupies a lock between processes, and establishes a long connection between the process and the zookeeper server, and the process monitors all configuration information in the local disk, including:
starting a process special for monitoring process configuration information to occupy the lock between processes;
and establishing a long connection between the process special for monitoring the process configuration information and the zookeeper server, and monitoring all the configuration information in the local disk through the process special for monitoring the configuration information.
In another preferred embodiment, the starting a process occupies a lock between processes, and establishes a long connection between the process and the zookeeper server, and the process monitors all configuration information in the local disk, including:
starting a lock between processes occupied by the process to be read with the configuration information;
starting a thread or a process in the process of the configuration information to be read, establishing long connection between the thread or the process and the zookeeper server, and monitoring all the configuration information in the local disk through the thread or the process.
In another preferred embodiment, one thread started in the process to read the configuration information is an asynchronous thread.
In a second aspect, the present invention provides a zookeeper-based information configuring apparatus, the apparatus comprising:
the system comprises a monitoring unit, a data processing unit and a data processing unit, wherein the monitoring unit is used for starting a process to occupy a lock between processes, establishing long connection between the process and a zookeeper server, and monitoring all configuration information in a local disk through the process;
and the updating unit is used for acquiring corresponding configuration information from the zookeeper server through the process according to the update configuration information notification and updating the configuration information into a local disk.
In another preferred embodiment, the updated configuration information notification is sent by the zookeeper server to the process when a change in configuration information occurs.
In another preferred embodiment, the configuration information updated to the local disk is used for a process of the configuration information to be read to search, so as to read the configuration information corresponding to the process of the configuration information to be read.
In another preferred embodiment, if the corresponding configuration information is found, the process to be read reads the configuration information corresponding to the process to be read from the local disk, and if the corresponding configuration information is not found, the process to be read establishes a short connection with the zookeeper server, and acquires the configuration information corresponding to the process to be read from the zookeeper server and stores the configuration information in the local disk.
In another preferred embodiment, the apparatus further comprises:
and the storage unit is used for updating the corresponding configuration information acquired from the zookeeper server into a shared memory.
In another preferred embodiment, the configuration information updated to the local disk and the shared memory is used for the process of the configuration information to be read to search, so as to read the configuration information corresponding to the process of the configuration information to be read.
In another preferred embodiment, the process of the configuration information to be read firstly searches in the shared memory, and if the corresponding configuration information is found, the process of the configuration information to be read reads the corresponding configuration information from the shared memory; if the corresponding configuration information is not found, the process of the configuration information to be read searches the local disk, if the corresponding configuration information is found, the process of the configuration information to be read reads the corresponding configuration information from the local disk, if the corresponding configuration information is not found, the process of the configuration information to be read establishes short connection with the zookeeper server, and the corresponding configuration information is obtained from the zookeeper server and stored in the shared memory and the local disk.
In another preferred embodiment, the monitoring unit includes:
the first possessor unit is used for starting a lock which is specially used for monitoring the process configuration information and used for occupying the process space;
and the first monitoring subunit is used for establishing a long connection between the process special for monitoring the process configuration information and the zookeeper server, and monitoring all the configuration information in the local disk through the process special for monitoring the configuration information.
In another preferred embodiment, the monitoring unit includes:
the second possessor unit is used for starting the lock of the progress occupation progress of the configuration information to be read;
and the second monitoring subunit is used for starting a thread or a process in the process of the configuration information to be read, establishing long connection between the thread or the process and the zookeeper server, and monitoring all the configuration information in the local disk through the thread or the process.
In another preferred embodiment, the second monitoring subunit is configured to start an asynchronous thread in the process to be read with the configuration information, establish a long connection between the asynchronous process and the zookeeper server, and monitor the configuration information of all processes in the inter-process communication through the asynchronous process.
Compared with the prior art, the invention has the beneficial effects that:
by starting a process to occupy the lock among the processes, other processes in the time period of modifying the shared data by the process cannot perform any operation on the shared data, and other processes in the time period of modifying the shared data by the process can immediately know that the long connection between the process and the zookeeper server is established, and all configuration information in the local disk is monitored by the process.
According to a received notification of updating configuration information sent by a zookeeper server, the process acquires corresponding configuration information from the zookeeper server and updates the configuration information into a local disk, when any one process runs and needs to acquire the configuration information, only the local disk needs to be searched for the corresponding configuration information, and long connection with the zookeeper server is not needed; if the local disk does not have corresponding configuration information, it is indicated that the relevant configuration information of the process is not monitored, in this case, only a short connection is needed to be established with the zookeeper server, the configuration information of the process is acquired from the zookeeper server and stored in the local disk, so that the configuration information of the process is also monitored. It can be seen from this that: by using the method of the invention, each server does not need to establish long connection between each process and the zookeeper server, but only establishes long connection and initial short connection between one process and the zookeeper server by using the sharing of a lock between processes and a local disk, thereby ensuring that the configuration information of all processes is monitored and simultaneously reducing the load of the zookeeper server.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
Fig. 1 is a schematic flowchart of an embodiment 1 of a zookeeper-based information configuration method according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of an embodiment 2 of a zookeeper-based information configuration method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a zookeeper-based information configuration framework according to an embodiment of the present invention;
FIG. 4 is a prior art zookeeper-based information configuration framework schematic;
fig. 5 is a schematic diagram of an information configuring apparatus embodiment 1 based on zookeeper according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an information configuring apparatus embodiment 2 based on zookeeper according to an embodiment of the present invention.
Detailed Description
In order to make the technical field of the invention better understand the scheme of the invention, the following detailed description of the embodiments of the invention is provided in conjunction with the accompanying drawings and the implementation mode.
Referring to fig. 1, it is shown that fig. 1 is a schematic flowchart of an embodiment 1 of a zookeeper-based information configuration method according to an embodiment of the present invention, where the method includes:
step 101, starting a process to occupy a lock between processes, establishing a long connection between the process and a zookeeper server, and monitoring all configuration information in a local disk through the process.
The process is the basis of an operating system structure and is an executing program, and when any process in the server starts running, configuration information needs to be read first, and then program flow processing is carried out according to the configuration.
The embodiment of the invention starts the lock of one process occupying the processes so as to realize the resource sharing among the processes in the server, when one process modifies the shared data, other processes can not operate the shared data at the same time, any process operates the shared data, and other processes can see the shared data immediately. To achieve this, a server system kernel is required to provide a lock, and the specific implementation of the lock is as follows: file lock (record lock), System V semaphore lock (semaphore), mutex lock, condition variable, read-write lock, etc. File locks and semaphore locks are generally used for synchronization between processes, and mutex locks, conditional variables, and read-write locks are generally used for synchronization between threads, and are specifically determined according to actual conditions, and are not specifically limited herein.
In this step, a process is started to occupy the lock between processes, a long connection between the process and the zookeeper server is established, and all configuration information in the local disk is monitored through the process.
The first implementation manner can include:
starting a process special for monitoring process configuration information to occupy the lock between processes;
and establishing long connection between the process special for monitoring the process configuration information and the zookeeper server, and monitoring all the configuration information in the local disk through the process special for monitoring the process configuration information.
The server starts a process dedicated to monitoring the configuration information of the process, which is only used to monitor the configuration information of all processes in the server. The process possesses the lock between the processes to realize the synchronization between all the processes, the actual function of the lock is that when the process specially used for monitoring the process configuration information writes data in the local disk, other processes can not perform any operation on the data of the local disk, and similarly, when other processes read data from the local disk, the process specially used for monitoring the process configuration information can not perform any operation on the local disk, so that the process possessing the lock between the processes can ensure that all the processes do not interfere with the operation of the local disk.
The second implementation manner may include:
starting the lock of the process occupation of the configuration information to be read;
starting a thread or a process in the process of the configuration information to be read, establishing long connection between the thread or the process and the zookeeper server, and monitoring all the configuration information in the local disk through the thread or the process.
When a process in the server needs to read the configuration information, the process using the configuration information to be read occupies the lock between processes, and the specific function of the lock is the same as the above function, which is not described herein again.
The process to be read the configuration information can start a long connection between a thread and the zookeeper server, and monitor all the configuration information in the local disk through the thread; or, the process to be read for configuration information starts a long connection between one process and the zookeeper server, and monitors all configuration information in the local disk through the process.
In order not to affect the main process execution rate of the process and not to waste system resources, the embodiment of the invention also provides an optional scheme.
As an optional solution in the embodiment of the present invention, a thread started by the process to read the configuration information may be an asynchronous thread.
Because the process and the zookeeper server are in long connection, when an operator needs to modify the configuration, the operator only needs to modify the corresponding configuration information on the zookeeper server, and when the configuration information monitored in the zookeeper server changes, an update configuration information notification is immediately sent to the process.
And step 102, the process acquires corresponding configuration information from the zookeeper server according to the received updated configuration information notification and updates the configuration information to a local disk.
Wherein the updated configuration information notification may be sent by the zookeeper server to the process when the configuration information changes. The configuration information updated to the local disk is used for searching for the process of the configuration information to be read so as to read the configuration information corresponding to the process of the configuration information to be read. If the corresponding configuration information is found, the process to be read reads the corresponding configuration information from the local disk; if the corresponding configuration information is not found, the process to be read establishes short connection with the zookeeper server, and acquires the corresponding configuration information from the zookeeper server and stores the configuration information in a local disk.
And acquiring configuration information from the zookeeper server according to the update configuration information notification and updating the configuration information into the local disk, so that the latest configuration information is stored in the local disk, any process reads the corresponding configuration information from the local disk and is the latest configuration information, and the real-time update of the configuration information is ensured, so that each process in the server is correctly executed.
When a process in the server needs to run, the configuration information of the process must be read first, and only the configuration information corresponding to the process needs to be searched from the local disk.
When the corresponding configuration information is not found, it is indicated that the configuration information of the process of the current configuration information to be read is not monitored, so that a short connection between the process of the current configuration information to be read and the zookeeper server is established, the configuration information corresponding to the process of the current configuration information to be read is obtained from the zookeeper server, and the configuration information is stored in a local disk.
The short connection is two connection modes opposite to the long connection, and the operation steps of the short connection are as follows: establishing a connection, data transmission and closing the connection. The short connection is made by disconnecting the connection immediately after the connection is established to transmit data. The operation steps of the long connection are as follows: connection is established-data transmission- (always kept connection) -data transmission-connection is closed. The long connection mode is not disconnected after the connection is established to transmit data, and the communication connection relation exists all the time. Compared with the short connection mode, the server system resource can be saved.
The process establishing long connection with the zookeeper server scans the local disk, and when configuration information of a new process is found in the local disk (namely, when configuration information which is not monitored yet exists in the local disk), the configuration information is added into monitoring content, so that the configuration information of all processes in the local disk can be monitored in time, and the configuration information of all processes can be updated in real time.
Compared with the prior art, the invention has the beneficial effects that:
by starting a process to occupy the lock among the processes, other processes in the time period of modifying the shared data by the process cannot perform any operation on the shared data, and other processes in the operation of the process on the shared data can immediately know that the long connection between the process and the zookeeper server is established, and the process scans the local disk to monitor all the configuration information in the local disk.
Acquiring corresponding configuration information from the zookeeper server according to the received notification, and updating the configuration information into a local disk, wherein when any process runs and needs to acquire the configuration information, only the local disk needs to be acquired to acquire the corresponding configuration information, and long connection with the zookeeper server does not need to be established; if the local disk does not have corresponding configuration information, it is indicated that the relevant configuration information of the process is not monitored, in this case, the process only needs to establish a short connection with the zookeeper server, obtain the configuration information of the process from the zookeeper server and store the configuration information in the local disk, so that the configuration information is also monitored.
It can be seen from this that: according to the method provided by the invention, each server does not need to establish long connection between each process and the zookeeper server, but only establishes long connection between one process and the zookeeper server by using the lock between the processes and the sharing of a local disk, and can ensure that the configuration information of all the processes is monitored by using the initial short connection mode, and simultaneously reduce the load of the zookeeper server.
In the aforementioned zookeeper-based information configuration method, in embodiment 1, the configuration information is stored in the local disk, and the processing performance of the process of the whole server is low due to the low processing rate of the process for viewing the local disk and obtaining the configuration information corresponding to the process from the local disk, so that the present invention also provides an optional scheme.
Referring to fig. 2, a flowchart of an embodiment 2 of a zookeeper-based information configuration method according to an embodiment of the present invention is shown, where the method includes:
When the configuration information changes, the zookeeper server sends a configuration information updating notification to the process;
The configuration information updated to the local disk and the shared memory is used for searching for the process of the configuration information to be read so as to read the configuration information corresponding to the process of the configuration information to be read. The process of the configuration information to be read firstly searches in the shared memory, and if the corresponding configuration information is searched, the process of the configuration information to be read reads the corresponding configuration information from the shared memory; if the corresponding configuration information is not found, the process of the configuration information to be read searches the local disk, if the corresponding configuration information is found, the process of the configuration information to be read reads the corresponding configuration information from the local disk, if the corresponding configuration information is not found, the process of the configuration information to be read establishes short connection with the zookeeper server, and the corresponding configuration information is obtained from the zookeeper server and stored in the shared memory and the local disk.
When any process in the server needs to run, it needs to acquire its own configuration information first, and first checks whether the corresponding configuration information is stored in the shared memory, and if not, checks whether the corresponding configuration information is stored in the local disk, and in step 202, it can be known that the configuration information of any process is stored in the local disk while being stored in the shared memory, and therefore, the configuration information stored in the local disk and the shared memory is consistent. Only one of the two needs to be checked, but it is considered that the data in the shared memory may be lost for some reasons, and the data in the local disk is generally persisted and not easily lost. Therefore, the reliability of the configuration information is ensured by storing the configuration information in the local disk, the running speed of the system is ensured by storing the configuration information in the shared memory, and the performance of the system is improved. Therefore, the information is checked in the shared memory and the local disk to avoid overlooking.
Different from embodiment 1, in this embodiment, after the configuration information is stored in the local disk, the configuration information is additionally stored in the shared memory, and the speed of reading data from the shared memory by the process is much faster than the speed of reading data from the local disk, so that the rate of reading the configuration information by the process is increased by reading the corresponding configuration information from the shared memory.
The following further explains embodiment 2 of the information configuration method based on zookeeper by taking a web server as an example, and particularly refers to fig. 3, which is a schematic diagram of an information configuration framework based on zookeeper provided by an embodiment of the present invention.
A web server usually runs 128 fast page mode (fpm) processes to provide services, according to the above steps 201 and 202, a process is started to occupy a lock between the processes, a long connection between the process and a zookeeper server is established, then the process acquires configuration information from the zookeeper server according to the updated configuration information notification and updates the configuration information to a local disk and a shared memory, when any one process in the web server needs to acquire the configuration information, the corresponding configuration information is directly searched from the shared memory, and if the configuration information is searched, the configuration information is directly acquired; if the configuration information is not found, the configuration information corresponding to the configuration information is obtained by establishing short connection with the zookeeper server and is stored in the local disk and the shared memory. Therefore, the whole web server only needs to establish a long connection with the zookeeper server, and the configuration information of all processes can be monitored and updated in time through the short connection established when the configuration information is acquired for the first time.
Please note that, although the technical solution in the above embodiment stores the configuration information in the shared memory and the local disk, the configuration information may be only searched from the shared memory during the search, and when the configuration information is not searched, a short link is directly established with the zookeeper server to obtain the configuration information corresponding to the configuration information, instead of continuously searching in the local disk. It should be understood by those skilled in the art that, under the basic principle of the present invention, the storage manner (location) and the query manner (location) of the configuration information may be changed according to the specific requirement, and the present invention is not limited to the fixed manner in the above embodiments.
Referring to fig. 4, which is a schematic diagram of an information configuration framework based on zookeeper in the prior art, each process in a web server must establish a long connection with a zookeeper server, and compared with the zookeeper-based information configuration method provided in the embodiment of the present invention, the long connection with the zookeeper can be greatly reduced, so that the load of the zookeeper server is reduced while it is ensured that all process configuration information in the server is monitored.
Compared with the prior art, the invention has the beneficial effects that:
by starting a process to occupy the lock between the processes and establishing long connection between the process and the zookeeper server, the configuration information used by the process running can be ensured to be updated and stored in time, and the load of the zookeeper server is reduced. In addition, the configuration information is stored in the shared memory, so that the process to be read acquires the configuration information corresponding to the process through the shared memory, the speed of acquiring the configuration information is improved, and the processing performance of the server is improved.
Corresponding to embodiment 1 of the information configuration method based on zookeeper, the embodiment of the invention also provides an information configuration device based on zookeeper.
Referring to fig. 5, a schematic diagram of an embodiment 1 of a zookeeper-based information configuration apparatus according to an embodiment of the present invention is shown, where the apparatus includes: the monitoring unit 301 and the updating unit 302 are further described below in connection with the operation principle of the apparatus.
A monitoring unit 301, configured to start a lock that a process occupies an inter-process, establish a long connection between the process and a zookeeper server, and monitor all configuration information in a local disk through the process;
an updating unit 302, configured to acquire, by the process according to the update configuration information notification, corresponding configuration information from the zookeeper server and update the configuration information to a local disk.
Wherein the updated configuration information notification may be sent by the zookeeper server to the process when the configuration information changes. The configuration information updated to the local disk is used for searching for the process of the configuration information to be read so as to read the configuration information corresponding to the process of the configuration information to be read. If the corresponding configuration information is found, the process to be read reads the corresponding configuration information from the local disk; if the corresponding configuration information is not found, the process to be read establishes short connection with the zookeeper server, and acquires the corresponding configuration information from the zookeeper server and stores the configuration information in a local disk.
As an optional aspect of the present invention, the monitoring unit includes:
the first possessor unit is used for starting a lock which is specially used for monitoring the process configuration information and used for occupying the process space;
and the first monitoring subunit is used for establishing a long connection between the process special for monitoring the process configuration information and the zookeeper server, and monitoring all the configuration information in the local disk through the process special for monitoring the configuration information.
As an optional aspect of the present invention, the monitoring unit includes:
the second possessor unit is used for starting the lock of the progress occupation progress of the configuration information to be read;
and the second monitoring subunit is used for starting a thread or a process in the process of the configuration information to be read, establishing long connection between the thread or the process and the zookeeper server, and monitoring all the configuration information in the local disk through the thread or the process.
As an optional solution of the present invention, the second monitoring subunit is specifically configured to start an asynchronous thread in the process of the configuration information to be read, establish a long connection between the asynchronous thread and the zookeeper server, and monitor the configuration information of all processes in the inter-process communication through the asynchronous thread.
As can be seen from the above embodiments, the beneficial effects of the present invention are:
by starting a process to occupy the lock among the processes, other processes in the time period of modifying the shared data by the process cannot perform any operation on the shared data, and other processes in the operation of the process on the shared data can immediately know that the long connection between the process and the zookeeper server is established, and the process scans the local disk to monitor all the configuration information in the local disk.
Acquiring corresponding configuration information from the zookeeper server according to the received notification, and updating the configuration information into a local disk, wherein when any process runs and needs to acquire the configuration information, only the local disk needs to be acquired to acquire the corresponding configuration information, and long connection with the zookeeper server does not need to be established; if the local disk does not have corresponding configuration information, it is indicated that the relevant configuration information of the process is not monitored, in this case, the process only needs to establish a short connection with the zookeeper server, obtain the configuration information of the process from the zookeeper server and store the configuration information in the local disk, so that the configuration information is also monitored.
It can be seen from this that: according to the device provided by the invention, each server does not need to establish a long connection between each process and the zookeeper server, but only establishes a long connection between one process and the zookeeper server by using the lock between the processes and the sharing of a local disk, and reduces the load of the zookeeper server while ensuring that the configuration information of all the processes is monitored by using an initial short connection mode.
Corresponding to the zookeeper-based information configuration method embodiment 2, the embodiment of the invention also provides a zookeeper-based information configuration device.
Referring to fig. 6, a schematic diagram of an embodiment 2 of a zookeeper-based information configuration apparatus according to an embodiment of the present invention is shown, where the apparatus includes: a monitoring unit 401, an updating unit 402 and a storage unit 403, the internal structure and connection relationship of which will be further described below in conjunction with the operation principle of the device.
And the monitoring unit 401 is configured to start a lock that one process occupies an inter-process, establish a long connection between the process and the zookeeper server, and monitor all configuration information in the local disk through the process.
And when the configuration information changes, the zookeeper server sends an update configuration information notification to the process.
An updating unit 402, configured to acquire, by the process according to the update configuration information notification, corresponding configuration information from the zookeeper server and update the configuration information to a local disk.
The storage unit 403 is configured to update or store the configuration information in the shared memory while updating or storing the configuration information in the local disk.
The configuration information updated to the local disk and the shared memory is used for searching for the process of the configuration information to be read so as to read the configuration information corresponding to the process of the configuration information to be read. The process of the configuration information to be read firstly searches in the shared memory, and if the corresponding configuration information is searched, the process of the configuration information to be read reads the corresponding configuration information from the shared memory; if the corresponding configuration information is not found, the process of the configuration information to be read searches the local disk, if the corresponding configuration information is found, the process of the configuration information to be read reads the corresponding configuration information from the local disk, if the corresponding configuration information is not found, the process of the configuration information to be read establishes short connection with the zookeeper server, and the corresponding configuration information is obtained from the zookeeper server and stored in the shared memory and the local disk.
Different from embodiment 1 of the information configuring apparatus based on zookeeper, this embodiment adds a storage unit for updating or storing the configuration information to the local disk and also to the shared memory. The configuration information of all processes is simultaneously stored in the shared memory through the processing, so that the process to be read the configuration information reads the configuration information corresponding to the process from the shared memory.
Compared with the prior art, the invention has the beneficial effects that:
by starting a process to occupy the lock between the processes and establishing long connection between the process and the zookeeper server, the configuration information used by the process running can be ensured to be updated and stored in time, and the load of the zookeeper server is reduced. In addition, the configuration information is stored in the shared memory, so that the process acquires the configuration information through the shared memory, the speed of acquiring the configuration information is improved, and the processing performance of the server is improved.
Aspects of the invention may be described in the general context of computer-executable instructions, such as program elements, being executed by a computer. Generally, program elements include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The inventive arrangements may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program elements may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus embodiment, since it is substantially similar to the method embodiment, it is relatively simple to describe, and reference may be made to some descriptions of the method embodiment for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The above detailed description of the embodiments of the present invention, and the detailed description of the embodiments of the present invention used herein, is merely intended to facilitate the understanding of the methods and apparatuses of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (20)
1. A zookeeper-based information configuration method comprises the following steps:
starting a process to occupy a lock between processes, establishing long connection between the process and a zookeeper server, and monitoring all configuration information in a local disk through the process;
and the process acquires corresponding configuration information from the zookeeper server according to the received configuration information updating notification and updates the configuration information to a local disk.
2. The method of claim 1, wherein the update configuration information notification is sent by the zookeeper server to the process upon a change in configuration information.
3. The method according to claim 1, wherein the configuration information updated to the local disk is used for a process to read configuration information to search for, so as to read configuration information corresponding to the process to read configuration information.
4. The method according to claim 3, wherein if the corresponding configuration information is found, the process to be read reads the configuration information corresponding to the process to be read from the local disk, and if the corresponding configuration information is not found, the process to be read establishes a short connection with the zookeeper server, and acquires the configuration information corresponding to the process to be read from the zookeeper server and stores the configuration information in the local disk.
5. The method of claim 1, wherein after updating the corresponding configuration information obtained from the zookeeper server to a local disk, further comprising:
and updating the corresponding configuration information acquired from the zookeeper server into a shared memory.
6. The method according to claim 5, wherein the configuration information updated to the local disk and the shared memory is used for a process of the configuration information to be read to search for, so as to read the configuration information corresponding to the process of the configuration information to be read.
7. The method according to claim 6, wherein the process of the configuration information to be read firstly searches in the shared memory, and if the corresponding configuration information is found, the process of the configuration information to be read reads the corresponding configuration information from the shared memory; if the corresponding configuration information is not found, the process of the configuration information to be read searches the local disk, if the corresponding configuration information is found, the process of the configuration information to be read reads the corresponding configuration information from the local disk, if the corresponding configuration information is not found, the process of the configuration information to be read establishes short connection with the zookeeper server, and the corresponding configuration information is obtained from the zookeeper server and stored in the shared memory and the local disk.
8. The method according to any one of claims 1 to 7, wherein the initiating a process to take a lock between processes and establish a long connection between the process and the zookeeper server, and monitoring all configuration information in the local disk by the process comprises:
starting a process special for monitoring process configuration information to occupy the lock between processes;
and establishing a long connection between the process special for monitoring the process configuration information and the zookeeper server, and monitoring all the configuration information in the local disk through the process special for monitoring the configuration information.
9. The method according to any one of claims 1 to 7, wherein the initiating a process to take a lock between processes and establish a long connection between the process and the zookeeper server, and monitoring all configuration information in the local disk by the process comprises:
starting a lock between processes occupied by the process to be read with the configuration information;
starting a thread or a process in the process of the configuration information to be read, establishing long connection between the thread or the process and the zookeeper server, and monitoring all the configuration information in the local disk through the thread or the process.
10. The method of claim 9, wherein one thread started in the process to read configuration information is an asynchronous thread.
11. A zookeeper-based information configuring apparatus, comprising:
the system comprises a monitoring unit, a data processing unit and a data processing unit, wherein the monitoring unit is used for starting a process to occupy a lock between processes, establishing long connection between the process and a zookeeper server, and monitoring all configuration information in a local disk through the process;
and the updating unit is used for acquiring corresponding configuration information from the zookeeper server through the process according to the update configuration information notification and updating the configuration information into a local disk.
12. The apparatus of claim 11, wherein the update configuration information notification is sent by the zookeeper server to the process upon a change in configuration information.
13. The apparatus according to claim 11, wherein the configuration information updated to the local disk is used for a process that is to read configuration information to search for the configuration information to be read, so as to read configuration information corresponding to the process that is to read configuration information itself.
14. The apparatus according to claim 13, wherein if the corresponding configuration information is found, the process to be configured to read the configuration information reads the configuration information corresponding to the process to be configured from the local disk, and if the corresponding configuration information is not found, the process to be configured to read the configuration information establishes a short connection with the zookeeper server, and acquires the configuration information corresponding to the process to be configured from the zookeeper server and stores the configuration information in the local disk.
15. The apparatus of claim 11, wherein the apparatus further comprises:
and the storage unit is used for updating the corresponding configuration information acquired from the zookeeper server into a shared memory.
16. The apparatus according to claim 15, wherein the configuration information updated to the local disk and the shared memory is used for a process that is to read configuration information to search for the configuration information to be read, so as to read configuration information corresponding to the process that is to read configuration information itself.
17. The apparatus according to claim 16, wherein the process of the configuration information to be read first searches in the shared memory, and if the corresponding configuration information is found, the process of the configuration information to be read reads the corresponding configuration information from the shared memory; if the corresponding configuration information is not found, the process of the configuration information to be read searches the local disk, if the corresponding configuration information is found, the process of the configuration information to be read reads the corresponding configuration information from the local disk, if the corresponding configuration information is not found, the process of the configuration information to be read establishes short connection with the zookeeper server, and the corresponding configuration information is obtained from the zookeeper server and stored in the shared memory and the local disk.
18. The apparatus according to any one of claims 11-17, wherein the monitoring unit comprises:
the first possessor unit is used for starting a lock which is specially used for monitoring the process configuration information and used for occupying the process space;
and the first monitoring subunit is used for establishing a long connection between the process special for monitoring the process configuration information and the zookeeper server, and monitoring all the configuration information in the local disk through the process special for monitoring the configuration information.
19. The apparatus according to any one of claims 11-17, wherein the monitoring unit comprises:
the second possessor unit is used for starting the lock of the progress occupation progress of the configuration information to be read;
and the second monitoring subunit is used for starting a thread or a process in the process of the configuration information to be read, establishing long connection between the thread or the process and the zookeeper server, and monitoring all the configuration information in the local disk through the thread or the process.
20. The apparatus according to claim 19, wherein the second monitoring subunit is configured to start an asynchronous thread in the process to be configured for reading the configuration information, establish a long connection between the asynchronous thread and the zookeeper server, and monitor configuration information of all processes in the inter-process communication through the asynchronous thread.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610755159.3A CN106371932B (en) | 2013-10-11 | 2013-10-11 | Zookeeper-based information configuration method and device |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610755159.3A CN106371932B (en) | 2013-10-11 | 2013-10-11 | Zookeeper-based information configuration method and device |
| CN201310472913.9A CN103500111B (en) | 2013-10-11 | 2013-10-11 | A kind of information configuring methods based on zookeeper and device |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310472913.9A Division CN103500111B (en) | 2013-10-11 | 2013-10-11 | A kind of information configuring methods based on zookeeper and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106371932A CN106371932A (en) | 2017-02-01 |
| CN106371932B true CN106371932B (en) | 2020-02-28 |
Family
ID=49865326
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310472913.9A Expired - Fee Related CN103500111B (en) | 2013-10-11 | 2013-10-11 | A kind of information configuring methods based on zookeeper and device |
| CN201610755159.3A Active CN106371932B (en) | 2013-10-11 | 2013-10-11 | Zookeeper-based information configuration method and device |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310472913.9A Expired - Fee Related CN103500111B (en) | 2013-10-11 | 2013-10-11 | A kind of information configuring methods based on zookeeper and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (2) | CN103500111B (en) |
| WO (1) | WO2015051690A1 (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103500111B (en) * | 2013-10-11 | 2016-09-28 | 北京奇虎科技有限公司 | A kind of information configuring methods based on zookeeper and device |
| CN105320527A (en) * | 2014-06-12 | 2016-02-10 | 中兴通讯股份有限公司 | Configuration file renewing method, device and system based on zookeeper distributed type search engine |
| CN105320515A (en) * | 2014-07-31 | 2016-02-10 | 腾讯科技（深圳）有限公司 | Configuration method, device and system in flow computing system |
| CN104468547B (en) * | 2014-11-28 | 2019-03-15 | 北京奇安信科技有限公司 | Establish the method, apparatus and system of long connection |
| CN104636135B (en) * | 2014-12-31 | 2018-05-04 | 北京奇虎科技有限公司 | A kind of node visit method and system, Client Agent and client |
| CN105897817A (en) * | 2015-09-17 | 2016-08-24 | 乐视云计算有限公司 | Method for communication between servers, server and communication system |
| CN106909464A (en) * | 2015-12-22 | 2017-06-30 | 北京奇虎科技有限公司 | A kind of information synchronization method and device |
| CN106909456A (en) * | 2015-12-22 | 2017-06-30 | 北京奇虎科技有限公司 | A kind of information acquisition method and device |
| CN105677404B (en) * | 2015-12-31 | 2019-03-19 | 拉扎斯网络科技（上海）有限公司 | Zookeeper-based configuration updating method and device |
| CN106453501B (en) * | 2016-09-09 | 2019-05-31 | 北京奇虎科技有限公司 | A kind of method and apparatus of the configuration information of modification service |
| CN107819798B (en) * | 2016-09-13 | 2021-08-06 | 阿里巴巴集团控股有限公司 | Data acquisition method, foreground server and data acquisition system |
| CN106648648B (en) * | 2016-12-14 | 2020-09-15 | 深圳中顺易金融服务有限公司 | Zookeeper-based configuration management method and system |
| CN108874531B (en) * | 2017-05-12 | 2021-08-03 | 北京京东尚科信息技术有限公司 | Method, device and system for fusing service and electronic equipment |
| CN107220897B (en) * | 2017-05-24 | 2023-04-07 | 腾讯科技（深圳）有限公司 | Income posting processing method and device and processing system |
| CN109525406A (en) * | 2017-09-18 | 2019-03-26 | 飞狐信息技术（天津）有限公司 | A kind of operation system configuration method and device |
| CN109976822B (en) * | 2017-12-25 | 2022-04-08 | 天翼云科技有限公司 | Configuration method and device of distributed application and distributed system |
| CN110134453B (en) * | 2018-02-09 | 2022-07-08 | 网宿科技股份有限公司 | Server configuration overloading method and server |
| CN109714188B (en) * | 2018-11-02 | 2023-06-20 | 中国平安人寿保险股份有限公司 | Configuration data management method, device and storage medium based on Zookeeper |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103034540A (en) * | 2012-11-16 | 2013-04-10 | 北京奇虎科技有限公司 | Distributed information system, device and coordinating method thereof |
| CN103034541A (en) * | 2012-11-16 | 2013-04-10 | 北京奇虎科技有限公司 | Distributing type information system and equipment and method thereof |
| CN103092698A (en) * | 2012-12-24 | 2013-05-08 | 中国科学院深圳先进技术研究院 | System and method of cloud computing application automatic deployment |
| CN103294701A (en) * | 2012-02-24 | 2013-09-11 | 联想(北京)有限公司 | Distributed file system and data processing method |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8700683B2 (en) * | 2011-10-24 | 2014-04-15 | Nokia Corporation | Method and apparatus for providing a key-value based storage interface |
| US9063939B2 (en) * | 2011-11-03 | 2015-06-23 | Zettaset, Inc. | Distributed storage medium management for heterogeneous storage media in high availability clusters |
| CN102591934A (en) * | 2011-12-23 | 2012-07-18 | 国网电力科学研究院 | Zookeeper-based method for realizing automatic expansion and switching of multiple Solr Shards |
| CN103297456B (en) * | 2012-02-24 | 2016-09-28 | 阿里巴巴集团控股有限公司 | Access method and the distributed system of resource is shared under a kind of distributed system |
| CN102710554B (en) * | 2012-06-25 | 2015-09-02 | 深圳中兴网信科技有限公司 | The service state detection method of distributed information system and distributed information system |
| CN102982141B (en) * | 2012-11-20 | 2016-01-20 | 北京搜狐新媒体信息技术有限公司 | A kind of method and device realizing distributed data base agency |
| CN103500111B (en) * | 2013-10-11 | 2016-09-28 | 北京奇虎科技有限公司 | A kind of information configuring methods based on zookeeper and device |
-
2013
- 2013-10-11 CN CN201310472913.9A patent/CN103500111B/en not_active Expired - Fee Related
- 2013-10-11 CN CN201610755159.3A patent/CN106371932B/en active Active
-
2014
- 2014-09-16 WO PCT/CN2014/086653 patent/WO2015051690A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103294701A (en) * | 2012-02-24 | 2013-09-11 | 联想(北京)有限公司 | Distributed file system and data processing method |
| CN103034540A (en) * | 2012-11-16 | 2013-04-10 | 北京奇虎科技有限公司 | Distributed information system, device and coordinating method thereof |
| CN103034541A (en) * | 2012-11-16 | 2013-04-10 | 北京奇虎科技有限公司 | Distributing type information system and equipment and method thereof |
| CN103092698A (en) * | 2012-12-24 | 2013-05-08 | 中国科学院深圳先进技术研究院 | System and method of cloud computing application automatic deployment |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2015051690A1 (en) | 2015-04-16 |
| CN106371932A (en) | 2017-02-01 |
| CN103500111B (en) | 2016-09-28 |
| CN103500111A (en) | 2014-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106371932B (en) | Zookeeper-based information configuration method and device | |
| CN111090699A (en) | Service data synchronization method and device, storage medium and electronic device | |
| KR101963917B1 (en) | Automatic synchronization of most recently used document lists | |
| CN108712457B (en) | Method and device for adjusting dynamic load of back-end server based on Nginx reverse proxy | |
| CN113254466B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN107391276B (en) | Distributed monitor method, interception control device and system | |
| CN101895540B (en) | For the system and method that application service process is guarded | |
| CN106161145A (en) | A kind of monitoring method and system of server system operation status information | |
| US8448188B2 (en) | Web user interface component coordination | |
| CN109783151B (en) | Method and device for rule change | |
| CN112118315A (en) | Data processing system, method, device, electronic equipment and storage medium | |
| CN113452617B (en) | Dynamic gateway route management method, device and storage medium | |
| CN111800462A (en) | Micro-service instance processing method and device, computer equipment and storage medium | |
| CN106940671B (en) | Method, device and system for monitoring running of task threads in cluster | |
| CN110740145A (en) | Message consumption method, device, storage medium and electronic equipment | |
| CN109257396B (en) | Distributed lock scheduling method and device | |
| CN105373563B (en) | Database switching method and device | |
| US20150205675A1 (en) | Method and System for Improving Reliability of a Background Page | |
| CN112527519A (en) | High-performance local cache method, system, equipment and medium | |
| CN115033656B (en) | Data processing method and device based on map caching technology and electronic device | |
| US20240146601A1 (en) | Intelligent real-time internet-of-things alert | |
| CN116700779A (en) | Code dynamic adjustment method, device, equipment, medium and program product | |
| CN112187916B (en) | Cross-system data synchronization method and device | |
| CN114201508A (en) | Data processing method, data processing apparatus, electronic device, and storage medium | |
| CN111858260A (en) | Information display method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240115 Address after: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee after: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Patentee before: Qizhi software (Beijing) Co.,Ltd. |