CN105357296A - Elastic caching system based on Docker cloud platform - Google Patents

Abstract

The invention discloses an elastic caching system based on Docker cloud platform. The elastic caching system consists of a Webserver module, a Docker cluster management module, a Docker host machine cluster, a MySQL database, an Etcd cluster, a Docker mirror registry and a client. The client transmits user authentication information to the WebServer module; the WebServer module transmits Docker container control associated data and a host machine state obtaining command to the Docker cluster management module; the WebSever module reads and writes a host machine state and the user information to the MySQL database; the WebServer module transmits container starting state information to the Etcd cluster; the client reads cached data and Proxy node list information; the Docker host machine cluster transmits host machine state data and Docker management data to the Docker cluster management module; Docker Image data is transmitted from the Docker mirror registry to the Docker cluster management module; the Docker host machine cluster transmits caching cluster start associated data, Docker container state data, elastic event data and disaster tolerance event data to the Etcd cluster.

Description

Elastic caching system under a kind of Docker cloud platform

Technical field

The invention belongs to Computer Applied Technology field, the present invention is applied on cloud platform more precisely, uses Docker technology and cache policy to be integrated into by the computational resources such as CPU, internal memory, bandwidth and has flexible buffer service, be supplied to user.

Background technology

Cloud is made up of the computational resource of a large amount of integration management and storage resources, generally include provide computing capability, virtual network is provided and server cluster that massive store is provided.Owing to having probabilistic peak flow from the access of user, when flow is lower, server average utilization is low.Guaranteeing data security and serving on reliable basis, improving resource utilization, reducing energy consumption, is the problem that cloud computing platform needs solution badly.Therefore, the elasticity distribution of virtual resource has realistic meaning for raising resource utilization with cutting operating costs.

Distributed caching can make data transmission event between object and application greatly reduce, and is a kind of important means that cloud platform promotes application performance.Especially data-intensive Web is applied, adopt distributed caching technology can promote the performance of cloud platform.The distributed cache system that current comparative maturity, customer volume are larger has the ElastiCache of Amazon and the OCS of Ali's cloud, and these two systems and most of caching system could realize the elastic telescopic of cluster after all needing user manually to change configuration.If system access amount is uprushed, fixing cache cluster scale can not bear large discharge access, and system access speed can be caused to reduce.Therefore, automatic elastic is flexible in real time according to demand for system, brings great convenience to user.But existing caching system mostly adopts traditional virtual technology, virtual machine activation speed is very slow, does not reach requirement of real-time far away, so the present invention adopts Docker technology, realizes the elastic caching under cloud platform.

Summary of the invention

Goal of the invention: for problems of the prior art, the present invention realizes the Docker cluster management platform with cluster state monitoring (CPU usage, internal storage state, network state etc.) function, and platform has the startup of container and destruction function simultaneously; Each user can distribute a cache cluster alone, and cache cluster configures according to user and automatically creates and use of reaching the standard grade; User can each cache cluster run time behaviour of Real-Time Monitoring, and comprise cache size, number of request per second, each container state, the scale of cache cluster can according to the actual service condition dynamic change of user; Realize disaster detection and Restoration Mechanism.When certain node in a cluster occurs extremely and cannot serve, system can make corresponding process according to node type difference, makes cluster recovery normal service, ensure that the high availability of system; Realize being convenient to the data cached client of user operation, for raising the efficiency, client adopts connection pool, and shields user and realize details.

Technical scheme: elastic caching system under a kind of Docker cloud platform, comprises WebServer module, Docker cluster management module, Docker host cluster, MySQL database, Etcd cluster, Docker mirror image warehouse and client.

WebServer module is the system core, is responsible for the startup of Docker container and deletion, process Redis cluster elastic telescopic request and the request of disaster tolerance aspect, and processes Client and hold various request, and provides visualization interface for keeper and user.WebServer uses JSON as data interchange format.

Use open source projects Swarm and etcd as the basis of Docker cluster management.Docker cluster management module is by Etcd cluster, and management Docker host cluster, uses the Dcoker consultative management Docker host cluster of standard.Subscribe to respective directories during startup, self information is regularly published to same catalogue by Docker host.After Docker cluster management module receives the information of subscription, upgrade host list and host machine system state according to the subscription information received.Simultaneously, Docker cluster management module provides service for WebServer module, comprises the information of collecting container in Docker host, newly-built or destruction container, timed collection Docker host machine system situation (cpu load, memory information, bandwidth, I/O information).

Docker host cluster: storage engines selects Redis, builds Redis cluster based on open source projects Codis.Docker host cluster has multiple Redis cluster to form.The corresponding DockerImage of each Redis cluster.In DockerImage, an integrated container finger daemon module, and dissimilar container node.In Docker host cluster, container node type is divided into Redis pattern, Proxy pattern and Manager pattern.According to the difference of Docker container node type, start corresponding flow process, mainly comprise start-up course management, node state monitoring, elasticity realizes, node failure reports.

In Docker cloud platform, in order to realize many tenants isolation, each user has an exclusive Redis cluster.Client end can direct communication with Proxy node in cluster.In order to ensure high availability, user can select to set up RedisSlave node when configuring Redis cluster, and based on the leader follower replication function of Redis, slave node has and master node identical data.When RedisMaster node failure, in real time slave node is promoted to master node.

Client end, by ICP/IP protocol and Redis trunking communication, is communicated with WebServer by http protocol.The function of Client end is identifying user identity, the Proxy list upgrading Docker host cluster, provides accessed cache interface.Because Redis cluster Proxy node is not fixed, all connections of upgrade in time in order to ensure Client end Proxy list, employing connection pool management Client and Proxy.

MySQL database: for storing the data of WebServer module.

Ectd cluster: for the service discovery platform of whole system.

Docker mirror image warehouse: for Docker host provides DockerImage download service.

WebServer module is system management maincenter, has extended capability, adopts stateless method for designing, can be extending transversely according to demand.After WebServer module starts, in Etcd cluster/webserver catalogue issue self information.When other modules send out request to WebServer module, first random certain WebServer address of service of acquisition from this catalogue, employs the service discovery function of etcd herein.According to request object, WebServer is subdivided into user's service module, keeper's service module, Client service module, Docker trunking communication module.

User's service module: user's service module mainly with B/S framework for user provides service, service comprises: the initialization of Redis cache cluster, Redis cache cluster monitoring.User is by cluster needed for Web interface configurations, and configuration item comprises Initial R edis service group number, elasticity high threshold, elasticity Low threshold, whether ensures High Availabitity, information is submitted to after selection.After network analysis process configuration information, the node container needed for startup, within about five minutes, Redis cluster just can successfully create and cluster information is saved in MySQL database.Whether successfully after creating cluster, user monitors cluster state by Web page, comprise cluster ID value, cluster startup, cache size, process number of request per minute, hit rate, Redis node number, Proxy node number.

Keeper's service module: keeper's service module mainly with B/S framework for keeper provides service, service is mainly the monitoring of whole system.After keeper logs in, there is provided whole Docker host cluster state with form web page, information comprises: the information (as cluster ID, Redis nodal information, Proxy nodal information etc.) of host number, the company's of mistake host information, host machine system information (as IP address, CPU usage, memory usage, bandwidth), all Redis clusters.

Client service module: Client service module mainly processes the HTTP request from Client, these requests comprise user rs authentication, obtain Proxy list.For user rs authentication, after the user name that Client submits to, password, cluster id information are verified, return Proxy address list.Need to process Client in addition and can send the request obtaining Proxy address list at set intervals.

Docker trunking communication module: Docker trunking communication module realizes Redis cluster initialization module, elastic telescopic module, disaster tolerance module by Docker Administration API (dispose separately and be SwarmAPI, other are deployed as the API that corresponding platform provides).

Container finger daemon module integration is in DockerImage, and when container starts, first container finger daemon module starts.Container start-up mode is different according to node type, is divided into Redis pattern, Proxy pattern, Manager pattern.Wherein Manager node is responsible for the management of whole cluster and cluster relevant information is kept in etcd cluster.Proxy is the agency service that Client connects Redis, and itself achieves Redis agreement, and performance is served identical with a primary Redis.Redis is divided into master, slave two kinds of nodes as storage engines, and slave copies data in master, ensures high availability.These two kinds of nodes constitute a ServerGroup.The mode of Hash groove is adopted to carry out burst to data, data are distributed in 1024 slot according to CRC32 (key) %1024, slot is a virtual concept, data are actually stored in multiple ServerGroup, for ServerGroup without any distributed logic wherein, each ServerGroup is responsible for a part of key-value data.

In order to realize elastic caching function, the elastic telescopic request of Manager node in elastic telescopic module in charge process Redis cluster, request type comprises: increase Redis service group, delete Redis service group, increase Proxy node, delete Proxy node.

Elastic cloud caching system key property is exactly that cache cluster scale can be corresponding flexible along with the change of load.Along with the increase of buffer memory speed is read in application, the internal memory required for Redis cache cluster, CPU, bandwidth resources also can correspondingly increase, and at this moment, need, by increasing Redis node or Proxy node, to promote Redis cluster bearing capacity.On the contrary, when the speed minimizing of buffer memory is read in the application of user, the resource required for Redis cluster also can correspondingly reduce, and now can reduce Redis node or Proxy node, promotes the utilance of whole elastic caching system.

In order to realize the elastic telescopic function of cluster, the present invention uses the cluster elastic telescopic based on dynamic Three-exponential Smoothing prediction algorithm to realize.

In Redis cluster, the load information of all Proxy, Redis nodes of Manager node timed collection.The load information of Proxy comprises memory usage M _usage, CPU usage C _usage, bandwidth utilization rate N _usage.The information of Redis node only includes memory usage M _usage.

For Redis node, Manager can by the M of all Redis nodes _usagedata, combine the overall load forming Redis node.When after system long-play, each node load situation skewness may be caused, therefore, the dispersion of these Redis node load information of Water demand during integration, employing formula (1) calculates Redis node load information and is:

$\mathrm{\σ}=\sqrt{\frac{1}{n}\underset{i=1}{\overset{n}{\Σ}}{(100M_i-\mathrm{\μ})}^2},\mathrm{\μ}=\frac{100}{n}\underset{i=1}{\overset{n}{\Σ}}{M}_i---\left(1\right)$

Wherein i represents the sequence number of Redis node, if σ >=20, illustrates that load Distribution is uneven, now needs to perform Rebalance operation, redistributes these Redis node loads.M _irepresent the memory usage of node i, namely use internal memory divided by internal memory altogether.μ represents the overall load of Reids node, and be saved in Redis load time series Rt, t=1,2 ..., in n}.The queue of time series regular length is kept in the internal memory of Manager node.

For Proxy node, Client end adopts polling mode to communicate with Proxy, and the loading condition of each Proxy node is more or less the same.The overall load P of all Proxy nodes _allformula (2) can be utilized to calculate:

$P_{all}=\frac{100}{n}\underset{i=1}{\overset{n}{\Σ}}{P}_i,P_i=max(M_i,C_i,N_i),i=1,2,...,N---\left(2\right)$

Equally, P _allrepresent the overall load of Proxy node, and be saved to Proxy load time series Pt, t=1,2 ..., n}, M _irepresent the memory usage of node i, C _irepresent the CPU usage of node i, N _irepresent the bandwidth utilization rate of node i.

After the overall load of Redis node and Proxy node calculates, elastic telescopic engine can be analyzed corresponding two time serieses, and two kinds of time Series Processing methods are identical.

Algorithm elastic shows load data time Series Processing process.

Docker cluster management module is divided into host pattern, manager's pattern.In manager's pattern, main frame is actual is the agent node of host, and its in store all host information, outside use standard Docker agreement request manager manages host.During newly-built container, manager will start container Requests routing to certain host with load balance scheduling strategy, after deletion container operation only need find the host of this container of actual motion, send removal request.The cluster that the server of two kinds of patterns is formed a whole by Etcd cluster.

Container finger daemon module integration, in DockerImage, when container starts, first starts container and guards module.Conveniently manage, container running state information adopts Context structure to describe, and preserves wherein.

Container running state information is defined as:

When WebServer starts container, by all container Set up-conservancy in system parameters, according to system parameters initialization Context structure.After initialization, start corresponding container pattern according to Context, comprise Redis pattern, Proxy pattern, Manager pattern.

Container major function under Redis pattern is: start and register Redis service.Redis Starting mode is divided into: initialization startup and elasticity start two kinds of start-up courses, can be master, slave any type.

The major function of Proxy pattern starts above-mentioned Proxy software, collects and Accreditation System state.The Starting mode of Proxy pattern is divided into initialization startup and elasticity to start, and the occurrence scene of two kinds of Starting mode is with Redis pattern.

Manager pattern is most crucial pattern.Container major function under Manager pattern is: monitoring cluster server run time behaviour, collect cluster load information, Processing Cluster is stretched, and slot moves, management Proxy node and Redis node.Manager Starting mode is divided into: start two kinds of modes after initialization startup and fault.

Client holds module: the Client end node that user uses must be in consolidated network with caching system, and Client holds module by ICP/IP protocol and Redis trunking communication, by http protocol and WebServer module communication.In order to ensure high efficiency, adopt all connections of connection pool management Client and Proxy.Before user uses Client end, need to submit user name, password, Redis cluster ID to, carry out user rs authentication by WebServer module.After being verified, WebServer module returns the Proxy list of corresponding Redis cluster.In addition, the timing of Client end upgrades Proxy list, ensures the correctness of Proxy list.After Client end obtains Proxy list, corresponding load-balancing algorithm can be selected to send read-write requests to Proxy.User can arrange load-balancing algorithm voluntarily, and load-balancing algorithm comprises: randomized, polling method, WRR method, dynamic weighting polling method, fastest response precedence method.

Docker mirror image warehouse: Docker mirror image warehouse for preserving DockerImage, and provides image download service for Docker host cluster.The present invention adopts the privately owned Docker mirror image warehouse set up based on container.Construction method based on container first downloads Registry mirror image from DockerHub, then creates container based on this mirror image in the server, and starting this container successfully just can outwards provide registry to serve.

Accompanying drawing explanation

Fig. 1 is system construction drawing;

Fig. 2 is WebServer function structure chart;

Fig. 3 is Redis aggregated structure figure;

Fig. 4 is for increasing Redis service group flow chart;

Fig. 5 is for deleting Redis service group flow chart;

Fig. 6 is for increasing Proxy node flow chart;

Fig. 7 is for deleting Proxy node flow chart;

Fig. 8 is Manager node failure process chart;

Fig. 9 is Proxy node failure process chart;

Figure 10 is Redismaster node failure process chart;

Figure 11 is manager's work pattern flow chart in Docker cluster management;

Figure 12 is host work pattern flow chart in Docker cluster management;

Figure 13 is that container finger daemon module Redis mode initialisation starts workflow diagram;

Figure 14 is that container finger daemon module Redis mode elastic starts workflow diagram;

Figure 15 is that under container finger daemon module Proxy pattern, initialization starts workflow diagram;

Figure 16 is that under container finger daemon module Proxy pattern, elasticity starts workflow diagram;

Figure 17 is that container finger daemon module Manager mode initialisation starts workflow diagram;

Figure 18 starts workflow diagram after fault under container finger daemon module Manager pattern;

Figure 19 is that client upgrades Proxy list workflow diagram.

Embodiment

Below in conjunction with specific embodiment, illustrate the present invention further, these embodiments should be understood only be not used in for illustration of the present invention and limit the scope of the invention, after having read the present invention, the amendment of those skilled in the art to the various equivalent form of value of the present invention has all fallen within the application's claims limited range.

Fig. 1 is system construction drawing of the present invention.System is made up of WebServer module, Docker cluster management module, Docker host cluster, MySQL database, Etcd cluster, Docker mirror image warehouse, client.

(1.1) WebServer module is the system core, is responsible for the startup of Docker container and deletion, process Redis cluster elastic telescopic request and the request of disaster tolerance aspect, and processes Client and hold various request, and provides visualization interface for keeper and user.WebServer uses JSON as data interchange format.

(1.2) Docker cluster management module is by Etcd cluster, and management Docker host cluster, uses the Dcoker consultative management Docker host cluster of standard.Subscribe to respective directories during startup, self information is regularly published to same catalogue by Docker host.After Docker cluster management module receives the information of subscription, upgrade host list and host machine system state according to the subscription information received.Simultaneously, Docker cluster management module provides service for WebServer module, comprises the information of collecting container in Docker host, newly-built or destruction container, timed collection Docker host machine system situation (cpu load, memory information, bandwidth, I/O information).

(1.3) Docker host cluster: Docker host cluster has multiple Redis cluster to form.The corresponding DockerImage of each Redis cluster.In DockerImage, an integrated container finger daemon module, and dissimilar container node.In Docker host cluster, container node type is divided into Redis pattern, Proxy pattern and Manager pattern.According to the difference of Docker container node type, start corresponding flow process, mainly comprise start-up course management, node state monitoring, elasticity realizes, node failure reports.

(1.4) in Docker cloud platform, in order to realize many tenants isolation, each user has an exclusive Redis cluster.Client end can direct communication with Proxy node in cluster.In order to ensure high availability, user can select to set up RedisSlave node when configuring Redis cluster, and based on the leader follower replication function of Redis, slave node has and master node identical data.When RedisMaster node failure, in real time slave node is promoted to master node.

(1.5) Client is by ICP/IP protocol and Redis trunking communication, by http protocol and WebServer module communication.The function of Client end is identifying user identity, the Proxy list upgrading Docker host cluster, provides accessed cache interface.Because Redis cluster Proxy node is not fixed, all connections of upgrade in time in order to ensure Client end Proxy list, employing connection pool management Client and Proxy.

(1.6) MySQL database: for storing the data of WebServer module.

(1.7) Ectd cluster: for the service discovery platform of whole system.

(1.8) Docker mirror image warehouse: for Docker host provides DockerImage download service.The present invention adopts the privately owned Docker mirror image warehouse set up based on container.Construction method based on container first downloads Registry mirror image from DockerHub, then creates container based on this mirror image in the server, and starting this container successfully just can outwards provide registry to serve.

(1.9) relation between modules: Client holds by http protocol, and transmission user and authorization information are to WebServer module; Docker container is controlled related data by WebServer module, acquisition host status command is sent to Docker cluster management module; WebServer module reads with write host state, user profile to MySQL database; WebServer module sends container starting state information to Etcd cluster; Client holds by ICP/IP protocol, reading cache data, Proxy node listing information from Docker host cluster; Docker host cluster transmission host status data, Docker management data are to Docker cluster management module; DockerImage data are sent to Docker cluster management module from Docker mirror image warehouse; Docker host cluster transmission cache cluster starts related data, Docker container state data, elasticity event data, disaster tolerance event data to Etcd cluster.

Fig. 2 illustrates the composition structure of WebServer module.User's service module with B/S access mode for user provides Redis cache cluster initialization services, Redis cache cluster monitor service.Keeper's service module in B/S mode for keeper provides entire system monitor service.Client service module mainly processes the HTTP request from Client, as user rs authentication, obtains Proxy list.Docker cluster management module realizes Redis cluster initialization module, elastic telescopic module, disaster tolerance module by Docker Administration API.

(2.1) user's service module: user is by cluster needed for Web interface configurations, and configuration item comprises Initial R edis service group number, elasticity high threshold, elasticity Low threshold, whether ensures High Availabitity, is submitted to by configuration information after selection.After network analysis process configuration information, the node container needed for startup, Redis cluster creates successfully, and cluster information is saved in MySQL database.After creating cluster, whether successfully user monitors cluster state by Web page, comprise cluster ID value, cluster startup, cache size, process number of request per minute, hit rate, Redis node number, Proxy node number.

(2.2) keeper's service module: after keeper logs in, there is provided whole Docker host cluster state with form web page, information comprises: the information (as cluster ID, Redis nodal information, Proxy nodal information etc.) of host number, the company's of mistake host information, host machine system information (as IP address, CPU usage, memory usage, bandwidth), all Redis clusters.

(2.3) Client service module: Client service module mainly processes the HTTP request from Client, these requests comprise user rs authentication, obtain Proxy list.For user rs authentication, after the user name that Client submits to, password, cluster id information are verified, return Proxy address list.Need to process Client in addition and can send the request obtaining Proxy address list at set intervals.

(2.4) Docker trunking communication module: Docker trunking communication module realizes Redis cluster initialization module, elastic telescopic module, disaster tolerance module by Docker Administration API (dispose separately and be SwarmAPI, other are deployed as the API that corresponding platform provides).

Fig. 3 illustrates Redis aggregated structure.Redis cluster is divided three classes node: Manager, Proxy, Redis.Wherein Manager node is responsible for the management of whole cluster and cluster relevant information is kept in etcd cluster.Proxy is the agency service that Client connects Redis, and itself achieves Redis agreement, and performance is served identical with a primary Redis.Redis is divided into master, slave two kinds of nodes as storage engines, and slave copies data in master, ensures high availability.These two kinds of nodes constitute a ServerGroup.The mode of Hash groove is adopted to carry out burst to data, data are distributed in 1024 slot according to CRC32 (key) %1024, slot is a virtual concept, data are actually stored in multiple ServerGroup, for ServerGroup without any distributed logic wherein, each ServerGroup is responsible for a part of key-value data.

In order to realize elastic caching function, the elastic telescopic request of Manager node in elastic telescopic module in charge process Redis cluster, request type comprises: increase Redis service group, delete Redis service group, increase Proxy node, delete Proxy node.

Elastic cloud caching system key property is exactly that cache cluster scale can be corresponding flexible along with the change of load.Along with the increase of buffer memory speed is read in application, the internal memory required for Redis cache cluster, CPU, bandwidth resources also can correspondingly increase, and at this moment, need, by increasing Redis node or Proxy node, to promote Redis cluster bearing capacity.On the contrary, when the speed minimizing of buffer memory is read in the application of user, the resource required for Redis cluster also can correspondingly reduce, and now can reduce Redis node or Proxy node, promotes the utilance of whole elastic caching system.

In order to realize the elastic telescopic function of cluster, the present invention uses the cluster elastic telescopic based on dynamic Three-exponential Smoothing prediction algorithm to realize.

In Redis cluster, the load information of all Proxy, Redis nodes of Manager node timed collection.The load information of Proxy comprises memory usage M _usage, CPU usage C _usage, bandwidth utilization rate N _usage.The information of Redis node only includes memory usage M _usage.

For Redis node, Manager can by the M of all Redis nodes _usagedata, combine the overall load forming Redis node.When after system long-play, each node load situation skewness may be caused, therefore, the dispersion of these Redis node load information of Water demand during integration, employing formula (1) calculates Redis node load information and is:

$\mathrm{\σ}=\sqrt{\frac{1}{n}\underset{i=1}{\overset{n}{\Σ}}{(100M_i-\mathrm{\μ})}^2},\mathrm{\μ}=\frac{100}{n}\underset{i=1}{\overset{n}{\Σ}}{M}_i---\left(1\right)$

Wherein i represents the sequence number of Redis node, if σ >=20, illustrates that load Distribution is uneven, now needs to perform Rebalance operation, redistributes these Redis node loads.μ represents the overall load of Reids node, and be saved in Redis load time series Rt, t=1,2 ..., in n}.The queue of time series regular length is kept in the internal memory of Manager node.

For Proxy node, Client end adopts polling mode to communicate with Proxy, and the loading condition of each Proxy node is more or less the same.The overall load P of all Proxy nodes _allformula (2) can be utilized to calculate:

$P_{all}=\frac{100}{n}\underset{i=1}{\overset{n}{\Σ}}{P}_i,P_i=max(M_i,C_i,N_i),i=1,2,...,N---\left(2\right)$

Equally, P _allrepresent the overall load of Proxy node, and be saved to Proxy load time series Pt, t=1,2 ..., n}.

After the overall load of Redis node and Proxy node calculates, elastic telescopic engine can be analyzed corresponding two time serieses, and two kinds of time Series Processing methods are identical.

Algorithm elastic shows time Series Processing process, wherein time series with Ut, t=1,2 ..., n} represents, Un represents current node load situation.

Fig. 4 illustrates in WebServer module increases Redis service group, provides the implementation procedure of cache cluster elastic telescopic.Its idiographic flow is:

(4.1), after WebServer module receives and increases the request of Redis service group, corresponding cluster information is obtained according to cluster ID;

(4.2) according to cluster information, new service group ID is calculated.Adopt NewID=MAX{ID}+1.

(4.3) according to cluster information, determine whether to start slave node.

(4.4) if start slave node, then start slave and master node simultaneously; Otherwise only start master node.

(4.5) to subscribe in etcd cluster/elastic/servergroup catalogue, according to receiving Data Update under this catalogue, whether the service group of judgement starts successfully.

(4.6) if be not activated successfully, then jump to (4.3), otherwise upgrade the cluster information in MySQL database.

(4.7) newly-built service group information is returned to the container sending request.

Fig. 5 illustrates the implementation procedure of deleting Redis service group in WebServer module.Its idiographic flow is:

(5.1), after WebServer module receives and deletes the request of Redis service group, corresponding cluster information is obtained according to cluster ID;

(5.2) according to cluster information, slave node is determined whether there is.

(5.3) if determine to there is slave node, then delete master and slave node simultaneously; Otherwise only delete master node.

(5.4) according to Redis group feedack, confirm whether master/slave node is deleted successfully.If delete unsuccessfully, then respective nodes ID is put into container queue to be deleted; Otherwise, perform (5.5).

(5.5) cluster information in MySQL database is upgraded.

Fig. 6 illustrates the implementation procedure increasing Proxy node in WebServer module.Its idiographic flow is:

(6.1), after WebServer module receives and increases the request of Proxy node, corresponding cluster information is obtained according to cluster ID;

(6.2) according to cluster information, calculate and obtain Proxy node ID.

(6.3) according to Proxy_ID, corresponding Proxy node is started.

(6.4)/elastic/proxy catalogue is subscribed in etcd cluster.

(6.5) according to the information of Proxy node feeding back, judge whether newly-increased Proxy node starts successfully.If success, then the cluster information in MySQL database will be upgraded.If start unsuccessfully, destroy corresponding Proxy node, and jump to (6.3), re-execute Proxy node and start.

(6.6) newly-built Proxy nodal information is returned to the container sending request.

Fig. 7 illustrates the implementation procedure of deleting Proxy node in WebServer module, substantially similar with deletion Redis service group flow process.Its idiographic flow is:

(7.1), after WebServer module receives the request of deleting Proxy node, corresponding cluster information is obtained according to cluster ID;

(7.2) according to cluster information, Proxy node ID is obtained.

(7.3) according to Proxy node ID, corresponding Proxy node is deleted.

(7.4) according to the information of WebServer module feedback, confirm whether Proxy node is deleted successfully.If delete unsuccessfully, then respective nodes ID is put into container queue to be deleted; Otherwise, perform (7.5).

(7.5) cluster information in MySQL database is upgraded.

For Redis cache cluster, Manager node, Proxy node, Redis node all likely lost efficacy.The high availability of Manager node and Proxy node can be used for applying disaster tolerance, and the high availability of edis node can ensure data disaster tolerance.Therefore, when in Redis cluster during any one node failure, all need quick response, guarantee the continuity of cloud service and the consistency of data.The function of disaster tolerance module mainly processes the request of Proxy node in Redis cluster, Manager node.Fault type comprises Manager node failure, Proxy node failure, Redismaster node failure.

Fig. 8 illustrates the workflow of Manager node failure process.Its idiographic flow is as follows:

(8.1) request is from Proxy node.After receiving the request of Proxy node, obtain corresponding cluster information according to cluster ID;

(8.2) by initiate request ProxyID be published to/failure/manager/proxy catalogue under;

(8.3)/elastic/servergroup catalogue is subscribed in etcd cluster;

(8.4) monitored directory lastest imformation;

(8.5) judge whether that whole ProxyID is registered.If be all registered, then enter and seize distributed lock; Otherwise, jump to (8.4)

(8.6) judge whether to obtain distributed lock.If obtain lock, then enter (8.7); Otherwise, terminate.

(8.7) start manager node, upgrade the cluster information in MySQL database.

Fig. 9 illustrates the workflow of Proxy node failure process.Its idiographic flow is as follows:

(9.1) request is from manager node, after receiving the request of manager node, obtains corresponding cluster information according to asking the cluster ID provided;

(9.2) according to the cluster ID asking to provide and Proxy node ID, this Proxy node is deleted.

(9.3) whether successfully judge to delete Proxy node.If delete unsuccessfully, then this Proxy node ID is put into container queue to be deleted; Otherwise, terminate.

(9.4) enter in elastic telescopic module and increase Proxy nodal operation flow process.

Figure 10 illustrates the workflow of Redismaster node failure process.Its idiographic flow is as follows:

(10.1) request is from manager node, after receiving the request of manager node, obtains corresponding cluster information according to asking the cluster ID provided;

(10.2) according to the cluster ID asking to provide and Redis service group ID, this Redis container is deleted.

(10.3) whether successfully judge to delete Proxy node.If delete unsuccessfully, then this Redismaster node ID is put into container queue to be deleted; Otherwise, terminate.

(10.4) if Redis service group has slave node, then the slave node under this Redis service group being promoted to master node, meanwhile, is the newly-built slave node of this Redis service group.

(10.5) cluster information in MySQL database is upgraded.

Docker cluster management module is divided into host pattern, manager's pattern.In manager's pattern, main frame is actual is the agent node of host, and its in store all host information, outside use standard Docker agreement request manager manages host.During newly-built container, manager will start container Requests routing to certain host with certain scheduling strategy, after deletion container operation only need find the host of this container of actual motion, send removal request.The cluster that the server of two kinds of patterns is formed a whole by Etcd cluster.

Figure 11 is the workflow of manager's pattern in Docker cluster management module, and its specific works flow process is:

(11.1) start HTTP service, the API of access host state is outwards provided.API comprises acquisition cluster state, obtains certain host machine system state, obtains certain Image information, obtains certain information of container, sets up Image, starts container, destroys container, in a reservoir fill order.

(11.2) subscribe to host catalogue in host, wait to be updated.

(11.3) find upgrade and process the data obtained.Data are Key-Value structure, and Key is host IP, Value is host state.

(11.4) if there is this IP in host list, then the system mode of this IP host is upgraded; Otherwise, this IP and state thereof are increased in list.Jump to (11.2) to repeat.

Figure 12 is the workflow of host pattern in Docker cluster management module, and its specific works flow process is:

(12.1) Docker service is started.

(12.2) start successfully, obtain system mode (CPU usage, bandwidth, internal memory service condition) by the stats interface of Docker.

(12.3) in etcd cluster, issue self information, the key value of information is host IP, value is host machine system state.Timer, through 10 seconds, turns back to (12.2) circulation and performs.

Container finger daemon module integration is in DockerImage.When WebServer starts container, start container finger daemon module, by all container Set up-conservancy in system parameters, according to system parameters initialization Context structure.After initialization, start corresponding container pattern according to Context, comprise Redis pattern, Proxy pattern, Manager pattern.

Container major function under Redis pattern is: start and register Redis service.Redis Starting mode is divided into: initialization startup and elasticity start two kinds of start-up courses, can be master, slave any type.

Figure 13 shows the workflow that Redis mode initialisation starts, and its idiographic flow is as follows:

(13.1) when the initialization of Redis cache cluster, Redis pattern container is created;

(13.2) start Redis program, self information is published to etcd cluster/$ cid/init/redis catalogue under;

(13.3) start successfully to WebServer report, wait for being found by Manager node.

Figure 14 shows the workflow that Redis mode elastic starts, and its idiographic flow is as follows:

(14.1) when cluster increases Redis service group, Redis pattern container is created;

(14.2) start Redis program, self information is published to etcd cluster/$ cid/elastic/redis catalogue under;

(14.3) start successfully to WebServer report, wait for being found by Manager node.

Proxy pattern: after the container being in Proxy pattern normally works, in store complete slot is to the routing table of concrete Redis server.Due to its subscription/slot catalogue, so when belonging to certain slot, service group changes, can upgrade in time route.After container reception to the request of Client, after key is carried out hash process, obtain slot belonging to this key.Then routing table is looked into, by request forward to corresponding Redis server.Finally, by the acknowledgement transmissions of Redis server to this Client module.

The major function of Proxy pattern starts above-mentioned Proxy software, collects and Accreditation System state.The Starting mode of Proxy pattern is divided into initialization startup and elasticity to start, and the occurrence scene of two kinds of Starting mode is with Redis pattern.

The workflow that Figure 15 shows Proxy mode initialisation startup container is:

(15.1) enter Proxy pattern, subscription/init/manager catalogue, wait for that all Manager containers are registered and start;

(15.2) judge whether Manager container starts successfully, if start successfully, continue to perform (15.3); Otherwise, terminate.

(15.3) newly-built Manager client, and start Proxy program;

(15.4) issue self information in/init/Proxy catalogue, wait for that cluster starts successfully;

(15.5) cycle criterion event type, if receive timer-signals, then jumps to (15.6); If receive Manager fault-signal, then jump to (15.8);

(15.6) by Ping order, connect Manager node, obtain system mode;

(15.7) system mode is published to/stats/proxy catalogue under, then to jump to (15.5);

(15.8) subscription/failure/manager catalogue;

(15.9) wait for that catalogue upgrades, the new manager address after obtaining after renewal;

(15.10) upgrade the manager address in Context, and jump to (15.5).

The workflow that Figure 16 shows Proxy mode elastic startup container is:

(16.1) enter Proxy pattern, subscription/init/manager catalogue, obtain Manager node address;

(16.2) newly-built Manager client, and start Proxy program;

(16.3) cycle criterion event type, if receive timer-signals, then jumps to (16.4); If receive Manager fault-signal, then jump to (16.6);

(16.4) by Ping order, connect Manager node, obtain system mode;

(16.5) system mode is published to/stats/proxy catalogue under, then to jump to (16.3);

(16.6) subscription/failure/manager catalogue;

(16.7) wait for that catalogue upgrades, the new manager address after obtaining after renewal;

(16.8) upgrade the manager address in Context, and jump to (16.3).

Manager pattern is most crucial pattern.Container major function under Manager pattern is: monitoring cluster server run time behaviour, collect cluster load information, Processing Cluster is stretched, and slot moves, management Proxy node and Redis node.Manager Starting mode is divided into: start two kinds of modes after initialization startup and fault.

Figure 17 shows the idiographic flow that initialization under Manager pattern starts container:

(17.1) according to system parameters generating configuration file, configuration file content comprises etcd cluster address, Manager address of service, cache cluster ID.

(17.2) Manager software-Codis is started according to configuration file.

(17.3) Codis order initialization slot information in etcd is used.

(17.4) subscription/init/redis catalogue, waits for that all Redis service groups are registered.

(17.5) Codis order is used by all Redis Node registry to Redis cluster-list.

(17.6) perform rebalance order and join slot for each Redis serves component.

(17.7) notify that Proxy node executes rebalance operation, make the pending subsequent flows journeys such as it terminates.

(17.8) subscription/init/proxy catalogue, waits for proxy registration.

(17.9) to WebServer module report cluster success log-on message.

(17.10) start successfully, finger daemon regularly collects Proxy node by etcd cluster, collects Redis node system information, collector node run time behaviour, decision event type by long-distance support Info order.

(17.11) if there is Proy fault, then Proxy fault message is submitted to WebServer module, and jump to Proxy troubleshooting process.

(17.12) if there is Redis fault, then Redis fault message is submitted to WebServer module, and jump to increase Redis node flow process.

(17.13) if do not have fault to occur, the node run time behaviour of collection is integrated, judges whether elastic telescopic condition of setting out.If meet condition of setting out, then further decision event type, jumps to corresponding handling process (increase Redis service group, delete Redis, increase Proxy, delete Proxy).

Figure 18 shows the idiographic flow starting container under Manager pattern after fault:

(18.1) according to system parameters generating configuration file, configuration file content comprises etcd cluster address, Manager address of service, cache cluster ID.

(18.2) Manager software-Codis is started according to configuration file.

(18.3) start successfully, self information is published to/failure/manager catalogue by finger daemon, performs Cyclic Rings decision event type of process.

(18.4) if there is Proy fault, then Proxy fault message is submitted to WebServer module, and jump to Proxy troubleshooting process.

(18.5) if there is Redis fault, then Redis fault message is submitted to WebServer module, and jump to increase Redis node flow process.

(18.6) if do not have fault to occur, the node run time behaviour of collection is integrated, judges whether elastic telescopic condition of setting out.If meet condition of setting out, then further decision event type, jumps to corresponding handling process (increase Redis service group, delete Redis, increase Proxy, delete Proxy).

Before user uses Client end, need to submit user name, password, Redis cluster ID to, carry out user rs authentication by WebServer module.After being verified, WebServer module returns the Proxy list of corresponding Redis cluster.In addition, the timing of Client end upgrades Proxy list, ensures the correctness of Proxy list.After Client end obtains Proxy list, corresponding load-balancing algorithm can be selected to send read-write requests to Proxy.User can arrange load-balancing algorithm voluntarily, and load-balancing algorithm comprises: randomized, polling method, WRR method, dynamic weighting polling method, fastest response precedence method.

Figure 19 illustrates Client end and upgrades Proxy list workflow.Conveniently user operation, Client end directly provides Do method, performs all Redis and operates.Proxy connects work and scheduling strategy Hide All performs, and performs user transparent.During each execution Do method, first, obtained the Proxy that connect by scheduler, then from the connection pool of this Proxy, obtain connection, initiate command request by this connection.Due to elasticity event, Proxy node can irregularly upgrade, and in order to ensure that Client rectifies normal work, client needs the Proxy list that upgrades in time.Its idiographic flow is:

(19.1) decision event type, event type is divided into Proxy node to lose company, timer then.

(19.2) lose even if event is Proxy, then redirect performs step (19.3); If event be timer then, then redirect perform step (19.6).

(19.3) the IP address of the company of mistake Proxy node is obtained.

(19.4) connection status of this node in Porxy list is set to " abnormal ", prevents other threads from using this connection.

(19.5) close set up with this Proxy in connection pool all to be connected.Jump to (19.1) to continue to perform.

(19.6) WebServer address of service is obtained by etcd client.

(19.7) send out request to WebServer module, obtain all Proxy address lists.

(19.8) circulation address acquisition from the list newly obtained, judges whether there is this address in existing list.

(19.9) connection pool of this Proxy if there is no, is then set up; Otherwise, do not set up.

(19.10) after all address process complete, circulate address acquisition from existing list, and judge whether there is this address in new acquisition list.

(19.11) if do not exist, then this Proxy is set to " abnormal ", and closes all connections; Otherwise, do not process.

(19.12) jump to (19.1) to continue to perform.

Claims (10)

1. an elastic caching system under Docker cloud platform, is characterized in that, comprises WebServer module, Docker cluster management module, Docker host cluster, MySQL database, Etcd cluster, Docker mirror image warehouse and client; Client holds transmission user and authorization information to WebServer module; Docker container is controlled related data by WebServer module, acquisition host status command is sent to Docker cluster management module; WebServer module reads with write host state, user profile to MySQL database; WebServer module sends container starting state information to Etcd cluster; Client end reading cache data, Proxy node listing information from Docker host cluster; Docker host cluster transmission host status data, Docker management data are to Docker cluster management module; DockerImage data are sent to Docker cluster management module from Docker mirror image warehouse; Docker host cluster transmission cache cluster starts related data, Docker container state data, elasticity event data, disaster tolerance event data to Etcd cluster.

2. elastic caching system under Docker cloud platform as claimed in claim 1, it is characterized in that, the startup of WebServer module in charge Docker container and deletion, process Redis cluster elastic telescopic request and the request of disaster tolerance aspect, and process Client holds various request, and provide visualization interface for keeper and user; WebServer uses JSON as data interchange format;

Docker cluster management module, by Etcd cluster, manages Docker host cluster; Subscribe to respective directories during startup, self information is regularly published to same catalogue by Docker host; After Docker cluster management module receives the information of subscription, upgrade host list and host machine system state according to the subscription information received; Simultaneously, Docker cluster management module provides service for WebServer module, comprises the information of collecting container in Docker host, newly-built or destruction container, timed collection Docker host machine system situation (cpu load, memory information, bandwidth, I/O information).

3. elastic caching system under Docker cloud platform as claimed in claim 1, is characterized in that, Docker host cluster: Docker host cluster has multiple Redis cluster to form; The corresponding DockerImage of each Redis cluster; In DockerImage, an integrated container finger daemon module, and dissimilar container node; In Docker host cluster, container node type is divided into Redis pattern, Proxy pattern and Manager pattern; According to the difference of Docker container node type, start corresponding flow process, mainly comprise start-up course management, node state monitoring, elasticity realizes, node failure reports.

4. elastic caching system under Docker cloud platform as claimed in claim 1, is characterized in that, in Docker cloud platform, for each user distributes exclusive Redis cluster, without any communication between each Redis cluster, realizes many tenants isolation; Client end can direct communication with Proxy node in cluster; User can select to set up RedisSlave node when configuring Redis cluster, and based on the leader follower replication function of Redis, slave node has and master node identical data; When RedisMaster node failure, in real time slave node is promoted to master node.

5. elastic caching system under Docker cloud platform as claimed in claim 1, it is characterized in that, the Client end node that user uses must be in consolidated network with caching system, and Client end, by ICP/IP protocol and Redis trunking communication, is communicated with WebServer by http protocol; Adopt all connections of connection pool management Client and Proxy; After user carries out user rs authentication by WebServer module, WebServer module returns the Proxy list of corresponding Redis cluster; In addition, the timing of Client end upgrades Proxy list, ensures the correctness of Proxy list; After Client end obtains Proxy list, corresponding load-balancing algorithm can be selected to send read-write requests to Proxy; User can arrange load-balancing algorithm voluntarily, and load-balancing algorithm comprises: randomized, polling method, WRR method, dynamic weighting polling method, fastest response precedence method.

6. elastic caching system under Docker cloud platform as claimed in claim 1, is characterized in that, Docker mirror image warehouse: for Docker host provides DockerImage download service; Adopt the privately owned Docker mirror image warehouse set up based on container; Construction method based on container first downloads Registry mirror image from DockerHub, then creates container based on this mirror image in the server, and starting this container successfully just can outwards provide registry to serve.

7. elastic caching system under Docker cloud platform as claimed in claim 1, it is characterized in that, WebServer module adopts stateless method for designing, can be extending transversely according to demand; After WebServer module starts, in Etcd cluster/webserver catalogue issue self information; When other modules send out request to WebServer module, first random certain WebServer address of service of acquisition from this catalogue, employs the service discovery function of etcd herein; According to request object, WebServer is subdivided into user's service module, keeper's service module, Client service module, Docker trunking communication module.

8. elastic caching system under Docker cloud platform as claimed in claim 1, it is characterized in that, Docker cluster management module is divided into host pattern, manager's pattern; In manager's pattern, main frame is actual is the agent node of host, its in store all host information, and outside use standard Docker agreement request manager manages host; During newly-built container, manager, with starting container Requests routing to certain host, after deletion container operation only need find the host of this container of actual motion, sends removal request; The cluster that the server of two kinds of patterns is formed a whole by Etcd cluster.

9. elastic caching system under Docker cloud platform as claimed in claim 3, is characterized in that, container finger daemon module integration, in DockerImage, when container starts, first starts container and guards module; Container running state information adopts Context structure to describe, and preserves wherein; When WebServer starts container, by all container Set up-conservancy in system parameters, according to system parameters initialization Context structure; After initialization, start corresponding container pattern according to Context, wherein Manager node is responsible for the management of whole cluster and is kept in etcd cluster by cluster relevant information; Proxy is the agency service that Client connects Redis, and itself achieves Redis agreement, and performance is served identical with a primary Redis; Redis is divided into master, slave two kinds of nodes as storage engines, and slave copies data in master, and these two kinds of nodes constitute a ServerGroup; The mode of Hash groove is adopted to carry out burst to data, data are distributed in 1024 slot according to CRC32 (key) %1024, slot is a virtual concept, data are actually stored in multiple ServerGroup, for ServerGroup without any distributed logic wherein, each ServerGroup is responsible for a part of key-value data.

10. elastic caching system under Docker cloud platform as claimed in claim 1, is characterized in that, in order to realize the elastic telescopic function of cluster, using the cluster elastic telescopic based on dynamic Three-exponential Smoothing prediction algorithm to realize;

In Redis cluster, the load information of all Proxy, Redis nodes of Manager node timed collection; The load information of Proxy comprises memory usage M _usage, CPU usage C _usage, bandwidth utilization rate N _usage; The information of Redis node only includes memory usage M _usage;

For Redis node, Manager can by the M of all Redis nodes _usagedata, combine the overall load forming Redis node; When after system long-play, each node load situation skewness may be caused, therefore, the dispersion of these Redis node load information of Water demand during integration, employing formula (1) calculates Redis node load information and is:

$\mathrm{\σ}=\sqrt{\frac{1}{n}\underset{i=1}{\overset{n}{\Σ}}{(100M_i-\mathrm{\μ})}^2},\mathrm{\μ}=\frac{100}{n}\underset{i=1}{\overset{n}{\Σ}}{M}_i---\left(1\right)$

Wherein i represents the sequence number of Redis node, if σ >=20, illustrates that load Distribution is uneven, now needs to perform Rebalance operation, redistributes these Redis node loads; μ represents the overall load of Reids node, and be saved in Redis load time series Rt, t=1,2 ..., in n}; The queue of time series regular length is kept in the internal memory of Manager node;

For Proxy node, Client end adopts polling mode to communicate with Proxy, and the loading condition of each Proxy node is more or less the same; The overall load P of all Proxy nodes _allformula (2) can be utilized to calculate:

$P_{all}=\frac{100}{n}\underset{i=1}{\overset{n}{\Σ}}{P}_i,P_i=max(M_i,C_i,N_i),i=1,2,...,N---\left(2\right)$

Equally, P _allrepresent the overall load of Proxy node, and be saved to Proxy load time series Pt, t=1,2 ..., n};

After the overall load of Redis node and Proxy node calculates, elastic telescopic engine can be analyzed corresponding two time serieses, and two kinds of time Series Processing methods are identical.