CN106161644A - The distributed system of data process and data processing method thereof - Google Patents
The distributed system of data process and data processing method thereof Download PDFInfo
- Publication number
- CN106161644A CN106161644A CN201610666221.1A CN201610666221A CN106161644A CN 106161644 A CN106161644 A CN 106161644A CN 201610666221 A CN201610666221 A CN 201610666221A CN 106161644 A CN106161644 A CN 106161644A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- server
- unit
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
nullThe invention provides distributed system and data processing method thereof that a kind of data process，This system includes: monitor client，Unit to be monitored，Data processing server group and data base，Unit to be monitored is provided with the GPRS module for transmitting unit initial data，Wherein，Data processing server group includes application server、Caching server、Data Access Server and interface server，Wherein，Application server is connected with GPRS module and caching server，Interface server respectively with caching server、Data base and monitor client are connected，Data Access Server is connected with caching server and data base，The scheme that the present invention provides can efficiently solve autgmentability in the distributed system that in prior art, data process、Compatible and that safety is the most perfect problem，Improve autgmentability in the distributed system that data process、Compatibility and safety.
Description
Technical field
The present invention relates to technical field of data processing, the distributed system that processes in particular to a kind of data and
Data processing method.
Background technology
Long-distance intelligent service center system can carry out remote collection and remotely divide the service data of Central air-conditioning unit
Analysis, provides the user unit daily maintenance information in time, and when unit produces fault and alarm, thus effectively can subtract
Few unit fault produces probability and stops failure propagation, improve that Central air-conditioning unit is many, management dispersion, man-machine interaction difference etc.
Economy and safe operation problem, provide the user central air-conditioning service more laborsaving, that save worry, save money.
And along with the arrival of air conditioning unit big data age, system needs the engineering unit of monitoring also to get more and more.Existing
Being applied in technology remotely monitors the server distribution formula system of Central air-conditioning unit as it is shown in figure 1, this system can centering
Entreat air conditioning unit service data to carry out remote collection and remote analysis, but this system is in autgmentability, compatibility and safety
Property aspect is the most perfect, and some situation fails to consider.Such as: the client in the system of Fig. 1 directly connects data base, can carry
Carry out certain potential safety hazard.Relationship processing module in the system of Fig. 1 makes the complexity that GPRS module and client relationship process
Property, and easily become magnanimity GPRS module and the mutual bottleneck of a large amount of client.Additionally, Client (client) AM access module is adopted
Connect with socket, it is difficult to extend compatible multiple client.Such as: mobile client, desktop client end and third-party application journey
Sequence etc..
For autgmentability, compatibility and the most perfect the asking of safety in the distributed system that data in prior art process
Topic, the most not yet proposes effective settling mode.
Summary of the invention
Embodiments provide distributed system and data processing method thereof that a kind of data process, existing to solve
The problem that in the distributed system that in technology, data process, autgmentability, compatibility and safety are the most perfect.
According to an aspect of the invention, it is provided the distributed system that a kind of data process, including monitor client, treat
Monitoring unit, data processing server group and data base, unit to be monitored is provided with for transmitting unit initial data
GPRS module, wherein, data processing server group includes application server, caching server, Data Access Server
And interface server, wherein, application server is connected with GPRS module and caching server, and interface server is respectively with slow
Depositing server, data base and monitor client to be connected, Data Access Server is connected with caching server and data base.
In one embodiment, this system also includes: load equalizer, and application server is one or more, application clothes
Business device is connected with GPRS module by load equalizer.
In one embodiment, application server includes the DTU AM access module for being connected with GPRS module, with
The data resolution module that DTU AM access module is connected, DTU AM access module and data resolution module are respectively with caching server even
Connect.
In one embodiment, interface server includes WCF intermediate cluster module, WCF intermediate cluster module respectively with
Caching server, data base and monitor client are connected.
In one embodiment, caching server includes that Redis cluster cache module, Redis cluster cache module are divided
It is not connected with DTU AM access module, data resolution module and WCF intermediate cluster module.
In one embodiment, Data Access Server includes unit data store set group's module, and unit data store
Cluster module is connected with Redis cluster cache module and data base respectively.
In one embodiment, also including engineering information synchronization module in Data Access Server, engineering information synchronizes mould
Block is connected with Redis cluster cache module and data base respectively.
In one embodiment, data base includes: data-base cluster module.
In one embodiment, monitor client includes: pc client, and/or mobile client, and/or webpage client
End, and/or third-party application client.
According to another aspect of the present invention, it is provided that a kind of data process side being applied in above-mentioned distributed system
Method, including: application server obtains the initial data of the unit to be monitored that GPRS module sends；Application server is to initial data
Carry out parameterized treatment, and the data after parameterized treatment are sent to caching server；Data Access Server is from caching clothes
Data after getting parms the data after parametrization are stored to data base in business device；Interface server is from buffer service
Data after getting parms in device, and the data after parametrization are distributed to the monitor client of correspondence.
In one embodiment, distributed system includes: load equalizer, and application server is multiple, application server
Obtain the initial data of the unit to be monitored that GPRS module sends, including: GPRS module is connected to load equalizer；Load balancing
Connection request, according to the load condition of application server, is sent the application server to load minimum by device；What load was minimum should
The initial data of the unit to be monitored that GPRS module sends is obtained with server.
In one embodiment, application server carries out parameterized treatment to initial data, and by after parameterized treatment
Data send to caching server, including: the DTU AM access module in application server receives the machine to be monitored that GPRS module sends
The initial data of group；The initial data of the unit to be monitored received is sent the data to application server by DTU AM access module
Parsing module；The initial data of unit to be monitored is resolved by data resolution module, and sends the data resolved to slow
Deposit the Redis cluster module of server.
In one embodiment, Data Access Server data after getting parms by parameter from caching server
Data after change store to data base, including: the unit data store set group's module in Data Access Server is according in advance
The subscribing relationship set up with the Redis cluster module of caching server, number after getting parms from Redis cluster module
According to；Data after the parametrization of acquisition are stored in data base by unit data store set group's module.
In one embodiment, interface server data after getting parms from caching server, and by parametrization
After data be distributed to the monitor client of correspondence, including: the WCF intermediate cluster module in interface server according in advance with delay
Deposit the subscribing relationship that the Redis cluster module of server is set up, data after getting parms from Redis cluster module；WCF
Data after the parametrization of acquisition are distributed to the monitor client of correspondence by intermediate cluster module.
In one embodiment, Data Access Server also includes engineering information synchronization module, knows caching in monitoring
When the Redis cluster module of server triggers engineering information amendment, the engineering information of amendment is synchronized by engineering information synchronization module
To data base.
The present invention data process distributed system and data processing method in, whole distributed system is by existing
In technology, unit GPRS module has replaced to caching server by the way of relationship processing module is connected with client and has utilized it
The mode of self publish/subscribe characteristic.Which unit data is client need then go at caching server to subscribe to that unit
Data, it is not necessary to do not subscribe to, these subscribe to relations gone by caching server self, application server is without removing record
Which unit GPRS module of which client control, decreases unit GPRS module complicated with the logic that client relationship processes
Property, therefore reduce the coupling between unit GPRS module and client, for follow-up independent expanded application server or list
Solely expansion interface providing convenience property of server.When expanded application server, it is not necessary to consider what its extension was caused by client
Impact.When expansion interface server, without considering that unit GPRS module extends, on it, the impact caused.Meanwhile, interface clothes
Business device can arrange supported interface mode (such as WCF interface mode) and be supplied to various types of client and be attached, side
Continue after an action of the bowels and can extend the connection of further types of client, and, client-access data base can be limited by interface server
Some authority, improve security of system.The scheme that the present invention provides can efficiently solve that data in prior art process point
The problem that in cloth system, autgmentability, compatibility and safety are the most perfect, improves in the distributed system that data process and extends
Property, compatibility and safety.
Accompanying drawing explanation
Constitute the accompanying drawing of a part of the application for providing a further understanding of the present invention, the signal of the present invention herein
Property embodiment and explanation thereof are used for explaining the present invention, are not intended that inappropriate limitation of the present invention.In the accompanying drawings:
Fig. 1 is the distributed system frame diagram that a kind of data of the prior art process；
The system framework figure of the distributed system that a kind of data that Fig. 2 provides for the embodiment of the present application process；
The flow process of the data processing method of the distributed system that a kind of data that Fig. 3 provides for the embodiment of the present application process is shown
It is intended to；And
The schematic flow sheet of a kind of data processing method that Fig. 4 provides for the embodiment of the present application.
Detailed description of the invention
For the technical scheme making those skilled in the art be more fully understood that in the application, real below in conjunction with the application
Execute the accompanying drawing in example, the technical scheme in the embodiment of the present application is clearly and completely described, it is clear that described enforcement
Example is only some embodiments of the present application rather than whole embodiments.Based on the embodiment in the application, this area is common
All other embodiments that technical staff is obtained under not making creative work premise, all should belong to the application protection
Scope.
The distributed system that the data provided in the embodiment of the present invention process includes monitor client, unit to be monitored, number
According to processing server group and data base, unit to be monitored is provided with the GPRS module for transmitting unit initial data,
It is characterized in that, data processing server group includes application server, caching server, Data Access Server and connects
Mouthful server, wherein, application server is connected with GPRS module and caching server, interface server respectively with buffer service
Device, data base and monitor client are connected, and Data Access Server is connected with caching server and data base.Wherein, excellent
Selection of land, above-mentioned application server, caching server and Data Access Server can be all Linux server.
In the embodiment of the present application, in order to the function of each server in above-mentioned data processing server group is carried out solely
Vertical, can be by each function modoularization in each server.Specifically, a kind of data that Fig. 2 provides for the embodiment of the present application
The frame diagram of the distributed system processed.As in figure 2 it is shown, this distributed system also includes load equalizer, when implementing, can
Load equalizer to be arranged at a computer center.The application server provided in the embodiment of the present invention can be one, also
Can be multiple multiple situations of application server 1 to n (Fig. 2 illustrate), application server be by load equalizer and GPRS mould
Block connects.
Specifically, application server includes DTU (the Data Transfer for being connected with GPRS module
Unit, data transmission unit) AM access module, the data resolution module being connected with DTU AM access module, DTU AM access module sum
It is connected with caching server respectively according to parsing module.
In one embodiment, interface server includes WCF intermediate cluster module, and WCF intermediate cluster module is respectively
It is connected with caching server, data base and monitor client.Wherein it is preferred to, as in figure 2 it is shown, monitor client includes: PC
Client, and/or mobile client, and/or webpage client, and/or third-party application client.WCF herein is by micro-
The application framework of a series of support data communication of soft exploitation, is properly termed as Windows communication development platform.It incorporates
The .net Remoting of original windows communication, the mechanism of WebService, Socket, and fusion have HTTP's and FTP
Correlation technique, is the practice mode that on windows platform, development DDB application is optimal.
It addition, in the embodiment of the present application, in order to improve the performance of caching server, memory database can be used
Redis carries out transfer and the distribution of unit initial data.Therefore, caching server includes Redis cluster cache module,
Redis cluster cache module is connected with DTU AM access module, data resolution module and WCF intermediate cluster module respectively.Data are visited
Ask that server includes unit data store set group's module, unit data store set group's module respectively with Redis cluster cache mould
Block and data base are connected.
In the embodiment of the present application, Redis cache module can be in the way of using publish/subscribe, unit data store set
Group's module and WCF middleware cluster module all subscribe to the data after resolving from Redis.When receiving unit data on Redis
After, according to himself publish/subscribe characteristic, automatically unit data can be automatically pushed to subscriber.Such mode is to follow-up
Function of data distribution during distributed server extends on a large scale plays vital effect, and multiple program modules are attached to
Message is received, such that it is able to solve multi-to-multi between multiple module to need multi-to-multi when sending message on Redis cache module
The problem connected.Further, Redis utilizes the mode of himself publish/subscribe characteristic so that client can be ordered according to its demand
Read data, need which unit data then to go to subscribe to the data of that unit, it is not necessary to not subscribe to, these subscribe to relations by
Redis self has gone, and application server, without going to record which unit GPRS module of which client control, decreases machine
The logic complexity that group GPRS module processes with client relationship, therefore reduces the coupling between unit GPRS module and client
Conjunction property, for follow-up independent expanded application server or independent extension WCF middleware providing convenience property of cluster.In expanded application
During server, it is not necessary to consider that client extends, on it, the impact caused.In like manner when extending WCF middleware, without considering machine
Group GPRS module extends, on it, the impact caused.
In the prior art, the most logical between GPRS module and the server at intelligent server center of unit to be monitored
Cross BMS agreement (remote monitoring protocol) and carry out communication.This agreement is set by the display panel controller of each unit to be monitored often
Meter personnel provide, and the program needing to be write as this agreement correspondence joins in the display panel controller of unit to be monitored and just may be used
To realize this unit to be monitored is monitored.So, along with the increase of unit quantity to be monitored, BMS agreement is write as correspondence
The process of program then seem the heaviest.In order to alleviate the pressure of system maintenance personnel, the embodiment of the present application can be at GPRS
Modbus agreement is used to carry out data transmission between module and DTU AM access module, such that it is able to avoid showing at unit to be monitored
Show the step of the program that write is corresponding with agreement in plate controller, such that it is able to alleviate the pressure of system maintenance personnel.
In the embodiment of the present application, the design philosophy that object-oriented is inherited can be used, from top to bottom by agreement with wait to supervise
The type of control unit is inherited classification step by step and is processed.First Modbus class can be derived by parent, then under Modbus class
Can derive again multiple type subclass, the common logical process part of each type can be placed on Modbus apoplexy due to endogenous wind and process, no
Process then can be placed in respective type subclass with logical process part.This ensures that there the unit of either current monitor
Or follow-up unit newly developed, the distributed system compatible processing that its unit data can be provided by the application, its unit
Data can be monitored by distributed system.
The distributed system processed below in conjunction with the prior art data to providing in the present embodiment is described, in order to
It is best understood from the advantage place of the present invention:
Distributed system architecture of the prior art is as it is shown in figure 1, whole system is divided into DTU to access, Client accesses, pass
System's process, data parsing, Redis, data access interface, data 7 modules of storage.
System architecture of the prior art is the most perfect, and multiformity and system extension in view of client may bands
The bottleneck come.As it is shown in figure 1, relationship processing module is ined succession, DTU AM access module and Client AM access module, be between both
The bridge of communication.After unit GPRS module rolls up, for ensureing that server load is stable, DTU AM access module is also required to phase
Should increase, in like manner, after client rolls up, Client AM access module is also required to increase accordingly.In relationship processing module by
In have recorded the corresponding relation of all unit GPRS module and all client controls, so being difficult to be extended to multiple relation
Processing module, then becomes the bottleneck that magnanimity unit GPRS module is mutual with a large amount of clients, it is suppressed that whole system can
Autgmentability.Simultaneously take account of the multiformity of client type follow-up developments, it is difficult to accomplish that all types of client is all unified to adopt
Connect Client AM access module by socket mode, and client is directly connected to data base, also can give the data peace of data base
Hidden danger served by full band.
The system framework providing the present invention shown in this Fig. 2, whole system is divided into DTU access, data parsing, Redis
6 modules such as cluster, unit data store set group, engineering information synchronization, WCF middleware cluster.Whole system workflow diagram
As shown in Figure 3:
First being connected load equalizer by unit GPRS module, load equalizer can be according to the load shape of each application server
State, in the DTU AM access module of the application server that the connection request of unit GPRS module is forwarded to most lightly loaded；
After GPRS module is connected with DTU AM access module foundation, original unit data are issued DTU and are accessed mould by GPRS module
Block；
Original unit data are issued data resolution module by DTU AM access module more afterwards, and data resolution module is to original machine
Group data resolve, after being parsed, by data publication to Redis cluster.
Unit data store set group's module and WCF middleware cluster module all subscribe to the data after resolving from Redis.When
After receiving unit data on Redis, according to himself publish/subscribe characteristic, can automatically be automatically pushed to order by unit data
The side of readding.
After unit data store set group's module gets unit data, data are stored in data base.
After WCF middleware cluster gets unit data, data are distributed to client.
Early stage unit GPRS module is replaced to by the way of relationship processing module is connected by whole system with client
Redis utilizes the mode of himself publish/subscribe characteristic.Which unit data is client need then remove to subscribe to the number of that unit
According to, it is not necessary to not subscribing to, these relations subscribed to have been gone by Redis self, and application server is without going to record which visitor
Which unit GPRS module family end has monitored, and decreases the logic complexity that unit GPRS module processes with client relationship, because of
This reduces the coupling between unit GPRS module and client, for follow-up independent expanded application server or individually extends
WCF middleware providing convenience property of cluster.When expanded application server, it is not necessary to consider that client extends, to it, the shadow caused
Ring.In like manner when extending WCF middleware, without considering that unit GPRS module extends, on it, the impact caused.
System by general WCF interface mode be supplied to various types of client be attached rather than with
The mode of socket, this be follow-up can extend further types of client connect.Early stage client is permissible simultaneously
It is directly connected to data base, if the program developed by third party is to connect data base, the safety of data can be able to be brought necessarily
Hidden danger, so the system of the present invention is by the way of WCF interface, limits some authority of client-access data base, carry
Rise the safety of system data.
Embodiment 2
In the embodiment of the present application 2, also provide for what a kind of data being applied to the distributed system that above-mentioned data process processed
Method.The method flow diagram that a kind of data that Fig. 4 provides for the embodiment of the present application process.Although flow process being described below include with spy
Multiple operations that definite sequence occurs, but it should be clearly understood that these processes can include more or less of operation, these behaviour
Work can sequentially perform or executed in parallel (such as using parallel processor or multi-thread environment).As shown in Figure 4, method can be wrapped
Include S1-S4:
S1: application server obtains the initial data of the unit to be monitored that GPRS module sends；
Initial data in unit to be monitored can be transmitted by GPRS module thereon, and GPRS module can be by former
Beginning data send the DTU AM access module to application server.In an optional embodiment, distributed system includes:
Load equalizer, application server is multiple, and when stating S1 in realization, GPRS module is connected to load equalizer；Load balancing
Connection request, according to the load condition of application server, is sent the application server to load minimum by device；What load was minimum should
The initial data of the unit to be monitored that GPRS module sends is obtained with server.
S2: application server carries out parameterized treatment to initial data, and sends the data after parameterized treatment to slow
Deposit server；
When implementing, the DTU AM access module in application server receives the former of the unit to be monitored of GPRS module transmission
Beginning data；The initial data of the unit to be monitored received is sent the data parsing mould to application server by DTU AM access module
Block；The initial data of unit to be monitored is resolved by data resolution module, and sends the data resolved to buffer service
The Redis cluster module of device.
S3: Data Access Server data after getting parms the data after parametrization are deposited from caching server
Store up to data base；
When implementing, the unit data store set group's module in Data Access Server according in advance with caching server
The subscribing relationship set up of Redis cluster module, data after getting parms from Redis cluster module；Unit data are deposited
Data after the parametrization of acquisition are stored in data base by accumulation module.
S4: interface server data after getting parms from caching server, and the data after parametrization are distributed
Give corresponding monitor client.
When implementing, the WCF intermediate cluster module in interface server according in advance with the Redis collection of caching server
The subscribing relationship that group's module is set up, data after getting parms from Redis cluster module；WCF intermediate cluster module will obtain
Data after the parametrization taken are distributed to the monitor client of correspondence.
Additionally, as preferably, Data Access Server also includes engineering information synchronization module, know caching clothes in monitoring
When the Redis cluster module of business device triggers engineering information amendment, the engineering information of amendment is synchronized to by engineering information synchronization module
Data base.
The present invention provide preferred embodiment in, whole distributed system is by unit GPRS module in prior art
By the way of relationship processing module is connected, replaces to caching server with client utilizes himself publish/subscribe characteristic
Mode.Which unit data is client need then go to subscribe to the data of that unit at caching server, it is not necessary to do not subscribe to,
These relations subscribed to have been gone by caching server self, application server without go to record which client control which
Unit GPRS module, decreases the logic complexity that unit GPRS module processes with client relationship, therefore reduces unit
Coupling between GPRS module and client, carries for follow-up independent expanded application server or independent expansion interface server
Supply convenience.When expanded application server, it is not necessary to consider that client extends, on it, the impact caused.In expansion interface service
During device, without considering that unit GPRS module extends, on it, the impact caused.Meanwhile, interface server can arrange connecing of being supported
Mouthful mode (such as WCF interface mode) is supplied to various types of client and is attached, and facilitates and follow-up can extend more multiclass
The client of type connects, and, some authority of client-access data base can be limited by interface server, improve system peace
Quan Xing.The scheme that the present invention provides can efficiently solve autgmentability, compatibility in the distributed system that in prior art, data process
Property and the most perfect problem of safety, improve autgmentability, compatibility and safety in the distributed system that data process.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for the skill of this area
For art personnel, the embodiment of the present invention can have various modifications and variations.All within the spirit and principles in the present invention, made
Any modification, equivalent substitution and improvement etc., should be included within the scope of the present invention.
Claims (15)
1. data process a distributed system, including monitor client, unit to be monitored, data processing server group with
And data base, described unit to be monitored is provided with the GPRS module for transmitting unit initial data, it is characterised in that described
Data processing server group includes application server, caching server, Data Access Server and interface server, its
In, described application server is connected with described GPRS module and described caching server, described interface server respectively with institute
State caching server, described data base and described monitor client to be connected, described Data Access Server and described caching clothes
Business device and described data base are connected.
2. distributed system as claimed in claim 1, it is characterised in that also include: load equalizer, described application server
For one or more, described application server is connected with described GPRS module by described load equalizer.
3. distributed system as claimed in claim 1 or 2, it is characterised in that described application server include for institute
Stating the DTU AM access module that GPRS module is connected, the data resolution module being connected with described DTU AM access module, described DTU connects
Enter module and described data resolution module is connected with described caching server respectively.
4. distributed system as claimed in claim 3, it is characterised in that described interface server includes WCF intermediate cluster
Module, described WCF intermediate cluster module respectively with described caching server, described data base and described monitor client phase
Even.
5. distributed system as claimed in claim 4, it is characterised in that described caching server includes that Redis cluster delays
Storing module, described Redis cluster cache module respectively with described DTU AM access module, described data resolution module and described WCF
Intermediate cluster module is connected.
6. distributed system as claimed in claim 5, it is characterised in that described Data Access Server includes unit data
Storage cluster module, described unit data store set group's module respectively with described Redis cluster cache module and described data
Storehouse is connected.
7. distributed system as claimed in claim 6, it is characterised in that also include in described Data Access Server that engineering is believed
Breath synchronization module, described engineering information synchronization module is connected with described Redis cluster cache module and described data base respectively.
8. distributed system as claimed in claim 1, it is characterised in that described data base includes: data-base cluster module.
9. distributed system as claimed in claim 1, it is characterised in that described monitor client includes: pc client, and/
Or mobile client, and/or webpage client, and/or third-party application client.
10. the data process side in the distributed system being applied to as described in any claim in claim 1 to 9
Method, it is characterised in that including:
Described application server obtains the initial data of the unit described to be monitored that described GPRS module sends；
Described application server carries out parameterized treatment to described initial data, and sends the data after parameterized treatment to slow
Deposit server；
Described Data Access Server obtains the data after described parametrization and by described parametrization from described caching server
After data store to data base；
Described interface server obtains the data after described parametrization from described caching server, and by after described parametrization
Data are distributed to the described monitor client of correspondence.
11. data processing methods as claimed in claim 10, it is characterised in that described distributed system includes: load balancing
Device, described application server is multiple, and described application server obtains the unit described to be monitored of described GPRS module transmission
Initial data, including:
Described GPRS module is connected to load equalizer；
Connection request, according to the load condition of application server, is sent the application service to load minimum by described load equalizer
Device；
The initial data of the unit described to be monitored of the application server acquisition described GPRS module transmission that described load is minimum.
12. data processing methods as described in claim 10 or 11, it is characterised in that described application server is to described original
Data carry out parameterized treatment, and the data after parameterized treatment are sent to caching server, including:
DTU AM access module in described application server receives the original number of the unit described to be monitored that described GPRS module sends
According to；
The initial data of the unit described to be monitored received is sent the number to described application server by described DTU AM access module
According to parsing module；
The initial data of described unit to be monitored is resolved by described data resolution module, and the data resolved is sent extremely
The Redis cluster module of described caching server.
13. data processing methods as described in claim 10 or 11, it is characterised in that described Data Access Server is from described
Caching server obtains the data after described parametrization and the data after described parametrization are stored to data base, including:
Unit data store set group's module in described Data Access Server according in advance with the Redis of described caching server
The subscribing relationship that cluster module is set up, obtains the data after described parametrization from described Redis cluster module；
Data after the described parametrization that described unit data store set group's module will obtain are stored in data base.
14. data processing methods as described in claim 10 or 11, it is characterised in that described interface server is from described caching
Server obtains the data after described parametrization, and the data after described parametrization are distributed to the described monitoring client of correspondence
End, including:
WCF intermediate cluster module in described interface server according in advance with the Redis cluster module of described caching server
The subscribing relationship set up, obtains the data after described parametrization from described Redis cluster module；
Data after the described parametrization that described WCF intermediate cluster module will obtain are distributed to the monitor client of correspondence.
15. data processing methods as described in claim 10 or 11, it is characterised in that also include work in Data Access Server
Journey synchronizing information module, when monitoring the Redis cluster module knowing described caching server and triggering engineering information amendment, described
The described engineering information of amendment is synchronized to described data base by engineering information synchronization module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610666221.1A CN106161644B (en) | 2016-08-12 | 2016-08-12 | Distributed system for data processing and data processing method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610666221.1A CN106161644B (en) | 2016-08-12 | 2016-08-12 | Distributed system for data processing and data processing method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106161644A true CN106161644A (en) | 2016-11-23 |
| CN106161644B CN106161644B (en) | 2021-11-02 |
Family
ID=57330001
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610666221.1A Active CN106161644B (en) | 2016-08-12 | 2016-08-12 | Distributed system for data processing and data processing method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106161644B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107070977A (en) * | 2017-01-13 | 2017-08-18 | 广西电网有限责任公司电力科学研究院 | It is a kind of to reduce the data transmission method of delay |
| CN107180082A (en) * | 2017-05-03 | 2017-09-19 | 珠海格力电器股份有限公司 | A kind of data update system and method based on multi-level buffer mechanism |
| CN107404517A (en) * | 2017-07-10 | 2017-11-28 | 珠海格力电器股份有限公司 | Control method, dispatch server and the system of mixed-media network modules mixed-media |
| CN107864223A (en) * | 2017-12-14 | 2018-03-30 | 科大智能电气技术有限公司 | For the data communication system and method between internet of things equipment terminal and server |
| CN108833597A (en) * | 2018-08-14 | 2018-11-16 | 秭归县社会保险基金征收稽查局 | Golden insurance project intelligent monitoring and SiteServer LBS and monitoring method |
| CN110752970A (en) * | 2019-10-24 | 2020-02-04 | 北京纳兰德科技股份有限公司 | cuss platform monitoring system |
| CN111773738A (en) * | 2020-08-13 | 2020-10-16 | 网易（杭州）网络有限公司 | Game fighting method and device |
| CN114167794A (en) * | 2021-12-20 | 2022-03-11 | 苏州易助能源管理有限公司 | Remote data acquisition system and data acquisition method for intelligent electric meter |
| CN115190125A (en) * | 2022-06-27 | 2022-10-14 | 京东科技控股股份有限公司 | Monitoring method and device for cache cluster |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101876924A (en) * | 2009-04-30 | 2010-11-03 | 升东网络科技发展(上海)有限公司 | Database fault automatic detection and transfer method |
| CN102014355A (en) * | 2010-11-22 | 2011-04-13 | ***通信集团安徽有限公司 | System and method for displaying instant communication software signing information on calling cellphone |
| US20120110113A1 (en) * | 2009-07-02 | 2012-05-03 | Aim To G Co., Ltd. | Cooperative Caching Method and Contents Providing Method Using Request Apportioning Device |
| US20130097680A1 (en) * | 2011-10-17 | 2013-04-18 | Microsoft Corporation | High-density multi-tenant distributed cache as a service |
| CN103297485A (en) * | 2012-03-05 | 2013-09-11 | 日电（中国）有限公司 | Distributed cache automatic management system and distributed cache automatic management method |
| US20140068180A1 (en) * | 2012-08-31 | 2014-03-06 | International Business Machines Corporation | Data analysis system |
| CN105468763A (en) * | 2015-12-01 | 2016-04-06 | 广东电网有限责任公司电网规划研究中心 | Method for multi-person cooperation in big data operation |
| CN105592164A (en) * | 2016-01-25 | 2016-05-18 | 珠海格力电器股份有限公司 | Distributed system and method for data processing |
-
2016
- 2016-08-12 CN CN201610666221.1A patent/CN106161644B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101876924A (en) * | 2009-04-30 | 2010-11-03 | 升东网络科技发展(上海)有限公司 | Database fault automatic detection and transfer method |
| US20120110113A1 (en) * | 2009-07-02 | 2012-05-03 | Aim To G Co., Ltd. | Cooperative Caching Method and Contents Providing Method Using Request Apportioning Device |
| CN102014355A (en) * | 2010-11-22 | 2011-04-13 | ***通信集团安徽有限公司 | System and method for displaying instant communication software signing information on calling cellphone |
| US20130097680A1 (en) * | 2011-10-17 | 2013-04-18 | Microsoft Corporation | High-density multi-tenant distributed cache as a service |
| CN103297485A (en) * | 2012-03-05 | 2013-09-11 | 日电（中国）有限公司 | Distributed cache automatic management system and distributed cache automatic management method |
| US20140068180A1 (en) * | 2012-08-31 | 2014-03-06 | International Business Machines Corporation | Data analysis system |
| CN105468763A (en) * | 2015-12-01 | 2016-04-06 | 广东电网有限责任公司电网规划研究中心 | Method for multi-person cooperation in big data operation |
| CN105592164A (en) * | 2016-01-25 | 2016-05-18 | 珠海格力电器股份有限公司 | Distributed system and method for data processing |
Non-Patent Citations (2)
| Title |
|---|
| IGOR KOŠŤÁL: ""Benefits of the use of asynchronous methods in a WCF service and its client, synchronizing resource access between these methods"", 《INTERNATIONAL CONFERENCE ON MILITARY TECHNOLOGIES (ICMT) 2015》 * |
| 谭晓波等: ""多通道数字视频远程网络监控***及其数据缓存技术"", 《计算机应用》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107070977A (en) * | 2017-01-13 | 2017-08-18 | 广西电网有限责任公司电力科学研究院 | It is a kind of to reduce the data transmission method of delay |
| CN107180082A (en) * | 2017-05-03 | 2017-09-19 | 珠海格力电器股份有限公司 | A kind of data update system and method based on multi-level buffer mechanism |
| CN107180082B (en) * | 2017-05-03 | 2020-12-18 | 珠海格力电器股份有限公司 | Data updating system and method based on multi-level cache mechanism |
| CN107404517A (en) * | 2017-07-10 | 2017-11-28 | 珠海格力电器股份有限公司 | Control method, dispatch server and the system of mixed-media network modules mixed-media |
| CN107864223A (en) * | 2017-12-14 | 2018-03-30 | 科大智能电气技术有限公司 | For the data communication system and method between internet of things equipment terminal and server |
| CN108833597B (en) * | 2018-08-14 | 2024-04-26 | 秭归县社会保险基金征收稽查局 | Intelligent monitoring and load balancing system and monitoring method for gold protection engineering |
| CN108833597A (en) * | 2018-08-14 | 2018-11-16 | 秭归县社会保险基金征收稽查局 | Golden insurance project intelligent monitoring and SiteServer LBS and monitoring method |
| CN110752970A (en) * | 2019-10-24 | 2020-02-04 | 北京纳兰德科技股份有限公司 | cuss platform monitoring system |
| CN110752970B (en) * | 2019-10-24 | 2021-11-30 | 北京纳兰德科技股份有限公司 | cuss platform monitoring system |
| CN111773738A (en) * | 2020-08-13 | 2020-10-16 | 网易（杭州）网络有限公司 | Game fighting method and device |
| CN111773738B (en) * | 2020-08-13 | 2024-04-26 | 网易（杭州）网络有限公司 | Game sightseeing method and device |
| CN114167794A (en) * | 2021-12-20 | 2022-03-11 | 苏州易助能源管理有限公司 | Remote data acquisition system and data acquisition method for intelligent electric meter |
| CN115190125A (en) * | 2022-06-27 | 2022-10-14 | 京东科技控股股份有限公司 | Monitoring method and device for cache cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106161644B (en) | 2021-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106161644A (en) | The distributed system of data process and data processing method thereof | |
| CN102880475B (en) | Based on the real-time event disposal system of cloud computing and method in computer software | |
| CN109951331B (en) | Method, device and computing cluster for sending information | |
| CN112069265B (en) | Synchronization method of configuration data, business data system, computer system and medium | |
| CN106371932B (en) | Zookeeper-based information configuration method and device | |
| CN111090699A (en) | Service data synchronization method and device, storage medium and electronic device | |
| CN113254466B (en) | Data processing method and device, electronic equipment and storage medium | |
| US11753056B2 (en) | Cloud platform-based comprehensive line network monitoring method and system | |
| CN103761309A (en) | Operation data processing method and system | |
| CN105592164B (en) | A kind of distributed system and method for data processing | |
| CN103067230A (en) | Method for achieving hyper text transport protocol (http) service monitoring through embedding monitoring code | |
| CN107357571A (en) | Maintaining method, system and the component server of apparatus assembly program | |
| CN103870977A (en) | Method and system thereof for data synchronizing of mobile advertising platform | |
| CN109783151B (en) | Method and device for rule change | |
| CN107368583A (en) | A kind of method and system of more cluster information inquiries | |
| CN104167817A (en) | Power equipment real-time information integration system and method | |
| CN108228755A (en) | The data of MySQL database based on daily record analytic technique to Hadoop platform synchronize clone method | |
| CN113742033A (en) | Kubernetes cluster federal system and implementation method thereof | |
| CN110019539A (en) | A kind of method and apparatus that the data of data warehouse are synchronous | |
| CN102026228B (en) | Statistical method and equipment for communication network performance data | |
| CN112650545A (en) | Configuration management system, method and storage medium | |
| CN105302670A (en) | Method and device for monitoring station by multi-machine redundancy way | |
| CN109657000B (en) | Real-time data synchronization method and device of rail transit comprehensive monitoring system | |
| CN110534136A (en) | The method and apparatus of recording | |
| CN108874531A (en) | Method, apparatus, system and electronic equipment for the service of fusing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |