CN102929696A - Method and apparatus for constructing, submitting and monitoring center node of distributed system - Google Patents
Method and apparatus for constructing, submitting and monitoring center node of distributed system Download PDFInfo
- Publication number
- CN102929696A CN102929696A CN2012103710639A CN201210371063A CN102929696A CN 102929696 A CN102929696 A CN 102929696A CN 2012103710639 A CN2012103710639 A CN 2012103710639A CN 201210371063 A CN201210371063 A CN 201210371063A CN 102929696 A CN102929696 A CN 102929696A
- Authority
- CN
- China
- Prior art keywords
- message
- node
- election
- centroid
- distributed system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a construction method of a center node of a distributed system. The method comprises the following steps of: starting an election process by sending an election message, comparing an election round and a message request number of a node which receives the election message, and carrying out updating by adopting a large election round and a large message request number; forming a recommending message of the center node according to the updated election round and message request number, and sending the recommending message to the distributed system; and counting the recommending of the center node in the recommending message by the node, and setting the node having a recommending number reaching a preset threshold value as the center node. The embodiment of the invention further discloses a data request submitting method and a monitoring method based on the center node of the distributed system, and apparatuses of the data request submitting method and the monitoring method based on the center node of the distributed system. The center node constructed by the embodiment of the invention is stable.
Description
Technical field
The application relates to the distributed system technical field, particularly relates to a kind of distributed system Centroid construction method, based on data commit method and the method for supervising of distributed system Centroid, and each self-corresponding device.
Background technology
Distributed system (Distributed Systems) is to support the software systems of distributed treatment, and it is mainly used is to be executed the task by the multiprocessor architecture of interconnection of telecommunication network.Distributed system generally includes the classifications such as distributed operating system, distributed program design language and compiling thereof (explanation) system, distributed file system and distributed data base system according to having the difference of use scenes.Distributed system is compared with integrated system, and its stability is higher, has the ability that the automatic recovery capability of data and good anti-single-point collapse.But these abilities of distributed system depend on the performance of its Centroid to a great extent, and especially in network and the unsettled situation of hardware environment, Centroid concerns the normal operation of whole distributed system.The ability that does not have Centroid or a Centroid a little less than, all can not satisfy in the reality requirement to distributed system.
Summary of the invention
For solving the problems of the technologies described above, the embodiment of the present application provides a kind of method that makes up distributed Centroid, based on method for supervising and the corresponding intrument thereof of this distributed system Centroid, strengthening the stability of distributed system Centroid, and then improve the performance of whole distributed system.
The construction method of the distributed system Centroid that the embodiment of the present application provides comprises:
Node sends election message to distributed system, and described election message comprises election round and message request number;
Judge the election round of the node that receives election message and the magnitude relationship between the election round in the election message, if the election round of election message is large, then the election round of use election message is upgraded the election round of the node that receives election message; If receiving the election round of the node of election message equates with the election round of election message, magnitude relationship between the message request number of then judging the node receive election message and the message request number of electing in the message, if the message request number of election message is large, then use the message request of election message number to upgrade the message request number of the node that receives election message; Number form recommendation message to Centroid according to the election round after upgrading and message request, and send recommendation message to distributed system;
To the recommendation of Centroid, and recommended number is reached node centered by the node determination of predetermined threshold value in the node statistics recommendation message.
Preferably, to the recommendation of Centroid, and node comprises centered by recommended number being reached the node determination of predetermined threshold value in the described node statistics recommendation message:
Statistical form is put into the address of the Centroid of the node recommendation that sends recommendation message in the address of node that sends recommendation message, add up the recommended number of each recommended node as Centroid, recommended number is reached the node determination Centroid of predetermined threshold value.
Preferably, election message sends by the election thread of node; The node that receives election message receives election message by receiving thread, and the node that sends recommendation message sends recommendation message by send-thread.
The embodiment of the present application also provides a kind of request of data commit method based on the distributed system Centroid.The method comprises:
The data request information that non-central node is received is transmitted to Centroid, makes up according to the method for aforementioned structure Centroid to obtain;
The data request information that Centroid is received sends to the non-central node in the distributed system;
Size between ID number of the ID of the data request information that more non-central node receives number and non-central node self if the ID of data request information number large, is then returned acknowledge message to Centroid;
If the affirmation message that Centroid receives reaches predetermined threshold value, then the data of data request information are synchronized to all non-central nodes of distributed system.
The embodiment of the present application also provides a kind of method for supervising based on the distributed system Centroid.The method comprises:
Register monitor node in distributed system, described monitor node comprises config directory and state catalogue;
After the distributed system Centroid receives the message that the node in the distributed system changes, compare the config directory of monitor node and the node under the state catalogue, if node is present in config directory and is not present in the state catalogue, then send monitoring alarm, make up according to the method for aforementioned structure Centroid and obtain.
Preferably, described method also comprises:
Registration reported data catalogue in distributed system is so that node is set up new warning node after changing under this reported data catalogue;
After the distributed system Centroid receives the message that the node in the distributed system changes, obtain the node under the reported data catalogue, send monitoring alarm according to the node under the catalogue that reports that obtains.
Preferably, described method also comprises:
Registration distributed lock nodal directory in distributed system;
Obtain the node under the distributed lock nodal directory;
Judge the node of node number minimum in the node under the distributed node catalogue, to the node assignment profile formula lock of node number minimum.
The embodiment of the present application provides a kind of construction device of distributed system Centroid.This device comprises: transmitting element, the first judging unit, the second judging unit, updating block, formation unit, statistic unit and setting unit, wherein:
Described transmitting element is used for sending election message to distributed system, and described election message comprises election round and message request number;
Described the first judging unit, be used for judging the magnitude relationship between the election round that receives the node of electing message and the election round of the electing message, if the election round of election message is large, then trigger updating block, equate with the election round of election message if receive the election round of the node of election message, then trigger the second judging unit;
Magnitude relationship between described the second judging unit, message request number that be used for to judge the node that receives election message and the message request of election message number is if the message request of election message number greatly, then triggers updating block;
Described updating block, be used for when the election round of electing message is large, use the election round of election message to upgrade the election round that receives the node of electing message, when the message request of election message is number large, use the message request of election message number to upgrade the message request number of the node that receives election message;
Described formation unit is used for number forming recommendation message to Centroid according to the election round after upgrading and message request, and sends recommendation message to distributed system;
Described recommendation unit is used for the statistics recommendation message to the recommendation of Centroid;
Described determining unit is for node centered by the node determination that recommended number is reached predetermined threshold value.
The embodiment of the present application also provides a kind of request of data based on the distributed system Centroid to submit device to.This device comprises: retransmission unit, the mass-sending unit, the first comparing unit, return unit and lock unit, wherein:
Described retransmission unit is used for that non-central node is received data request information and is transmitted to Centroid, and described Centroid obtains according to the method structure of aforementioned structure Centroid;
Described mass-sending unit is used for the data request information that Centroid receives is sent to the non-central node of distributed system;
Described the first comparing unit is used for the size between ID number of data request information that more non-central node receives with non-central node self ID number, if the ID of data request information number large, then the unit is returned in triggering;
The described unit that returns is used for returning acknowledge message to Centroid;
Described lock unit is used for the data of data request information being synchronized to all non-central nodes of distributed system when the affirmation message that Centroid receives reaches predetermined threshold value.
The embodiment of the present application also provides a kind of supervising device based on the distributed system Centroid.This device comprises: registering unit, the second comparing unit and monitoring alarm unit, wherein:
Described registering unit is used for registering monitor node in distributed system, and described monitor node comprises config directory and state catalogue;
Described the second comparing unit, be used for after the distributed system Centroid receives the message that the node of distributed system changes the relatively config directory of monitor node and the node under the state catalogue, if node is present in config directory and is not present in the state catalogue, then trigger the monitoring alarm unit, described Centroid obtains according to the method structure of aforementioned structure Centroid;
Described monitoring alarm unit is used for sending monitoring alarm.
The node that the embodiment of the present application will send election message compares with the election round and the message request size that receive the node of electing message, according to the result after relatively more the election round of new node and message request number, generating recommendations message after upgrading, other nodes that receive recommendation message are added up recommended node as Centroid according to this recommendation message, and according to statistics Centroid are set.Compared with prior art, the single-point collapse can not appear in the Centroid that the embodiment of the present application makes up, in network and the unsettled situation of hardware, and safer, more easy-to-use, manageability more.The Centroid of setting up in the manner described above can be realized a series of application: when the Centroid that makes up by the embodiment of the present application comes the receive data request, to confirm through the Zookeeper service of predetermined number because every secondary data is submitted to all, guarantee preferably the data consistency between many Zookeeper; The Centroid that makes up by the embodiment of the present application comes the management and monitoring distributed system, because the node beyond monitor node, the monitor node all can carry out mutual with the Zookeeper service, when node state changes, can send monitor prompt, thereby in time monitor the running status of each node of distributed system.
Description of drawings
In order to be illustrated more clearly in the embodiment of the present application or technical scheme of the prior art, the below will do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art, apparently, the accompanying drawing that the following describes only is some embodiment that put down in writing among the application, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.
Fig. 1 is the process flow diagram of the distributed system Centroid construction method of the embodiment of the present application one;
Fig. 2 is the request of data commit method process flow diagram based on the distributed system Centroid of the embodiment of the present application two;
Fig. 3 is the process flow diagram based on the method for supervising of distributed system Centroid of the embodiment of the present application three;
Fig. 4 is the synoptic diagram of type of the register node of the embodiment of the present application three;
Fig. 5 is the composition frame chart of construction device of the distributed system Centroid of the embodiment of the present application four;
Fig. 6 is the supervising device composition frame chart based on the distributed system Centroid of the embodiment of the present application five.
Embodiment
In order to make those skilled in the art person understand better technical scheme among the application, below in conjunction with the accompanying drawing in the embodiment of the present application, technical scheme in the embodiment of the present application is clearly and completely described, obviously, described embodiment only is the application's part embodiment, rather than whole embodiment.Based on the embodiment among the application, those of ordinary skills are not making the every other embodiment that obtains under the creative work prerequisite, all should belong to the scope of the application's protection.
Embodiment one
Referring to Fig. 1, the figure shows the flow process of the distributed system Centroid construction method of the embodiment of the present application.This embodiment comprises:
Step S101: node sends election message to distributed system, and described election message comprises election round and message request number;
By sending the election process of the message initiated distributed system of election, to select Centroid (Leader) managing distributing type system.Node will send the message initiated election process of election to distributed system after being subject to some triggering, the triggering here includes but not limited to: whole nodes of distributed system (such as Zookeeper server) just start, and do not have Centroid; Occur new node in the distributed system and add, this new node is in init state (Looking state); Existing Centroid in the distributed system is because (Leader) inefficacy etc. appears in a variety of causes.After node is triggered, will send election message to all nodes in the distributed system by the election thread of node, to initiate election process.
Each node has the thread of at least three classifications usually: the one, and the election thread is used for the initiation election in distributed system, and finishes the election of this node; The 2nd, send-thread, being used for sending need to be to the various types of messages of other nodes propelling movements, such as notification message, recommendation message etc.; The 3rd, receiving thread is used for receiving the various types of messages that other nodes send in the distributed system.Send-thread and receiving thread are in blocked state when not initiating election, until just solve blocked state and processing messages when having message to arrive.Need to prove: in actual application, different according to the number of nodes in the distributed system and interacting message amount, each node also may arrange transmit queue and receiving queue, the message that send-thread sends may be sent to first transmit queue, the message that receiving thread receives also may be introduced in the receiving queue, then reads message by node from transmit queue and finishes transmit operation or read message and finish relevant treatment operations from receiving queue.
Node be subject to triggering start election process by the election thread after, send election message by send-thread to the node in the distributed system, include in this election message and elect round and message request number.Election round (epoch) is the number of times of self initiating election process of each nodes records, usually automatically increases by 1 at the beginning of election process is initiated, and automatically increases by 1 again after finishing election; Message request number (Zookeeper transtion id, can Zookeeper affairs ID) be the key factor that become Leader in the election process, it determines that current Server throws the ticket (recommendation ticket) of oneself to which node in the distributed system.
Step S102: the magnitude relationship between the election round in the election round of judging the node receive election message and the election message, if elect the election round of message large, execution in step S105 then; Equate with the election round of election message if receive the election round of the node of election message, then execution in step S103;
The node of initiating the election flow process will send election message to the node in the distributed system, include in this election message to elect round and message request number.The election round of node is that the election number of times that keeps after election last time (starts when being in init state if node is firm, the election round is 1, the election round of later every initiation single election self increases by 1), that is to say that each node maintenance has an election wheel sub-value.The node that receives election message will compare the election round of self with electing the election round of carrying in the message, be up-to-date election round to judge which elects round, thereby the election round of self is carried out different processing.The node that the receives election message election round with election message is compared and may produce three kinds of results: the one, the former illustrate that greater than the latter election round that receives the node of electing message is up-to-date election round, need not carry out any operational processes; The 2nd, the latter is greater than the former, and the election round that the node maintenance that receives election message is described is old election wheel sub-value, need to upgrade processing; The 3rd, both equate, also need in this case to carry out the comparison of message request number, to judge whether and should be further processed.
Step S103: the magnitude relationship between the message request in the message request number of judging the node receive election message and the election message number, if the message request of election message number greatly, then enters step S104; If not, then do not carry out any processing, execution in step S106;
In the situation that election round and the election round of election in the message of the node that receives election message equates, also need the size of both message request of comparison number.The election round equates, illustrates that the node that sends election message and the node that receives election message are in same the wheel in the election process, and further relatively message request ability determines who will most possibly be recommended as Centroid.The message request of the node that receives election message and election message number compared may produce two kinds of results: the one, the former is greater than the latter, at this moment without any need for processing; The one, the former is less than the latter, at this moment needs to carry out the renewal of message request number.
Step S104: use the message request of election message number to upgrade the message request number of the node that receives election message, enter step S106;
Step S105: use the election round of election message to upgrade the election round that receives the node of electing message;
Step S106: number form recommendation message to Centroid according to the election round after upgrading and message request, and send recommendation message to distributed system;
After the node of reception election message is finished aforesaid renewal operation, can form recommendation message according to upgrading the result, this recommendation message is that reception elects the node of message to the recommendation of Centroid.According to the rule of distributed system, usually will elect the node of round maximum, message request maximum to be recommended as Centroid.
Step S107: to the recommendation of Centroid, and recommended number is reached node centered by the node determination of predetermined threshold value in the node statistics recommendation message;
Node in the distributed system will be received the recommendation message that other nodes send, and know that according to this recommendation message each node is to the recommendation of Centroid.During specific implementation, the node that receives recommendation message will be put into statistical form to the address of the Centroid of the node recommendation of the address of node that sends recommendation message and transmission recommendation message, then add up the recommended number of each recommended node as Centroid, recommended number is reached the node determination Centroid of predetermined threshold value.The predetermined threshold value here is different according to actual conditions, can set.Normal conditions take the mode of " majority rule " to finish the ballot statistics, and the node of namely being proposed as Centroid obtains (N/2+1) ticket, is about to be set to Centroid, and N is the total number of node that distributed system comprises.After obtaining statistics, except determining Centroid, the node that receives recommendation message is also set the state of oneself accordingly.In the Zookeeper group system, each Server comprises three kinds of possible states: the one, and init state (Looking state), this is the rear state that occurs of Leader inefficacy in just startup of Server or distributed system; The 2nd, leader's state (Leading state), this is the state of the Centroid selected after election is finished; The 3rd, follower's state (Following state), this is that other nodes are set behind the election Centroid in the distributed system the state of self.
The node that the embodiment of the present application will send election message compares with the election round and the message request size that receive the node of electing message, according to the result after relatively more the election round of new node and message request number, generate new notification message after upgrading, other nodes that receive this call message arrange node or non-central node centered by the oneself state according to this message and the situation of self.Compared with prior art, the single-point collapse can not appear in the Centroid that the embodiment of the present application makes up, in network and the unsettled situation of hardware, and safer, more easy-to-use, manageability more.
The form of foregoing by process flow diagram is illustrated the technical scheme of present embodiment, and for more clearly understanding the application, the below transfers the registration of Party membership, etc. from one unit to another angle from the data stream between the different nodes and describes.Start in the election process in distributed formula system, to the node of three types occur: passive receipt message end, initiatively send message end and initiatively initiate the election end, there is difference in the effect that dissimilar nodes is brought into play in the election of finishing Centroid.
For passive receipt message end: at first detect recommended zxid on the current Server, whether epoch is legal, and inspect statement can be: (currentServer.epoch＜=currentMsg.epoch﹠amp; ﹠amp; (currentMsg.zxid＞currentServer.zxid|| (currentMsg.zxid==currentServer.zxid﹠amp; ﹠amp; CurrentMsg.id＞currentServer.id))); If the illegal zxid that just uses in the message, epoch, id upgrade the recommended value of current Server, become Notification message to put into receiving queue the message conversion of receiving this moment, will send acknowledge message to the other side.Acknowledge message numbering is put into the ack formation, and the state that detects the other side is the LOOKING state whether, if not, illustrate has had Leader out selected this moment, the message that receives is transmitted into Notification message put into reception to formation.
For initiatively sending the message end: the message that will send becomes ToSend message by the Notification message conversion, then send the other side, and wait the other side's answer, if method is replied wait for finishing not receive, reform three times, whether detect current election (epoch) when time still not receiving the other side's answer if reform changes, if do not change, message is put into transmit queue again, till being repeated until that Leader selects or receive that the other side replys, mainly own relevant information is sent to the other side.
For initiatively initiating the election end: at first the epoch of oneself adds 1, then generate notification message, and message put into transmit queue, dispose several Server in the system and just generate several message, guarantee that each Server can both receive this message, if the state of current Server be Looking just always the circular test receiving queue whether message is arranged, if message is arranged, process accordingly according to the state of the other side in the message:
If be the Looking state: at first whether epoch is legal in the detect-message, large than current Server whether, if when the epoch of more current Server is large, upgrade epoch, detection is the zxid in the message, and whether id is larger than the Server of current recommendation, if upgrade correlation, and newly-generated notification message puts into and send out to close a formation, empties the ballot statistical form; If the epoch that message is little then do nothing; If zxid in the same detection message, whether id is legal, if the zxid in the message, id is large, upgrade so current Server relevant information, and newly-generated notification message puts into transmit queue, IP and the voting results of the message received are put into statistical form, and the counting statistics result, according to the result own corresponding state is set.
If for the Leading state: IP and the voting results of the message received are put into statistical form (the statistical form here is independently), and the counting statistics result, according to the result own corresponding state is set.
If for the Following state: IP and the voting results of the message received are put into statistical form (the statistical form here is independently), and the counting statistics result, according to the result own corresponding state is set.
Embodiment two
Previous embodiment has proposed a kind of construction method of distributed system Centroid, can construct more stable Centroid by the method, can realize various concrete application based on this Centroid.Referring to Fig. 2, the figure shows a kind of flow process of the request of data commit method based on the distributed system Centroid.This flow process comprises:
Step S201: start counter, the initial value of counter is set to 0;
Step S202: the data request information that non-central node is received is transmitted to Centroid;
Non-central node is in follower (Following) status, can not make a strategic decision to receiving request of data, therefore needs this request is forwarded to Centroid (Leader).The Centroid of present embodiment makes up according to the mode of previous embodiment, and namely described Centroid makes up in the following manner: each node sends election message to distributed system, and described election message comprises election round and message request number; Judge the election round of the node that receives election message and the magnitude relationship between the election round in the election message, if the election round of election message is large, then the election round of use election message is upgraded the election round of the node that receives election message; If receiving the election round of the node of election message equates with the election round of election message, magnitude relationship between the message request number of then judging the node receive election message and the message request number of electing in the message, if the message request number of election message is large, then use the message request of election message number to upgrade the message request number of the node that receives election message; Number form recommendation message to Centroid according to the election round after upgrading and message request, and send recommendation message to distributed system; To the recommendation of Centroid, and recommended number is reached node centered by the Node configuration of predetermined threshold value in each node statistics recommendation message.
Step S203: the data request information that Centroid is received sends to the non-central node in the distributed system;
Step S204: the size between ID number of the ID of the data request information that more non-central node receives number and non-central node self if the ID of data request information number large, then enters step S205;
Step S205: return acknowledge message to Centroid;
Step S206: if the affirmation message that Centroid receives reaches predetermined threshold value, then the data of data request information are synchronized to all non-central nodes of distributed system;
The setting of predetermined threshold value and distributed system require degree relevant to " consistance " of data synchronizing process, adopt the mode of " majority rule " that this threshold values is set in the common situation, if namely to receive the confirmation the quantity of message be whole nodes over half in the distributed system to Centroid, then be considered as these data and submit to request to be allowed to.After obtaining data submission request permission, Centroid will upgrade local data, and realize synchronous to all non-central nodes transmission data.
Present embodiment relies on the Centroid that makes up according to previous embodiment and has guaranteed that the data of a plurality of nodes in the cluster have strong consistency.
Embodiment three
Based on the distributed system Centroid that embodiment one makes up, can also carry out the monitoring nodes operation of distributed node.Referring to Fig. 3, Fig. 4, wherein: Fig. 3 the figure shows the flow process based on the method for supervising of distributed system Centroid, and Fig. 4 shows the synoptic diagram of the type of register node.The flow process of present embodiment comprises:
Step S301: in distributed system, register monitor node, described monitor node config directory and state catalogue;
Config directory and the node under the state catalogue that the registration monitor node comprises can be domain name service node (NameService) and data serving node (DataService), and concrete registration process can realize in the following manner:
For registering NameService:
(1) in distributed system, sets up config directory/Conf/NameService/ (NameServiceID) catalogue, the type of this catalogue is set to the Persistent type, the type catalogue is the persistence directory node, and the data that it is stored above can not lost;
(2) set up state catalogue/Stat/NameService/ (NameServiceID) catalogue, the type of this catalogue is set to the Ephemeral_Sequential type, the type catalogue is the directory node of order automatic numbering, belong to the temp directory node, the catalogue of the type is in case create, when the session (session) of client and server mouth was overtime, this directory node will be deleted automatically.
For registering DataService:
(1) if the Group at this DataService place is newly-built Group, then set up config directory/Conf/DataService/ (GroupID) catalogue, its type is set to the Persistent type;
(2) if the Group at this DataService place is newly-built Group, then set up state catalogue/Stat/DataService/ (GroupID) catalogue, its type is set to the Persistent type;
(3) set up config directory/Conf/DataService/ (GroupID)/(DataServiceID), its type is set to the Persistent type;
(4) set up state catalogue/Stat/DataService/ (GroupID)/(DataServiceID), its type is set to the Ephemeral_Sequential type.
Step S302: the distributed system Centroid receives the message that the node in the distributed system changes, and described Centroid obtains according to the method structure of previous embodiment;
Step S303: compare the config directory of monitor node and the node under the state catalogue:
Step S304: when node is present in config directory and is not present in the state catalogue, send monitoring alarm registration monitor node.
The Centroid that makes up by present embodiment comes the management and monitoring distributed system, because the node beyond monitor node, the monitor node all can carry out mutual with the Zookeeper service, when node state changes, can send monitor prompt, thereby in time monitor the running status of each node of distributed system.
Need to prove: on the present embodiment basis, can also increase the distributed lock function, detailed process can realize in the following manner: registration distributed lock nodal directory in distributed system, such as foundation/locknode/lock catalogue, the type of this catalogue is set to the Ephemeral_Sequential type; Then obtain the node under the distributed lock nodal directory; Judge again the node of node number minimum in the node under the distributed node catalogue, to the node assignment profile formula lock of node number minimum.
Embodiment four
Above content is described the application's embodiment of the method in detail, and correspondingly, the embodiment of the present application also provides the device embodiment corresponding with aforementioned manner embodiment.Referring to Fig. 5, the figure shows the composition frame chart of the construction device of distributed system Centroid.This device 500 comprises: transmitting element 501, the first judging unit 502, the second judging unit 503, updating block 504, formation unit 505, statistic unit 506 and determining unit 507, wherein:
Transmitting element 501 is used for sending election message to distributed system, and described election message comprises election round and message request number;
The first judging unit 502, be used for judging the magnitude relationship between the election round that receives the node of electing message and the election round of the electing message, if the election round of election message is large, then trigger updating block, equate with the election round of election message if receive the election round of the node of election message, then trigger the second judging unit;
Magnitude relationship between the second judging unit 503, message request number that be used for to judge the node that receives election message and the message request of election message number is if the message request of election message number greatly, then triggers updating block;
Updating block 504, be used for when the election round of electing message is large, use the election round of election message to upgrade the election round that receives the node of electing message, when the message request of election message is number large, use the message request of election message number to upgrade the message request number of the node that receives election message;
Determining unit 507 is for node centered by the node determination that recommended number is reached predetermined threshold value.
The course of work of said apparatus embodiment is: the first transmitting element sends the election comprise election round and message request number to distributed system and disappears, receive the election round of the node of electing message and the magnitude relationship between the election round in the election message by the first judgment unit judges, if the election round of election message is large, then triggering updating block uses the election round of election message to upgrade the election round that receives the node of electing message, if receiving the election round of the node of election message equates with the election round of election message, then trigger the second judging unit, received the magnitude relationship between message request number and the message request number of electing in the message of node of election message by the second judgment unit judges, if the message request number of election message is large, then trigger the message request number that message request that updating block uses election message number is upgraded the node that receives election message; Then form the unit and number form recommendation message to Centroid according to the election round of upgrading and message request, and send recommendation message to distributed system; To the recommendation of Centroid, determining unit reaches recommended number node centered by the node determination of predetermined threshold value in the recommendation unit statistics recommendation message.
The node that this device embodiment will send election message compares with the election round and the message request size that receive the node of electing message, according to the result after relatively more the election round of new node and message request number, generate new notification message after upgrading, other nodes that receive this call message arrange node or non-central node centered by the oneself state according to this message and the situation of self.Compared with prior art, the single-point collapse can not appear in the Centroid that the embodiment of the present application makes up, in network and the unsettled situation of hardware, and safer, more easy-to-use, manageability more.
Need to prove: in the election process of finishing a Centroid, the above-mentioned functions unit may not be positioned among the node, but is finished the election of Centroid by the common cooperation of the corresponding unit of a plurality of nodes.
Can carry out various application based on said apparatus embodiment, such as carrying out the data submit operation.Should submit to device to comprise based on request of data of distributed system Centroid: retransmission unit, mass-sending unit, the first comparing unit, to return unit and lock unit, wherein: described retransmission unit is used for that non-central node is received data request information and is transmitted to Centroid; Described mass-sending unit is used for the data request information that Centroid receives is sent to the non-central node of distributed system; Described the first comparing unit is used for the size between ID number of data request information that more non-central node receives with non-central node self ID number, if the ID of data request information number large, then the unit is returned in triggering; The described unit that returns is used for returning acknowledge message to Centroid; Described lock unit is used for the data of data request information being synchronized to all non-central nodes of distributed system when the affirmation message that Centroid receives reaches predetermined threshold value;
The Centroid of mentioning in this above-mentioned application makes up in the following manner:
Each node sends election message to distributed system, and described election message comprises election round and message request number;
Judge the election round of the node that receives election message and the magnitude relationship between the election round in the election message, if the election round of election message is large, then the election round of use election message is upgraded the election round of the node that receives election message; If receiving the election round of the node of election message equates with the election round of election message, magnitude relationship between the message request number of then judging the node receive election message and the message request number of electing in the message, if the message request number of election message is large, then use the message request of election message number to upgrade the message request number of the node that receives election message; Number form recommendation message to Centroid according to the election round after upgrading and message request, and send recommendation message to distributed system;
To the recommendation of Centroid, and recommended number is reached node centered by the node determination of predetermined threshold value in the node statistics recommendation message.
Above-mentioned data submit to the course of work of device to be: after retransmission unit received non-central node data request information and is transmitted to Centroid, the data request information that Centroid is received by the mass-sending unit sent to the non-central node in the distributed system; Then the size between ID number of ID number of the data request information that receives of the more non-central node of comparing unit and non-central node self if the ID of data request information number large, then trigger and returns the unit and return acknowledge message to Centroid; Lock unit when the affirmation message that receives at Centroid reaches predetermined threshold value, is synchronized to the data of data request information all non-central nodes of distributed system.
When the Centroid that makes up by the embodiment of the present application comes the receive data request, to confirm through the Zookeeper service of predetermined number because every secondary data is submitted to all, guarantee preferably the data consistency between many Zookeeper.
Embodiment five
The application also provides a kind of supervising device based on the distributed system Centroid.Referring to Fig. 6, the figure shows the composition frame chart of this supervising device.This device 600 comprises: registering unit 601, the second comparing unit 602 and monitoring alarm unit 603, wherein:
Registering unit 601 is used for registering monitor node in distributed system, and described monitor node comprises and is positioned at config directory and state catalogue;
The second comparing unit 602, be used for after the distributed system Centroid receives the message that the node of distributed system changes the relatively config directory of monitor node and the node under the state catalogue, if node is present in config directory and is not present in the state catalogue, then trigger the monitoring alarm unit, described Centroid obtains according to the method structure of previous embodiment;
Monitoring alarm unit 603 is used for sending monitoring alarm.
The course of work of this device is: registering unit is registered in distributed system and is comprised the domain name service node that is positioned under config directory and the state catalogue and the monitor node of data serving node, then after receiving the change message of node, compare the config directory of monitor node and the node under the state catalogue by comparing unit, if node is present in config directory and is not present in the state catalogue, then trigger the monitoring alarm unit and send monitoring alarm.
The Centroid that makes up by the embodiment of the present application comes the management and monitoring distributed system, because the node beyond monitor node, the monitor node all can carry out mutual with the Zookeeper service, when node state changes, can send monitor prompt, thereby in time monitor the running status of each node of distributed system.
For the convenience of describing, be divided into various unit with function when describing above device and describe respectively.Certainly, when implementing the application, can in same or a plurality of softwares and/or hardware, realize the function of each unit.
As seen through the above description of the embodiments, those skilled in the art can be well understood to the application and can realize by the mode that software adds essential general hardware platform.Based on such understanding, the part that the application's technical scheme contributes to prior art in essence in other words can embody with the form of software product, this computer software product can be stored in the storage medium, such as ROM/RAM, magnetic disc, CD etc., comprise that some instructions are with so that a computer equipment (can be personal computer, server, the perhaps network equipment etc.) carry out the described method of some part of each embodiment of the application or embodiment.
Each embodiment in this instructions all adopts the mode of going forward one by one to describe, and identical similar part is mutually referring to getting final product between each embodiment, and each embodiment stresses is difference with other embodiment.Especially, for system embodiment, because its basic simlarity is in embodiment of the method, so describe fairly simplely, relevant part gets final product referring to the part explanation of embodiment of the method.System embodiment described above only is schematic, wherein said unit as the separating component explanation can or can not be physically to separate also, the parts that show as the unit can be or can not be physical locations also, namely can be positioned at a place, perhaps also can be distributed on a plurality of network element.Can select according to the actual needs wherein some or all of module to realize the purpose of present embodiment scheme.Those of ordinary skills namely can understand and implement in the situation of not paying creative work.
The application can be used in numerous general or special purpose computingasystem environment or the configuration.For example: personal computer, server computer, handheld device or portable set, plate equipment, multicomputer system, the system based on microprocessor, set top box, programmable consumer-elcetronics devices, network PC, small-size computer, mainframe computer, comprise distributed computing environment of above any system or equipment etc.
The application can describe in the general context of the computer executable instructions of being carried out by computing machine, for example program module.Usually, program module comprises the routine carrying out particular task or realize particular abstract data type, program, object, assembly, data structure etc.Also can in distributed computing environment, put into practice the application, in these distributed computing environment, be executed the task by the teleprocessing equipment that is connected by communication network.In distributed computing environment, program module can be arranged in the local and remote computer-readable storage medium that comprises memory device.
The above only is the application's embodiment; should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the application's principle; can also make some improvements and modifications, these improvements and modifications also should be considered as the application's protection domain.
Claims (10)
1. the construction method of a distributed system Centroid is characterized in that, the method comprises:
Node sends election message to distributed system, and described election message comprises election round and message request number;
Judge the election round of the node that receives election message and the magnitude relationship between the election round in the election message, if the election round of election message is large, then the election round of use election message is upgraded the election round of the node that receives election message; If receiving the election round of the node of election message equates with the election round of election message, magnitude relationship between the message request number of then judging the node receive election message and the message request number of electing in the message, if the message request number of election message is large, then use the message request of election message number to upgrade the message request number of the node that receives election message; Number form recommendation message to Centroid according to the election round after upgrading and message request, and send recommendation message to distributed system;
To the recommendation of Centroid, and recommended number is reached node centered by the node determination of predetermined threshold value in the node statistics recommendation message.
2. method according to claim 1 is characterized in that, to the recommendation of Centroid, and node comprises centered by recommended number being reached the node determination of predetermined threshold value in the described node statistics recommendation message:
Statistical form is put into the address of the Centroid of the node recommendation that sends recommendation message in the address of node that sends recommendation message, add up the recommended number of each recommended node as Centroid, recommended number is reached the node determination Centroid of predetermined threshold value.
3. method according to claim 1 is characterized in that, election message sends by the election thread of node; The node that receives election message receives election message by receiving thread, and the node that sends recommendation message sends recommendation message by send-thread.
4. request of data commit method based on the distributed system Centroid is characterized in that the method comprises:
The data request information that non-central node is received is transmitted to Centroid, and described Centroid makes up according to any method in the claims 1 to 3 and obtains;
The data request information that Centroid is received sends to the non-central node in the distributed system;
Size between ID number of the ID of the data request information that more non-central node receives number and non-central node self if the ID of data request information number large, is then returned acknowledge message to Centroid;
If the affirmation message that Centroid receives reaches predetermined threshold value, then the data of data request information are synchronized to all non-central nodes of distributed system.
5. method for supervising based on the distributed system Centroid is characterized in that the method comprises:
Register monitor node in distributed system, described monitor node comprises config directory and state catalogue;
After the distributed system Centroid receives the message that the node in the distributed system changes, compare the config directory of monitor node and the node under the state catalogue, if node is present in config directory and is not present in the state catalogue, then send monitoring alarm, described Centroid makes up according to any method in the claims 1 to 3 and obtains.
6. method according to claim 5 is characterized in that, described method also comprises:
Registration reported data catalogue in distributed system is so that node is set up new warning node after changing under this reported data catalogue;
After the distributed system Centroid receives the message that the node in the distributed system changes, obtain the node under the reported data catalogue, send monitoring alarm according to the node under the catalogue that reports that obtains.
7. method according to claim 5 is characterized in that, described method also comprises:
Registration distributed lock nodal directory in distributed system;
Obtain the node under the distributed lock nodal directory;
Judge the node of node number minimum in the node under the distributed node catalogue, to the node assignment profile formula lock of node number minimum.
8. the construction device of a distributed system Centroid is characterized in that, this device comprises: transmitting element, the first judging unit, the second judging unit, updating block, formation unit, statistic unit and determining unit, wherein:
Described transmitting element is used for sending election message to distributed system, and described election message comprises election round and message request number;
Described the first judging unit, be used for judging the magnitude relationship between the election round that receives the node of electing message and the election round of the electing message, if the election round of election message is large, then trigger updating block, equate with the election round of election message if receive the election round of the node of election message, then trigger the second judging unit;
Magnitude relationship between described the second judging unit, message request number that be used for to judge the node that receives election message and the message request of election message number is if the message request of election message number greatly, then triggers updating block;
Described updating block, be used for when the election round of electing message is large, use the election round of election message to upgrade the election round that receives the node of electing message, when the message request of election message is number large, use the message request of election message number to upgrade the message request number of the node that receives election message;
Described formation unit is used for number forming recommendation message to Centroid according to the election round after upgrading and message request, and sends recommendation message to distributed system;
Described recommendation unit is used for the statistics recommendation message to the recommendation of Centroid;
Described determining unit is for node centered by the node determination that recommended number is reached predetermined threshold value.
9. submit device to based on the request of data of distributed system Centroid for one kind, it is characterized in that this device comprises: retransmission unit, mass-sending unit, the first comparing unit, return unit and lock unit, wherein:
Described retransmission unit is used for that non-central node is received data request information and is transmitted to Centroid, and described Centroid makes up according to any method in the claims 1 to 3 and obtains;
Described mass-sending unit is used for the data request information that Centroid receives is sent to the non-central node of distributed system;
Described the first comparing unit is used for the size between ID number of data request information that more non-central node receives with non-central node self ID number, if the ID of data request information number large, then the unit is returned in triggering;
The described unit that returns is used for returning acknowledge message to Centroid;
Described lock unit is used for the data of data request information being synchronized to all non-central nodes of distributed system when the affirmation message that Centroid receives reaches predetermined threshold value.
10. the supervising device based on the distributed system Centroid is characterized in that, this device comprises: registering unit, the second comparing unit and monitoring alarm unit, wherein:
Described registering unit is used for registering monitor node in distributed system, and described monitor node comprises config directory and state catalogue;
Described the second comparing unit, be used for after the distributed system Centroid receives the message that the node of distributed system changes the relatively config directory of monitor node and the node under the state catalogue, if node is present in config directory and is not present in the state catalogue, then trigger the monitoring alarm unit; Described Centroid makes up according to any method in the claims 1 to 3 and obtains;
Described monitoring alarm unit is used for sending monitoring alarm.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210371063.9A CN102929696B (en) | 2012-09-28 | 2012-09-28 | A kind of distributed system Centroid structure, submission, method for supervising and device |
| CN201510150055.5A CN104731648B (en) | 2012-09-28 | 2012-09-28 | A kind of distributed system Centroid structure, submission, monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210371063.9A CN102929696B (en) | 2012-09-28 | 2012-09-28 | A kind of distributed system Centroid structure, submission, method for supervising and device |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510150055.5A Division CN104731648B (en) | 2012-09-28 | 2012-09-28 | A kind of distributed system Centroid structure, submission, monitoring method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102929696A true CN102929696A (en) | 2013-02-13 |
| CN102929696B CN102929696B (en) | 2015-09-09 |
Family
ID=47644505
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210371063.9A Active CN102929696B (en) | 2012-09-28 | 2012-09-28 | A kind of distributed system Centroid structure, submission, method for supervising and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102929696B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103188110A (en) * | 2013-03-29 | 2013-07-03 | 汉柏科技有限公司 | Zookeeper-based cloud platform service hot backup method |
| CN104468163A (en) * | 2013-09-18 | 2015-03-25 | 腾讯科技（北京）有限公司 | Method and device for disaster tolerant network construction, and disaster tolerant network |
| CN104753994A (en) * | 2013-12-27 | 2015-07-01 | 杭州海康威视***技术有限公司 | Method and device for data synchronization based on cluster server system |
| CN105306583A (en) * | 2015-11-11 | 2016-02-03 | 广州新科佳都科技有限公司 | Distributed permission management method and system based on atomic broadcast technology |
| CN106470230A (en) * | 2015-08-19 | 2017-03-01 | 互联网域名***北京市工程研究中心有限公司 | Distributed DNS node data consistency synchronization method and apparatus |
| CN108089916A (en) * | 2017-12-15 | 2018-05-29 | 北京搜狐新媒体信息技术有限公司 | A kind of method and device for generating ID |
| CN108156203A (en) * | 2016-12-06 | 2018-06-12 | 杭州海康威视数字技术股份有限公司 | A kind of storage system and storage node management method |
| CN108183971A (en) * | 2015-03-13 | 2018-06-19 | 聚好看科技股份有限公司 | A kind of node electoral machinery in distributed system |
| CN108874531A (en) * | 2017-05-12 | 2018-11-23 | 北京京东尚科信息技术有限公司 | Method, apparatus, system and electronic equipment for the service of fusing |
| CN109032854A (en) * | 2018-07-13 | 2018-12-18 | 新华三技术有限公司成都分公司 | Elect request processing method, device, management node and storage medium |
| CN109995835A (en) * | 2017-12-29 | 2019-07-09 | 浙江宇视科技有限公司 | Host node electoral machinery, device and distributed memory system |
| CN110764690A (en) * | 2018-07-28 | 2020-02-07 | 阿里巴巴集团控股有限公司 | Distributed storage system and leader node election method and device thereof |
| CN112637268A (en) * | 2020-11-30 | 2021-04-09 | 广州大学 | Radio astronomical data storage consistency guarantee method, device, system and storage medium |
| CN112860790A (en) * | 2021-01-14 | 2021-05-28 | 华控清交信息科技(北京)有限公司 | Data management method, system and device |
| CN113641692A (en) * | 2021-08-18 | 2021-11-12 | 福建天晴数码有限公司 | Scheme and system for realizing distributed cluster node participation |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101035021A (en) * | 2007-02-08 | 2007-09-12 | 北京航空航天大学 | Stable energy-saving group maintenance method |
| CN101251797A (en) * | 2008-03-31 | 2008-08-27 | 中国船舶重工集团公司第七〇九研究所 | Element real time initiative transferring method based on domain model |
| CN101753610A (en) * | 2008-12-19 | 2010-06-23 | 华为技术有限公司 | Distributed network structure method, device, system and task processing method |
-
2012
- 2012-09-28 CN CN201210371063.9A patent/CN102929696B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101035021A (en) * | 2007-02-08 | 2007-09-12 | 北京航空航天大学 | Stable energy-saving group maintenance method |
| CN101251797A (en) * | 2008-03-31 | 2008-08-27 | 中国船舶重工集团公司第七〇九研究所 | Element real time initiative transferring method based on domain model |
| CN101753610A (en) * | 2008-12-19 | 2010-06-23 | 华为技术有限公司 | Distributed network structure method, device, system and task processing method |
Non-Patent Citations (1)
| Title |
|---|
| BECKER，DIOGO, FLAVIO JUNQUEIRA, AND MARCO SERAFINI: "Leader Election for Replicated Services Using Application Scores", 《MIDDLEWARE 2011: ACM/IFIP/USENIX 12TH INTERNATIONAL MIDDLEWARE CONFERENCE》, 16 December 2011 (2011-12-16) * |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103188110B (en) * | 2013-03-29 | 2016-01-06 | 汉柏科技有限公司 | Based on the heat backup method that the cloud platform of zookeeper is served |
| CN103188110A (en) * | 2013-03-29 | 2013-07-03 | 汉柏科技有限公司 | Zookeeper-based cloud platform service hot backup method |
| CN104468163A (en) * | 2013-09-18 | 2015-03-25 | 腾讯科技（北京）有限公司 | Method and device for disaster tolerant network construction, and disaster tolerant network |
| CN104468163B (en) * | 2013-09-18 | 2018-11-09 | 腾讯科技（北京）有限公司 | The method, apparatus and disaster tolerance network of disaster tolerance network organizing |
| CN104753994B (en) * | 2013-12-27 | 2019-04-02 | 杭州海康威视***技术有限公司 | Method of data synchronization and its device based on aggregated server system |
| CN104753994A (en) * | 2013-12-27 | 2015-07-01 | 杭州海康威视***技术有限公司 | Method and device for data synchronization based on cluster server system |
| CN108183971B (en) * | 2015-03-13 | 2021-08-10 | 聚好看科技股份有限公司 | Node election method in distributed system |
| CN108183971A (en) * | 2015-03-13 | 2018-06-19 | 聚好看科技股份有限公司 | A kind of node electoral machinery in distributed system |
| CN106470230A (en) * | 2015-08-19 | 2017-03-01 | 互联网域名***北京市工程研究中心有限公司 | Distributed DNS node data consistency synchronization method and apparatus |
| CN105306583A (en) * | 2015-11-11 | 2016-02-03 | 广州新科佳都科技有限公司 | Distributed permission management method and system based on atomic broadcast technology |
| CN108156203B (en) * | 2016-12-06 | 2021-01-08 | 杭州海康威视数字技术股份有限公司 | Storage system and storage node management method |
| CN108156203A (en) * | 2016-12-06 | 2018-06-12 | 杭州海康威视数字技术股份有限公司 | A kind of storage system and storage node management method |
| CN108874531A (en) * | 2017-05-12 | 2018-11-23 | 北京京东尚科信息技术有限公司 | Method, apparatus, system and electronic equipment for the service of fusing |
| CN108089916A (en) * | 2017-12-15 | 2018-05-29 | 北京搜狐新媒体信息技术有限公司 | A kind of method and device for generating ID |
| CN109995835A (en) * | 2017-12-29 | 2019-07-09 | 浙江宇视科技有限公司 | Host node electoral machinery, device and distributed memory system |
| CN109032854A (en) * | 2018-07-13 | 2018-12-18 | 新华三技术有限公司成都分公司 | Elect request processing method, device, management node and storage medium |
| CN110764690A (en) * | 2018-07-28 | 2020-02-07 | 阿里巴巴集团控股有限公司 | Distributed storage system and leader node election method and device thereof |
| CN110764690B (en) * | 2018-07-28 | 2023-04-14 | 阿里云计算有限公司 | Distributed storage system and leader node election method and device thereof |
| CN112637268A (en) * | 2020-11-30 | 2021-04-09 | 广州大学 | Radio astronomical data storage consistency guarantee method, device, system and storage medium |
| CN112637268B (en) * | 2020-11-30 | 2022-03-18 | 广州大学 | Radio astronomical data storage consistency guarantee method, device, system and storage medium |
| CN112860790A (en) * | 2021-01-14 | 2021-05-28 | 华控清交信息科技(北京)有限公司 | Data management method, system and device |
| CN112860790B (en) * | 2021-01-14 | 2023-05-30 | 华控清交信息科技(北京)有限公司 | Data management method, system and device |
| CN113641692A (en) * | 2021-08-18 | 2021-11-12 | 福建天晴数码有限公司 | Scheme and system for realizing distributed cluster node participation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102929696B (en) | 2015-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102929696B (en) | A kind of distributed system Centroid structure, submission, method for supervising and device | |
| CN110311899A (en) | Multiservice system access method, device and server | |
| CN106412113B (en) | A kind of energy cloud service system and its communication means | |
| CN102075369B (en) | Method and equipment for managing monitoring equipment | |
| CN105744555B (en) | A kind of terminal maintenance method, maintenance device and NM server | |
| CN109672582A (en) | Complete trails monitoring method, equipment, storage medium and device | |
| CN103984887A (en) | Method and device for controlling user permission | |
| US8930532B2 (en) | Session management in a thin client system for effective use of the client environment | |
| CN112905364A (en) | Calling method of micro service and electronic equipment | |
| CN108306787A (en) | A kind of application monitoring method and relevant device | |
| CN104184603B (en) | A kind of User Status statistical method, apparatus and system | |
| CN108134712B (en) | Distributed cluster split brain processing method, device and equipment | |
| CN102025728A (en) | Scheduling method under client-side/server-side architecture and server | |
| CN113162971B (en) | Block link point management method, device, computer and readable storage medium | |
| CN104731648B (en) | A kind of distributed system Centroid structure, submission, monitoring method and device | |
| JP2011107754A (en) | Plant-monitoring control system | |
| CN104980483B (en) | A kind of data transfering method, server and data transferring system | |
| CN105959292A (en) | Device using authority identification method and system, and management server | |
| CN101136756A (en) | Electric self-checking method, system and BMC chip on network long-range control host machine | |
| CN106254229A (en) | A kind of medical communication gateway | |
| CN103326892B (en) | The operating method and device of web interface | |
| CN106844073B (en) | Method for diagnosing application, diagnosis client and system | |
| CN111327447A (en) | Distributed system, arbitration method, node device and storage medium | |
| KR101790746B1 (en) | Method for providing message transaction service | |
| CN106230778A (en) | A kind of tele-control system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder |
Address after: 100086 12, 1201, 3 building, 2 South Road, Haidian District Academy of Sciences, Beijing. Patentee after: Beijing Sohu New Media Information Technology Co., Ltd. Address before: 100084 Beijing Haidian District Zhongguancun East Road 1 hospital 9 building Sohu cyber Building 8 floor 802 room. Patentee before: Beijing Sohu New Media Information Technology Co., Ltd. |