CN105912412A - Message distribution method, message distribution device and message distribution system - Google Patents
Message distribution method, message distribution device and message distribution system Download PDFInfo
- Publication number
- CN105912412A CN105912412A CN201610201563.6A CN201610201563A CN105912412A CN 105912412 A CN105912412 A CN 105912412A CN 201610201563 A CN201610201563 A CN 201610201563A CN 105912412 A CN105912412 A CN 105912412A
- Authority
- CN
- China
- Prior art keywords
- message
- information
- consumption end
- partition
- network condition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/541—Client-server
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a message distribution method, a message distribution device and a message distribution system. The message distribution method comprises the following steps: periodically collecting performance information and network condition information of multiple consumers; getting the related information of at least one message partition in a message queue, wherein each message partition in the message queue contains a group of messages; and assigning the at least one message partition to the consumers for processing according to the performance information and network condition information of the multiple consumers. According to the invention, message partitions can be assigned to consumers flexibly and reasonably, so that the consumers can process the messages in a message queue in parallel; the processing capacity of the consumers is taken into consideration, and the defect that messages cannot be processed in time due to poor processing capacity of the consumers is avoided; and the loads of the consumers are balanced, the overall throughput of the consumers is improved, and optimal allocation of message partitions is realized.
Description
Technical field
The present invention relates to Internet technical field, be specifically related to a kind of message distributing method, Apparatus and system.
Background technology
Currently for message produced by the Producer of business front end, it is common that by pending message storage
In the message partition of message queue, the consumption end of rear end is then transferred to process.In message partition
Distribution aspect, it is common that the message partition in message queue is averagely allocated to consume end termly, not
In view of the consumption performance difference of end and the quality of network condition, thus easily cause poor-performing or
The consumption end that person's network condition is bad, it can not dispose corresponding message in time, and performance good or
Person's network condition good consumption end after having processed the message distributing to it, also do not update at message queue
In the case of, these consumption ends can only wait assignment messages subregion again, thus causes consuming end treatment effeciency
Low, handling capacity is low.
Summary of the invention
In view of the above problems, it is proposed that the present invention is to provide one to overcome the problems referred to above or at least partly
Ground solves the message distributing method of the problems referred to above, message distribution device and corresponding message distributing system.
According to an aspect of the invention, it is provided a kind of message distributing method, comprising:
Periodically collect performance information and the network condition information of multiple consumption ends；
Obtaining the relevant information of at least one message partition in message queue, in message queue, each message is divided
District comprises one group of message；
Performance information according to multiple consumption ends and network condition information, distribute at least one message partition
Process to each consumption end.
According to a further aspect in the invention, it is provided that a kind of message distribution device, comprising:
Collection module, is suitable to periodically collect performance information and the network condition information of multiple consumption end；
First acquisition module, is suitable to obtain the relevant information of at least one message partition in message queue, disappears
In breath queue, each message partition comprises one group of message；
Distribution module, is suitable to the performance information according to multiple consumption ends and network condition information, by least one
Individual message partition is distributed to each consumption end and is processed.
According to a further aspect in the invention, it is provided that a kind of message distributing system, including above-mentioned message distribution
Device, third party's module, message queue and multiple consumption end.
The scheme provided according to the present invention, according to performance information and the network condition information of multiple consumption ends,
Distribute at least one message partition to each consumption end, for each consumption end, at least one received is disappeared
Breath subregion processes, it is achieved that flexibly, reasonably for consuming end assignment messages subregion, and make consumption
End can message in parallel processed messages queue；And combine the disposal ability of consumption end, thus keep away
Exempt from the defect causing message not processed in time because of consumption end disposal ability difference, consumption end has been born
It is loaded into having gone equilibrium treatment, improves the handling capacity that consumption end is overall, it is achieved that the optimal sorting of message partition
Join.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the skill of the present invention
Art means, and can being practiced according to the content of description, and in order to allow, the present invention's is above and other
Objects, features and advantages can become apparent, below especially exemplified by the detailed description of the invention of the present invention.
Accompanying drawing explanation
By reading the detailed description of hereafter preferred implementation, various other advantage and benefit for this
Field those of ordinary skill will be clear from understanding.Accompanying drawing is only used for illustrating the purpose of preferred implementation,
And it is not considered as limitation of the present invention.And in whole accompanying drawing, be denoted by the same reference numerals
Identical parts.In the accompanying drawings:
Fig. 1 shows the schematic flow sheet of message distributing method according to an embodiment of the invention；
Fig. 2 shows the schematic flow sheet of message distributing method in accordance with another embodiment of the present invention；
Fig. 3 shows the schematic flow sheet of message distributing method in accordance with another embodiment of the present invention；
Fig. 4 shows the structured flowchart of message distribution device according to an embodiment of the invention；
Fig. 5 shows the structured flowchart of message distribution device in accordance with another embodiment of the present invention；
Fig. 6 shows the structured flowchart of message distribution device in accordance with another embodiment of the present invention；
Fig. 7 shows the structured flowchart of message distributing system according to an embodiment of the invention.
Detailed description of the invention
It is more fully described the exemplary embodiment of the disclosure below with reference to accompanying drawings.Although accompanying drawing shows
The exemplary embodiment of the disclosure, it being understood, however, that may be realized in various forms the disclosure and not
Should be limited by embodiments set forth here.On the contrary, it is provided that these embodiments are able to more thoroughly
Understand the disclosure, and complete for the scope of the present disclosure can be conveyed to those skilled in the art.
Fig. 1 shows the schematic flow sheet of message distributing method according to an embodiment of the invention.Such as figure
Shown in 1, the method comprises the following steps:
Step S100, periodically collects performance information and the network condition information of multiple consumption ends.
Wherein, consumption end can be used for processing the message in message partition, the performance information of consumption end
Reflect the consumption end disposal ability to message, the network condition message reflection quality of current network, net
The good bad influence consumption end disposal ability to message of network.
Step S101, obtains the relevant information of at least one message partition in message queue.
Wherein, message queue is for storing pending message, and message queue comprises multiple message partition,
Message is stored in message partition, say, that in message queue, each message partition comprises one group and disappears
Breath.
Additionally, the relevant information of message partition may include that the mark of message partition, can according to this mark
So that multiple message partition are made a distinction.
Step S102, according to performance information and the network condition information of multiple consumption ends, disappears at least one
Breath subregion is distributed to each consumption end and is processed.
Specifically, according to performance information and the network condition letter of each consumption end collected by step S100
Breath, distributes to each consumption end by least one accessed message partition, for each consumption end docking
At least one message partition received processes.
The method provided according to the above embodiment of the present invention, according to performance information and the network of multiple consumption ends
Condition information, distribute at least one message partition to each consumption end, for each consumption end to receiving
At least one message partition processes, it is achieved that flexibly, reasonably for consuming end assignment messages subregion,
And make to consume end can message in parallel processed messages queue；And combine the process energy of consumption end
Power, thus avoid the defect causing message not processed in time because of consumption end disposal ability difference, right
The load of consumption end has carried out equilibrium treatment, improves the handling capacity that consumption end is overall, it is achieved that message is divided
The optimum allocation in district.
Fig. 2 shows the schematic flow sheet of message distributing method in accordance with another embodiment of the present invention.?
In the present embodiment, as a example by third party's module proactive notification message queue information, as in figure 2 it is shown, the party
Method comprises the following steps:
Step S200, sends data packet request to each consumption end, receives comprising of each consumption end return
Each response package of performance information.
Wherein, performance information comprises: CPU occupied information, memory occupation information, I/O occupied information,
And/or data throughout；The performance information of consumption end reflects the consumption end disposal ability to message, network
Condition information reflects the quality of current network, the good bad influence consumption end disposal ability to message of network.
Specifically, the performance information of consumption end can be obtained by third party's module, and pass through third party
Module determines the network condition of consumption end, and third party's module can be zookeeper assembly here.
Third party's module to each consumption end send data packet request, this data packet request contain needs from
The information that each consumption end obtains, such as, the performance information of each consumption end, each consumption end is receiving
After this data packet request, the packet including respective performance information is returned to third party's module, the
Tripartite's module extracts the performance information of each consumption end from the packet that each consumption end returns.
Step S201, speed or time according to sending data packet request with receive response package determine net
Network condition information.
Specifically, each consumption end is after receiving the data packet request that third party's module sends, it will to
Third party's module returns and receives successfully notice, according to this notice, third party's module may determine that packet please
Ask and be successfully received, so that it is determined that send speed or the time of data packet request；Third party's module receives each
The response package comprising respective performance information that individual consumption end returns, and determine reception response package
Speed or time, so that it is determined that network condition information.
Step S202, receives the notice of third party's module transmission to obtain the relevant of at least one message partition
Information.
Wherein, the relevant information of message partition may include that the mark of message partition, can according to this mark
So that multiple message partition are made a distinction.
Specifically, if the message partition in message queue changes, third party's module will send message
The notice that subregion changes, after receiving this notice, can obtain at least one changed and disappear
The relevant information of breath subregion such that it is able to distribute to the message in message partition in time consume at end
Reason, improves Message Processing efficiency.
Step S203, according to performance information and the network condition information of multiple consumption ends, determines that each is consumed
The number of the message partition that can consume of end.
Specifically, in step S200 and step S201, determine performance information and the net of multiple consumption end
Network condition information, i.e. determine the disposal ability of each consumption end, will disappear according to each in this step
The disposal ability of Fei Duan, determines that each consumes the number of the message partition that end can be consumed, i.e. root
According to the disposal ability of each consumption end, load is carried out equilibrium treatment, thus has avoided at because of consumption end
The defect managing ability and cause message not processed in time, it is achieved that flexibly, reasonably for consumption end
Assignment messages subregion.
Step S204, distributes to the message partition of corresponding number each consumption end and processes.
Specifically, the message partition responding number determined by step S203 is distributed to each consumption
End, so that each consumption end processes.
In a preferred embodiment of the invention, it is also possible to according to performance information and the network condition of multiple consumption ends
Information, determines the weight of the message partition that can consume of each consumption end；According to described weight by described
At least one message partition is distributed to each consumption end and is processed.
The method provided according to the above embodiment of the present invention, obtains at least one according to the notice of third party's module
The relevant information of individual message partition such that it is able to the message in message partition is distributed to consume end in time and enters
Row processes, and improves Message Processing efficiency, according to performance information and the network condition information of multiple consumption ends,
Determine the number of the message partition that can consume of each consumption end, the message partition of corresponding number is distributed
Process to each consumption end, i.e. according to the disposal ability of each consumption end, thus avoid because disappearing
Expense end disposal ability difference and cause the defect that message can not be processed in time, and make to consume end can be parallel
Process the message in message queue, it is achieved that flexibly, reasonably for consumption end assignment messages subregion, raising
The handling capacity that consumption end is overall, it is achieved that the optimum allocation of message partition.
Fig. 3 shows the schematic flow sheet of message distributing method in accordance with another embodiment of the present invention.As
Shown in Fig. 3, the method comprises the following steps:
Step S300, obtains multiple consumption end IP address of registration in third party's module in advance.
Specifically, in advance by the IP address registration of multiple consumption ends to third party's module, when needs are collected
When the performance information of multiple consumption ends and network condition information, need first to obtain multiple consumption end in advance
The IP address of registration in tripartite's module, can accurately collect the property of each consumption end according to this IP address
Can information and network condition information.Wherein, third party's module can be zookeeper assembly.
Step S301, sends data packet request according to IP address to each consumption end, receives each consumption
The response package comprising respective performance information that end returns.
According to IP address to each consumption end send data packet request, this data packet request contain needs from
The information that each consumption end obtains, such as, the performance information of each consumption end, wherein, performance information bag
Contain: CPU occupied information, memory occupation information, I/O occupied information and/or data throughout；Each
The packet including respective performance information, after receiving this data packet request, is returned by consumption end, from
The packet that each consumption end returns extracts the performance information of each consumption end.
Step S302, speed or time according to sending data packet request with receive response package determine net
Network condition information.
Specifically, return, after receiving sent data packet request, is received successfully by each consumption end
Notice, may determine that data packet request is successfully received according to this notice, so that it is determined that send packet
The speed of request or time；According to receiving the packet comprising respective performance information that each consumption end returns
Response, and determine speed or the time receiving response package, so that it is determined that network condition information.
Step S303, the subscribing mechanism provided by third party's module subscribes to the related announcement of message queue.
Specifically, the subscribing mechanism utilizing third party's module to provide can know the message of message queue in time
Whether subregion changes, and when a change, the subscribing mechanism provided by third party's module then can
Subscribe to the related announcement of message queue, such as, can subscribe to have increased which subregion newly, or which disappears
Breath subregion there occurs renewal.
Step S304, when at least one message partition in message queue occurs to update, receives third party
The subscribing notification that module sends is to obtain the relevant information of at least one message partition.
Specifically, when at least one message partition in message queue occurs to update, third party's module will
The notice that message partition changes can be sent, after receiving this notice, can obtain and to change
The relevant information of at least one message partition such that it is able to distribute to disappear to the message in message partition in time
Fei Duan process, improves Message Processing efficiency.
Step S305, according to performance information and the network condition information of multiple consumption ends, determines that each is consumed
The weight of the message partition that can consume of end.
Specifically, the weight of the message partition that can consume of each consumption end reflects each consumption end
Disposal ability, and the number of message partition can be processed.
Performance information and the network condition letter of multiple consumption end is determined in step S301 and step S302
Breath, i.e. determine the disposal ability of each consumption end, in this step by the place according to each consumption end
Reason ability, determines the weight of the message partition that can consume of each consumption end, i.e. consume according to each
The disposal ability of end, has carried out equilibrium treatment, thus has avoided because of consumption end disposal ability difference load
Cause the defect that message can not be processed in time, it is achieved that divide flexibly, reasonably for consumption end assignment messages
District.
Step S306, distributes at least one message partition each consumption end according to weight and processes.
Specifically, according to weight determined by step S305, at least one message partition is distributed to each disappear
Fei Duan, processes for each consumption end.
In a preferred embodiment of the invention, it is also possible to according to performance information and the network condition of multiple consumption ends
Information, determines the number of the message partition that can consume of each consumption end；The message of corresponding number is divided
Distinguish each consumption end of dispensing to process.
The method provided according to the above embodiment of the present invention, can obtain multiple consumption end in advance third party
The IP address of registration in module, can accurately collect the performance letter of each consumption end according to this IP address
Breath and network condition information, the subscribing mechanism provided by third party's module subscribes to the relevant logical of message queue
Know such that it is able to the message in message partition is distributed to consume end in time and processes, improve message
Treatment effeciency, according to performance information and the network condition information of multiple consumption ends, determines each consumption end
The number of the message partition that can consume, distributes to the message partition of corresponding number each consumption end and carries out
Process, i.e. according to the disposal ability of each consumption end, load has been carried out equilibrium treatment, thus has avoided
Cause the defect that message can not be processed in time because consumption end disposal ability difference, and make to consume end energy
Message in enough parallel processed messages queues, it is achieved that flexibly, reasonably for consuming end assignment messages subregion,
Improve the handling capacity that consumption end is overall, it is achieved that the optimum allocation of message partition.
Fig. 4 shows the structured flowchart of message distribution device according to an embodiment of the invention.Such as Fig. 4
Shown in, this device 400 includes: collection module the 410, first acquisition module 420 and distribution module 430.
Collection module 410, is suitable to periodically collect the performance information of multiple consumption end and network condition letter
Breath.
First acquisition module 420, is suitable to obtain the relevant information of at least one message partition in message queue,
In message queue, each message partition comprises one group of message.
Distribution module 430, is suitable to the performance information according to multiple consumption ends and network condition information, near
A few message partition is distributed to each consumption end and is processed.
The device provided according to the above embodiment of the present invention, according to performance information and the network of multiple consumption ends
Condition information, distribute at least one message partition to each consumption end, for each consumption end to receiving
At least one message partition processes, it is achieved that flexibly, reasonably for consuming end assignment messages subregion,
And make to consume end can message in parallel processed messages queue；And combine the process energy of consumption end
Power, thus avoid the defect causing message not processed in time because of consumption end disposal ability difference, right
The load of consumption end has carried out equilibrium treatment, improves the handling capacity that consumption end is overall, it is achieved that message is divided
The optimum allocation in district.
Fig. 5 shows the structured flowchart of message distribution device in accordance with another embodiment of the present invention.Such as figure
Shown in 5, this device 500 includes: collection module the 510, first acquisition module 520 and distribution module 530.
Collection module 510, is suitable to periodically collect the performance information of multiple consumption end and network condition letter
Breath.
Wherein, performance information comprises: CPU occupied information, memory occupation information, I/O occupied information,
And/or data throughout.
First acquisition module 520, is suitable to obtain the relevant information of at least one message partition in message queue,
In message queue, each message partition comprises one group of message.
Distribution module 530, is suitable to the performance information according to multiple consumption ends and network condition information, near
A few message partition is distributed to each consumption end and is processed.
Alternatively, the first acquisition module 520 is further adapted for: receive third party's module send notice with
Obtain the relevant information of at least one message partition.
Alternatively, collection module 510 is further adapted for: receive each consumption end that third party's module sends
Performance information and network condition information；Wherein, wherein, the performance information of each consumption end is third party
Module to each consumption end send data packet request, and receive each consumption end return comprise respective performance
The response package of information obtains, and network condition information is that third party's module is according to sending data packet request
With receive the speed of response package or the time determines.
Alternatively, distribution module 530 farther includes: second determines unit 531 and Dispatching Unit 532.
Second determines unit 531, is suitable to the performance information according to multiple consumption ends and network condition information,
Determine the number of the message partition that can consume of each consumption end；
Dispatching Unit 532, is suitable to that the message partition of corresponding number is distributed to each consumption end and processes.
The device provided according to the above embodiment of the present invention, obtains at least one according to the notice of third party's module
The relevant information of individual message partition such that it is able to the message in message partition is distributed to consume end in time and enters
Row processes, and improves Message Processing efficiency, according to performance information and the network condition information of multiple consumption ends,
Determine the number of the message partition that can consume of each consumption end, the message partition of corresponding number is distributed
Process to each consumption end, i.e. according to the disposal ability of each consumption end, load has been carried out all
Weighing apparatus processes, thus avoids the defect causing message not processed in time because of consumption end disposal ability difference,
And make to consume end can message in parallel processed messages queue, it is achieved that flexibly, reasonably for consumption
End assignment messages subregion, improves the handling capacity that consumption end is overall, it is achieved that the optimum allocation of message partition.
Fig. 6 shows the structured flowchart of message distribution device in accordance with another embodiment of the present invention.Such as figure
Shown in 6, this device 600 includes: collection module the 610, first acquisition module 620 and distribution module 630.
Collection module 610, is suitable to periodically collect the performance information of multiple consumption end and network condition letter
Breath.
Wherein, performance information comprises: CPU occupied information, memory occupation information, I/O occupied information,
And/or data throughout.
First acquisition module 620, is suitable to obtain the relevant information of at least one message partition in message queue,
In message queue, each message partition comprises one group of message.
Distribution module 630, is suitable to the performance information according to multiple consumption ends and network condition information, near
A few message partition is distributed to each consumption end and is processed.
Alternatively, this device also includes: subscribing module 640, is suitable to by ordering that third party's module provides
The mechanism of readding subscribes to the related announcement of message queue.
First acquisition module 620 is further adapted for: when at least one message partition in message queue occurs
During renewal, receive the subscribing notification of third party's module transmission to obtain the relevant letter of at least one message partition
Breath.
Alternatively, this device also includes: the second acquisition module 650, is suitable to obtain multiple consumption end in advance
The IP address of registration in third party's module；
Collection module 610 farther includes: transmitting element 611, reception unit 612 and first determine list
Unit 613.
Transmitting element 611, is suitable to send data packet request according to IP address to each consumption end；
Receive unit 612, be suitable to receive the packet comprising respective performance information that each consumption end returns
Response；
First determines unit 613, is suitable to according to sending data packet request and receiving the speed of response package
Or the time determines network condition information.
Alternatively, distribution module 630 farther includes: second determines unit 631 and Dispatching Unit 632.
Second determines unit 631, is suitable to the performance information according to multiple consumption ends and network condition information,
Determine the weight of the message partition that can consume of each consumption end；
Dispatching Unit 632, is suitable to, according to weight, at least one message partition is distributed to each consumption end and enters
Row processes.
The device provided according to the above embodiment of the present invention, obtains multiple consumption end in advance in third party's module
The IP address of middle registration, according to this IP address can accurately collect each consumption end performance information and
Network condition information, by the related announcement of the subscribing mechanism subscription message queue that third party's module provides,
Process it is thus possible to the message in message partition to be distributed to consume end in time, improve Message Processing
Efficiency, according to performance information and the network condition information of multiple consumption ends, determine each consumption end can
The number of the message partition of consumption, distributes to the message partition of corresponding number each consumption end and processes,
That is, according to the disposal ability of each consumption end, load has been carried out equilibrium treatment, thus has avoided because disappearing
Expense end disposal ability difference and cause the defect that message can not be processed in time, and make to consume end can be parallel
Process the message in message queue, it is achieved that flexibly, reasonably for consumption end assignment messages subregion, raising
The handling capacity that consumption end is overall, it is achieved that the optimum allocation of message partition.
Fig. 7 shows the structured flowchart of message distributing system according to an embodiment of the invention.Such as Fig. 7
Shown in, this system 700 includes: message distribution device 600, third party's module 710, message queue 720
With multiple consumption ends 730.
The system provided according to the above embodiment of the present invention, obtains multiple consumption end in advance in third party's module
The IP address of middle registration, according to this IP address can accurately collect each consumption end performance information and
Network condition information, by the related announcement of the subscribing mechanism subscription message queue that third party's module provides,
Process it is thus possible to the message in message partition to be distributed to consume end in time, improve Message Processing
Efficiency, according to performance information and the network condition information of multiple consumption ends, determine each consumption end can
The number of the message partition of consumption, distributes to the message partition of corresponding number each consumption end and processes,
That is, according to the disposal ability of each consumption end, load has been carried out equilibrium treatment, thus has avoided because disappearing
Expense end disposal ability difference and cause the defect that message can not be processed in time, and make to consume end can be parallel
Process the message in message queue, it is achieved that flexibly, reasonably for consumption end assignment messages subregion, raising
The handling capacity that consumption end is overall, it is achieved that the optimum allocation of message partition.
Algorithm and display be not solid with any certain computer, virtual system or miscellaneous equipment provided herein
Have relevant.Various general-purpose systems can also be used together with based on teaching in this.As described above,
Construct the structure required by this kind of system to be apparent from.Additionally, the present invention is also not for any specific
Programming language.It is understood that, it is possible to use various programming languages realize the content of invention described herein,
And the description done language-specific above is the preferred forms in order to disclose the present invention.
In description mentioned herein, illustrate a large amount of detail.It is to be appreciated, however, that this
Inventive embodiment can be put into practice in the case of not having these details.In some instances, not
It is shown specifically known method, structure and technology, in order to do not obscure the understanding of this description.
Similarly, it will be appreciated that in order to simplify the disclosure and help understand in each inventive aspect one
Or multiple, above in the description of the exemplary embodiment of the present invention, each feature of the present invention is sometimes
It is grouped together in single embodiment, figure or descriptions thereof.But, should be by the disclosure
Method be construed to reflect an intention that i.e. the present invention for required protection require ratio in each claim
The middle more feature of feature be expressly recited.More precisely, as the following claims reflect
As, inventive aspect is all features less than single embodiment disclosed above.Therefore, it then follows
Claims of detailed description of the invention are thus expressly incorporated in this detailed description of the invention, the most each right
Requirement itself is all as the independent embodiment of the present invention.
Those skilled in the art are appreciated that and can carry out the module in the equipment in embodiment certainly
Change adaptively and they are arranged in one or more equipment different from this embodiment.Permissible
Module in embodiment or unit or assembly are combined into a module or unit or assembly, and in addition may be used
To put them into multiple submodule or subelement or sub-component.Except such feature and/or process or
Outside at least some in unit excludes each other, can use any combination that (this specification is included companion
With claim, summary and accompanying drawing) disclosed in all features and so disclosed any method or
All processes of person's equipment or unit are combined.Unless expressly stated otherwise, this specification (includes companion
With claim, summary and accompanying drawing) disclosed in each feature can by provide identical, equivalent or phase
Replace like the alternative features of purpose.
Although additionally, it will be appreciated by those of skill in the art that embodiments more described herein include it
Some feature included in its embodiment rather than further feature, but the group of the feature of different embodiment
Close and mean to be within the scope of the present invention and formed different embodiments.Such as, in following power
In profit claim, one of arbitrarily can mode making in any combination of embodiment required for protection
With.
The all parts embodiment of the present invention can realize with hardware, or to process at one or more
The software module run on device realizes, or realizes with combinations thereof.Those skilled in the art should
Understand, microprocessor or digital signal processor (DSP) can be used in practice to realize basis
The some or all functions of the some or all parts in the message distribution device of the embodiment of the present invention.
The present invention be also implemented as part or all the equipment for performing method as described herein or
Person's device program (such as, computer program and computer program).Such realize the present invention's
Program can store on a computer-readable medium, or can be to have the form of one or more signal.
Such signal can be downloaded from internet website and obtain, or provides on carrier signal, or with
Any other form provides.
The present invention will be described rather than limits the invention to it should be noted above-described embodiment,
And those skilled in the art can design replacement in fact without departing from the scope of the appended claims
Execute example.In the claims, should not will be located in any reference marks between bracket to be configured to right is wanted
The restriction asked.Word " comprises " and does not excludes the presence of the element or step not arranged in the claims.It is positioned at
Word "a" or "an" before element does not excludes the presence of multiple such element.The present invention is permissible
By means of including the hardware of some different elements and realizing by means of properly programmed computer.?
If listing in the unit claim of equipment for drying, several in these devices can be by same
Hardware branch specifically embodies.Word first, second and third use do not indicate that any order.
Can be title by these word explanations.
The invention discloses: A1, a kind of message distributing method, comprising:
Periodically collect performance information and the network condition information of multiple consumption ends；
Obtain the relevant information of at least one message partition in message queue, each in described message queue disappear
Breath subregion comprises one group of message；
Performance information according to multiple consumption ends and network condition information, by least one message partition described
Distribute to each consumption end process.
A2, according to the method described in A1, wherein, in described acquisition message queue, at least one message is divided
The relevant information in district farther includes:
Receive the notice of third party's module transmission with the relevant information obtaining at least one message partition.
A3, according to the method described in A2, wherein, the performance of described periodic collection multiple consumption end
Information and network condition information farther include:
Sending data packet request to each consumption end, receive each consumption end return comprises respective performance letter
The response package of breath；
Speed or time according to sending data packet request with receive response package determine described network condition
Information.
A4, according to the method described in A1, wherein, described method also includes: carried by third party's module
The subscribing mechanism of confession subscribes to the related announcement of message queue；
In described acquisition message queue, the relevant information of at least one message partition farther includes: work as message
When at least one message partition in queue occurs to update, the subscription receiving the transmission of described third party's module is led to
Know to obtain the relevant information of at least one message partition.
A5, according to the method described in A4, wherein, in the property of described periodic collection multiple consumption end
Before energy information and network condition information, described method also includes: obtain multiple consumption end in advance the 3rd
The IP address of registration in side's module；
Performance information and the network condition information of described periodic collection multiple consumption end farther include:
Send data packet request according to IP address to each consumption end, receive comprising of each consumption end return
Each response package of performance information；
Speed or time according to sending data packet request with receive response package determine described network condition
Information.
A6, according to the method described in any one of A1-A5, wherein, described performance information comprises: CPU
Occupied information, memory occupation information, I/O occupied information and/or data throughout.
A7, according to the method described in any one of A1-A6, wherein, the described performance according to multiple consumption ends
Information and network condition information, distribute at least one message partition described each consumption end and process
Farther include:
Performance information according to multiple consumption ends and network condition information, determine can disappearing of each consumption end
The number of the message partition taken；
The message partition of corresponding number is distributed to each consumption end process.
A8, according to the method described in any one of A1-A6, wherein, the described performance according to multiple consumption ends
Information and network condition information, distribute at least one message partition described each consumption end and process
Farther include:
Performance information according to multiple consumption ends and network condition information, determine can disappearing of each consumption end
The weight of the message partition taken；
According to described weight, at least one message partition described is distributed to each consumption end process.
The invention also discloses: B9, a kind of message distribution device, comprising:
Collection module, is suitable to periodically collect performance information and the network condition information of multiple consumption end；
First acquisition module, is suitable to obtain the relevant information of at least one message partition, institute in message queue
State each message partition in message queue and comprise one group of message；
Distribution module, is suitable to the performance information according to multiple consumption ends and network condition information, by described extremely
A few message partition is distributed to each consumption end and is processed.
B10, according to the device described in B9, wherein, described first acquisition module is further adapted for: receive
The notice that third party's module sends is with the relevant information obtaining at least one message partition.
B11, according to the device described in B10, wherein, described collection module is further adapted for: receive the
The performance information of each consumption end that tripartite's module sends and network condition information；Wherein, each consumption end
Performance information to be third party's module send data packet request to each consumption end, and receive each consumption end
The response package comprising respective performance information returned obtains, and network condition information is third party's module
Determine according to the speed or time sending data packet request with receive response package.
B12, according to the device described in B9, wherein, described device also includes: subscribing module, is suitable to lead to
The subscribing mechanism that crossing third party's module provides subscribes to the related announcement of message queue；
Described first acquisition module is further adapted for: when at least one message partition in message queue occurs
During renewal, receive the subscribing notification of described third party's module transmission to obtain the phase of at least one message partition
Pass information.
B13, according to the device described in B12, wherein, described device also includes: the second acquisition module,
Be suitable to obtain multiple consumption end IP address of registration in third party's module in advance；
Described collection module farther includes:
Transmitting element, is suitable to send data packet request according to IP address to each consumption end；
Receive unit, be suitable to receive the response package comprising respective performance information that each consumption end returns；
First determines unit, be suitable to according to send data packet request and receive response package speed or time
Between determine described network condition information.
B14, according to the device described in any one of B9-B13, wherein, described performance information comprises: CPU
Occupied information, memory occupation information, I/O occupied information and/or data throughout.
B15, according to the device described in any one of B9-B14, wherein, described distribution module farther includes:
Second determines unit, is suitable to the performance information according to multiple consumption ends and network condition information, determines
The number of the message partition that can consume of each consumption end；
Dispatching Unit, is suitable to that the message partition of corresponding number is distributed to each consumption end and processes.
B16, according to the device described in any one of B9-B14, wherein, described distribution module farther includes:
Second determines unit, is suitable to the performance information according to multiple consumption ends and network condition information, determines
The weight of the message partition that can consume of each consumption end；
Dispatching Unit, is suitable to, according to described weight, at least one message partition described is distributed to each and consumes
End processes.
The invention also discloses: C17, a kind of message distributing system, including described in any one of B9-B16
Message distribution device, third party's module, message queue and multiple consumption end.
Claims (10)
1. a message distributing method, comprising:
Periodically collect performance information and the network condition information of multiple consumption ends；
Obtain the relevant information of at least one message partition in message queue, each in described message queue disappear
Breath subregion comprises one group of message；
Performance information according to multiple consumption ends and network condition information, by least one message partition described
Distribute to each consumption end process.
Method the most according to claim 1, wherein, in described acquisition message queue, at least one disappears
The relevant information of breath subregion farther includes:
Receive the notice of third party's module transmission with the relevant information obtaining at least one message partition.
Method the most according to claim 2, wherein, described periodic collection multiple consumption end
Performance information and network condition information farther include:
Sending data packet request to each consumption end, receive each consumption end return comprises respective performance letter
The response package of breath；
Speed or time according to sending data packet request with receive response package determine described network condition
Information.
Method the most according to claim 1, wherein, described method also includes: by third party's mould
The subscribing mechanism that block provides subscribes to the related announcement of message queue；
In described acquisition message queue, the relevant information of at least one message partition farther includes: work as message
When at least one message partition in queue occurs to update, the subscription receiving the transmission of described third party's module is led to
Know to obtain the relevant information of at least one message partition.
Method the most according to claim 4, wherein, at described periodic collection multiple consumption end
Performance information and network condition information before, described method also includes: obtains multiple consumption end and exists in advance
The IP address of registration in third party's module；
Performance information and the network condition information of described periodic collection multiple consumption end farther include:
Send data packet request according to IP address to each consumption end, receive comprising of each consumption end return
Each response package of performance information；
Speed or time according to sending data packet request with receive response package determine described network condition
Information.
6. according to the method described in any one of claim 1-5, wherein, described performance information comprises: CPU
Occupied information, memory occupation information, I/O occupied information and/or data throughout.
7. according to the method described in any one of claim 1-6, wherein, described according to multiple consumption ends
Performance information and network condition information, distribute at least one message partition described each consumption end and carry out
Process farther includes:
Performance information according to multiple consumption ends and network condition information, determine can disappearing of each consumption end
The number of the message partition taken；
The message partition of corresponding number is distributed to each consumption end process.
8. according to the method described in any one of claim 1-6, wherein, described according to multiple consumption ends
Performance information and network condition information, distribute at least one message partition described each consumption end and carry out
Process farther includes:
Performance information according to multiple consumption ends and network condition information, determine can disappearing of each consumption end
The weight of the message partition taken；
According to described weight, at least one message partition described is distributed to each consumption end process.
9. a message distribution device, comprising:
Collection module, is suitable to periodically collect performance information and the network condition information of multiple consumption end；
First acquisition module, is suitable to obtain the relevant information of at least one message partition, institute in message queue
State each message partition in message queue and comprise one group of message；
Distribution module, is suitable to the performance information according to multiple consumption ends and network condition information, by described extremely
A few message partition is distributed to each consumption end and is processed.
10. a message distributing system, including the message distribution device described in claim 9, third party
Module, message queue and multiple consumption end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610201563.6A CN105912412A (en) | 2016-03-31 | 2016-03-31 | Message distribution method, message distribution device and message distribution system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610201563.6A CN105912412A (en) | 2016-03-31 | 2016-03-31 | Message distribution method, message distribution device and message distribution system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105912412A true CN105912412A (en) | 2016-08-31 |
Family
ID=56745432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610201563.6A Pending CN105912412A (en) | 2016-03-31 | 2016-03-31 | Message distribution method, message distribution device and message distribution system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105912412A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106598760A (en) * | 2016-12-19 | 2017-04-26 | 北京奇虎科技有限公司 | Message queue consumption method and apparatus |
| CN111225046A (en) * | 2019-12-31 | 2020-06-02 | 广东金赋科技股份有限公司 | Method, device, medium and electronic equipment for internal and external network data transmission |
| CN113608870A (en) * | 2021-07-28 | 2021-11-05 | 北京金山云网络技术有限公司 | Load balancing method and device of message queue, electronic equipment and storage medium |
| CN116132367A (en) * | 2022-08-17 | 2023-05-16 | 马上消费金融股份有限公司 | Message processing method and device and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1859122A (en) * | 2006-02-23 | 2006-11-08 | 华为技术有限公司 | Method and device for realizing classified service to business provider |
| US7562365B2 (en) * | 2004-01-12 | 2009-07-14 | International Business Machines Corporation | Random access for processing messages in a message queue |
| CN103516738A (en) * | 2012-06-21 | 2014-01-15 | 中兴通讯股份有限公司 | Message distribution method, device and system |
| CN104092717A (en) * | 2013-12-13 | 2014-10-08 | 深圳市腾讯计算机***有限公司 | Message processing method and system, message destination end device and distributed message system |
| CN105117291A (en) * | 2015-06-30 | 2015-12-02 | 百度在线网络技术（北京）有限公司 | Method and system adjusting operation quantity of message push executor |
| CN105338061A (en) * | 2015-09-29 | 2016-02-17 | 华中科技大学 | Lightweight message oriented middleware realization method and system |
| CN105450784A (en) * | 2016-01-20 | 2016-03-30 | 北京京东尚科信息技术有限公司 | An apparatus and method for distributing consumption nodes to messages in an MQ |
-
2016
- 2016-03-31 CN CN201610201563.6A patent/CN105912412A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7562365B2 (en) * | 2004-01-12 | 2009-07-14 | International Business Machines Corporation | Random access for processing messages in a message queue |
| CN1859122A (en) * | 2006-02-23 | 2006-11-08 | 华为技术有限公司 | Method and device for realizing classified service to business provider |
| CN103516738A (en) * | 2012-06-21 | 2014-01-15 | 中兴通讯股份有限公司 | Message distribution method, device and system |
| CN104092717A (en) * | 2013-12-13 | 2014-10-08 | 深圳市腾讯计算机***有限公司 | Message processing method and system, message destination end device and distributed message system |
| CN105117291A (en) * | 2015-06-30 | 2015-12-02 | 百度在线网络技术（北京）有限公司 | Method and system adjusting operation quantity of message push executor |
| CN105338061A (en) * | 2015-09-29 | 2016-02-17 | 华中科技大学 | Lightweight message oriented middleware realization method and system |
| CN105450784A (en) * | 2016-01-20 | 2016-03-30 | 北京京东尚科信息技术有限公司 | An apparatus and method for distributing consumption nodes to messages in an MQ |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106598760A (en) * | 2016-12-19 | 2017-04-26 | 北京奇虎科技有限公司 | Message queue consumption method and apparatus |
| CN106598760B (en) * | 2016-12-19 | 2020-07-10 | 北京奇虎科技有限公司 | Consumption method and device of message queue |
| CN111225046A (en) * | 2019-12-31 | 2020-06-02 | 广东金赋科技股份有限公司 | Method, device, medium and electronic equipment for internal and external network data transmission |
| CN113608870A (en) * | 2021-07-28 | 2021-11-05 | 北京金山云网络技术有限公司 | Load balancing method and device of message queue, electronic equipment and storage medium |
| CN116132367A (en) * | 2022-08-17 | 2023-05-16 | 马上消费金融股份有限公司 | Message processing method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109218355B (en) | Load balancing engine, client, distributed computing system and load balancing method | |
| CN101887382B (en) | The referee method of dynamic priority and device | |
| US10574619B2 (en) | Cluster assisted MQTT client coverage for fat-pipe cloud applications | |
| CN101616083B (en) | Message forwarding method and device | |
| US8671134B2 (en) | Method and system for data distribution in high performance computing cluster | |
| CN105912412A (en) | Message distribution method, message distribution device and message distribution system | |
| CN106598760B (en) | Consumption method and device of message queue | |
| US20040024873A1 (en) | Load balancing the servicing of received packets | |
| CN109523202B (en) | Order processing method, device, equipment and computer readable storage medium | |
| CN105408879A (en) | Resource management for peripheral component interconnect-express domains | |
| CN101142552A (en) | Resource allocation in computing systems | |
| CN106790692A (en) | A kind of load-balancing method and device of many clusters | |
| CN103763343A (en) | Method and device for processing service access | |
| EP3295629B1 (en) | Query plan and operation-aware communication buffer management | |
| CN102594691A (en) | Method and device for processing message | |
| CN105491150A (en) | Load balance processing method based on time sequence and system | |
| CN109032800A (en) | A kind of load equilibration scheduling method, load balancer, server and system | |
| CN109450816A (en) | A kind of array dispatching method, device, the network equipment and storage medium | |
| CN113762830B (en) | Order splitting processing method, device, equipment and readable storage medium | |
| CN106952085B (en) | Method and device for data storage and service processing | |
| CN111770029A (en) | Dynamic queue forwarding method, system and storage medium based on RabbitMQ and ActiveMQ | |
| CN114710571B (en) | Data packet processing system | |
| US20100198971A1 (en) | Dynamically provisioning clusters of middleware appliances | |
| CN106856459B (en) | Message scheduling method and device | |
| CN113347238A (en) | Message partitioning method, system, device and storage medium based on block chain |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160831 |