CN102497601A - Adaptive data splitting method - Google Patents
Adaptive data splitting method Download PDFInfo
- Publication number
- CN102497601A CN102497601A CN2011104134798A CN201110413479A CN102497601A CN 102497601 A CN102497601 A CN 102497601A CN 2011104134798 A CN2011104134798 A CN 2011104134798A CN 201110413479 A CN201110413479 A CN 201110413479A CN 102497601 A CN102497601 A CN 102497601A
- Authority
- CN
- China
- Prior art keywords
- shunt method
- data
- shunting
- splitting
- shunt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides an adaptive data splitting method. A multilevel splitting idea is utilized. The method comprises the following steps of: performing primary splitting to split a data stream into groups, wherein certain data can be split according to the groups for post processing; and performing splitting in each group according to a certain conventional splitting method. Compared with the prior art, the method has the advantages that: the multilevel splitting idea is clearer and more rational in splitting; system resources can be effectively used by dynamic regulation; data packet loss can be prevented; and resources can be effectively utilized.
Description
Technical field
The invention belongs to the network data processing field, be specifically related to a kind of adaptive data distribution method.
Background technology
High speed development along with network; Network traffic data also increases rapidly, for real-time network data flow, some high performance servers occurred; These servers are multiple-path multiple-core mostly; In order to make full use of each processor, handle a large amount of network datas, generally adopt the polymerization and the dispersion technology of network data.At first, polymerization is from all data flow of outer net input, then; Be distributed in a plurality of formations according to the data flow of certain distributing strategy packet aggregation and go; Formation is corresponding one by one with CPU, and just a CPU handles the data in the formation, the cpu resource of the multiple-path multiple-core that makes full use of like this.
Some traditional distributing strategies are wheel diverting flow and hash shunting.The so-called diverting flow of taking turns is meant the number of shunt according to formation, and distributing data flows to each formation in turn successively, and each formation all is impartial acquisition data flow.The hash shunt method is meant that the territory (like four-tuple, five-tuple etc.) according to some appointment calculates the hash value, then according to the mapping relations of hash value and formation; Be distributed to data in certain specific formation and go; What wherein the mapping relations of hash value and formation can be impartial also can be unequal (can according to certain ratio), certainly in any case; When network equipment initialization, just having fixed of this mapping relations is a kind of method of static state.
Patent No. CN00118075.4 (according to the device of network bandwidth self adaptive ground control data transmission speed) disclose a kind of according to through the time network bandwidth self adaptive ground that changes control the device of the transmission rate of view data; Comprise: image compressor, the view data and the compression that quantize input according to the predetermined quantitative level quantize view data; Transmitter, buffer memory through the compression view data and then send to network; The complexity estimator estimates to want the image encoded complexity of data according to the complexity of previous coding view data; Controller according to buffer data size, estimate that the complexity and the network bandwidth confirm the quantized level of image compressor, and is controlled quantized level so that buffer data size equals in bandwidth range, desired value to be set before.
Patent No. CN200810166038.0 (content-adaptive device and content-adaptive method) discloses content-adaptive device and content-adaptive method.This device has the metadata (d101) of the segmentation viewpoint that data (d102) record that a plurality of segmentations (segment) are formed gives each segmentation and with user's hobby as the user preference (preference) of User Perspective record (d103); Through said segmentation viewpoint being compared according to said User Perspective; Extract the segmentation in the data, adapt to user's hobby video data with this.
Above-mentioned technology can not be adjusted shunt method dynamically according to the load of a plurality of CPU, no matter be wheel diverting flow or hash shunting, all is a kind of shunt method of static state.The data flow that gets into from outer net has just determined to flow to which formation behind system initialization, give which CPU and handle, and with the operation disposal ability of CPU, running status has no relation.Therefore, possibly cause effectively to utilize cpu resource, even cause timely data in the processing queue, cause packet loss than the CPU of heavy load.
Summary of the invention
The present invention overcomes the deficiency of prior art, and purpose is dynamic adaptive adjustment shunt method, makes full use of system resource, prevents that processing is not come and packet loss because local resource-constrained causes data traffic.Outer net data to fan-in network equipment are carried out adaptive streaming;
The present invention provides a kind of adaptive data distribution method, utilizes multi-level shunting thought;
At first, do preliminary shunting, packet data streams divides in groups.Can open some data by component, be convenient to post-processed;
Afterwards, in group, press certain traditional shunt method.
The adaptive streaming of dynamically adjusting; Can select the ratio of said shunt method or shunt method according to the cpu load situation.
Said flow process is:
1) a kind of adaptive data distribution method is characterized in that, all outer net data flow of polymerization input equipment;
2) according to tentatively shunting someway, be divided into different group (IPV4/IPV6, or VALN/ other);
3) according to certain shunt method component in formation, as in turn, the hash shunting.
4) according to cpu load, situation such as performance, the shunt method of dynamic adjustment step 3, or shunting ratio.
Compared with prior art, beneficial effect of the present invention is:
Multi-level shunting makes shunts more clear and rational, and dynamic adjustment is the using system resource effectively, prevents the data packet loss and makes full use of system resource.
Description of drawings
Fig. 1 is a schematic flow sheet of the present invention.
Embodiment
Fig. 1 is a schematic flow sheet of the present invention, and the shunt method of layering is multi-level shunt method; At first, do preliminary shunting, packet data streams divides in groups.Can open some data by component, be convenient to post-processed.Then, in group, carrying out formation by certain traditional shunt method divides.
2. the adaptive streaming of dynamically adjusting; Can select the ratio of shunt method or shunt method according to the cpu load situation.
The scheme detailed step is following:
1. all outer net data flow of polymerization input equipment;
2. according to tentatively shunting someway, be divided into different group (IPV4/IPV6, or VALN/ other);
According to certain shunt method component in formation, as in turn, the hash shunting.
4. according to cpu load, situation such as performance, the shunt method of dynamic adjustment step 3, or shunting ratio.
The present invention adjusts shunt method dynamically according to the load of a plurality of CPU, no matter be wheel diverting flow or hash shunting.The data flow that gets into from outer net can dynamic flow behind system initialization to formation that can not effectively utilize and cpu resource, therefore, satisfied that CPU than the CPU of heavy load and less load can in time coordinate and processing queue in data, thereby prevent packet loss.
Above embodiment is only in order to technical scheme of the present invention to be described but not to its restriction; Although the present invention has been carried out detailed explanation with reference to the foregoing description; The those of ordinary skill in said field is to be understood that: still can specific embodiments of the invention make amendment or replacement on an equal basis; And do not break away from any modification of spirit and scope of the invention or be equal to replacement, it all should be encompassed in the middle of the claim scope of the present invention.
Claims (6)
1. an adaptive data distribution method is characterized in that, utilizes multi-level shunt method;
At first, do preliminary shunting, packet data streams divides in groups.Can open some data by component, be convenient to post-processed;
Afterwards, in group, press certain traditional shunt method.
2. shunt method according to claim 1 is characterized in that, dynamically the adaptive streaming of adjustment; Can select the ratio of said shunt method or shunt method according to the cpu load situation.
3. according to the described shunt method of claim 1-2, it is characterized in that said shunting flow process is:
1) all outer net data flow of polymerization input equipment;
2) according to tentatively shunting someway, be divided into different groups;
3) according to certain shunt method component in formation;
4) according to cpu load, situation such as performance, the shunt method of dynamic adjustment step 3 or shunting ratio;
5) take ad hoc approach or shunting proportional diverting.
4. according to the described shunt method of claim 1-3, it is characterized in that tentatively dividing into groups according to IPV4, IPV6, VALN or other classifications.
5. according to the described shunt method of claim 1-4, it is characterized in that arriving formation to component according to the method for taking turns.
6. according to the described shunt method of claim 1-5, it is characterized in that arriving formation to component according to the hash shunt method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011104134798A CN102497601A (en) | 2011-12-13 | 2011-12-13 | Adaptive data splitting method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011104134798A CN102497601A (en) | 2011-12-13 | 2011-12-13 | Adaptive data splitting method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102497601A true CN102497601A (en) | 2012-06-13 |
Family
ID=46189382
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011104134798A Pending CN102497601A (en) | 2011-12-13 | 2011-12-13 | Adaptive data splitting method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102497601A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102916905A (en) * | 2012-10-18 | 2013-02-06 | 曙光信息产业(北京)有限公司 | Gigabit network card multi-path shunting method and system based on hash algorithm |
| CN107105015A (en) * | 2017-03-31 | 2017-08-29 | 北京奇艺世纪科技有限公司 | The shunt method and device of a kind of data flow |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101102281A (en) * | 2007-08-16 | 2008-01-09 | 中兴通讯股份有限公司 | Data processing method for reporting a large number of data in mobile communication system |
| CN101217486A (en) * | 2008-01-11 | 2008-07-09 | 东南大学 | A mobile Internet data load allocation method based on network processor |
| CN101217491A (en) * | 2008-01-04 | 2008-07-09 | 杭州华三通信技术有限公司 | A method of rectification processing unit load allocation method and device |
| CN102025593A (en) * | 2009-09-21 | 2011-04-20 | ***通信集团公司 | Distributed user access system and method |
-
2011
- 2011-12-13 CN CN2011104134798A patent/CN102497601A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101102281A (en) * | 2007-08-16 | 2008-01-09 | 中兴通讯股份有限公司 | Data processing method for reporting a large number of data in mobile communication system |
| CN101217491A (en) * | 2008-01-04 | 2008-07-09 | 杭州华三通信技术有限公司 | A method of rectification processing unit load allocation method and device |
| CN101217486A (en) * | 2008-01-11 | 2008-07-09 | 东南大学 | A mobile Internet data load allocation method based on network processor |
| CN102025593A (en) * | 2009-09-21 | 2011-04-20 | ***通信集团公司 | Distributed user access system and method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102916905A (en) * | 2012-10-18 | 2013-02-06 | 曙光信息产业(北京)有限公司 | Gigabit network card multi-path shunting method and system based on hash algorithm |
| CN107105015A (en) * | 2017-03-31 | 2017-08-29 | 北京奇艺世纪科技有限公司 | The shunt method and device of a kind of data flow |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103248884B (en) | A kind of control the system of video rate, base station and method | |
| CN109874031A (en) | Multipriority dynamic code rate method of adjustment and system based on network bandwidth prediction | |
| CN103338393A (en) | Video code rate selecting method driven by user experience under HSPA system | |
| CN104125006B (en) | Satellite communication network bandwidth allocation methods | |
| WO2002080518A3 (en) | Adaptive bandwidth system and method for video transmission | |
| DE102014012611A1 (en) | Spatial multiplexing receiver | |
| CN105187849A (en) | Scalable video multicast resource allocation method based on D2D and cellular network | |
| CN106550393A (en) | A kind of business WIFI user dynamic bandwidth method for limiting and system | |
| CN102724763B (en) | Time domain group scheduling method on basis of two-dimensional priority level | |
| CN101969408A (en) | Active queue management method based on packet DSCP (Differentiated Services Code Point) marks | |
| CN102404224A (en) | Self-adaptive balanced load shunting equipment and method | |
| CN102638457B (en) | Method and system for automatically allocating server resource | |
| CN102497601A (en) | Adaptive data splitting method | |
| CN103441943B (en) | A kind of traffic messages control method and device | |
| CN103023819B (en) | A kind of coarseness distribution method of dynamic bandwidth | |
| CN104683066B (en) | The anti-mutual interference method of multimode gateway | |
| CN107071919B (en) | LTE downlink real-time service scheduling improvement method based on packet loss rate | |
| CN110753101B (en) | Low-energy-consumption computing node selection and computing task allocation method in edge computing | |
| CN108738158A (en) | A kind of LTE downlink scheduling methods based on optimized throughput | |
| CN107770082B (en) | Transmission control method based on task flow characteristics in data center network | |
| CN109801246A (en) | A kind of color histogram equalization methods of adaptive threshold | |
| CN1284337C (en) | A network load balancing method in multipoint communication | |
| CN1738287A (en) | Sending controller channel stream limiting method | |
| CN109120480B (en) | Link optimization method and device based on data center | |
| WO2014066600A4 (en) | Systems and methods for sharing bandwidth across multiple video streams |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120613 |