WO2022232976A1 - 一种基于令牌桶的流量控制方法和网络设备 - Google Patents
一种基于令牌桶的流量控制方法和网络设备 Download PDFInfo
- Publication number
- WO2022232976A1 WO2022232976A1 PCT/CN2021/091845 CN2021091845W WO2022232976A1 WO 2022232976 A1 WO2022232976 A1 WO 2022232976A1 CN 2021091845 W CN2021091845 W CN 2021091845W WO 2022232976 A1 WO2022232976 A1 WO 2022232976A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- token
- network device
- length
- token bucket
- packet
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000001186 cumulative effect Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 6
- 238000007792 addition Methods 0.000 abstract description 32
- 238000004422 calculation algorithm Methods 0.000 description 29
- 238000004364 calculation method Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000010606 normalization Methods 0.000 description 4
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007493 shaping process Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
Definitions
- the present application relates to the field of network technologies, and in particular, to a token bucket-based flow control method and network device.
- the token bucket algorithm is a common network traffic shaping and rate limiting algorithm.
- the token bucket algorithm can evaluate the relationship between the traffic rate passing through the network device and the rated rate configured by the user, so as to realize the control of the traffic rate according to this relationship.
- a token bucket can be visualized as a bucket for storing tokens.
- the network device is set to fill the bucket with tokens according to the rate configured by the user, and each token corresponds to a certain amount of traffic.
- the network device deducts the token corresponding to the traffic size from the bucket.
- traffic can no longer continue to pass through the network device.
- the rate at which the network device fills the token bucket with tokens is determined by the number of tokens added, and the number of tokens deducted for each deduction is determined by the number of token deductions.
- network devices need to rely on division operations to calculate the number of tokens added and the number of tokens deducted. For hardware, performing the division operation consumes a large amount of hardware resources, and requires high hardware complexity and computing power.
- the embodiments of the present application provide a token bucket flow control method and network device, which can reduce the consumption of hardware resources by the token bucket algorithm, and have higher flow control accuracy.
- an embodiment of the present application provides a token bucket-based flow control method.
- the method is used to control the flow of packets passing through a network device.
- the method includes: periodically adding a token bucket to the token bucket.
- the number of tokens added in each period is an integer multiple of the natural power of 2; when a packet passes through the network device, the token is deducted from the token bucket according to the length of the packet.
- the token deductions of are integer multiples of natural powers of 2. Token Additions Token Deductions.
- the token addition number and the token deduction number are both integer multiples of the natural number power of 2, so they can be obtained accurately by the hardware shift operation, no need to introduce a divider, and no need to take approximate values, Therefore, the token bucket-based flow control method provided by the embodiment of the present application can achieve precise flow control while consuming relatively low hardware resources.
- the token addition number conforms to an integer multiple of a natural number power of 2, including: the token addition number is equal to the numerical product of the natural number power of 2 and the expected bandwidth of the token bucket, and the expected bandwidth is an integer .
- the token bucket-based flow control method provided by the embodiment of the present application can realize precise flow control at the integer level.
- the token deduction is an integer multiple of a natural number power of 2, including: the token deduction is equal to the numerical product of the natural number power of 2 and the clock frequency of the token bucket, and the clock frequency is an integer . Token deductions.
- the token bucket-based flow control method provided by the embodiment of the present application can perform flow control based on an integer-level clock frequency.
- the number of token additions is obtained by the following formula:
- TokenAdd is the number of tokens added
- ⁇ CycleCnt is the number of clock cycles of the token bucket corresponding to each cycle of adding tokens
- n is the expected bandwidth
- TokenUnit is the token unit of the token bucket, and the token unit represents each The length of the packet corresponding to the token
- m is the clock frequency of the token bucket
- the number of tokens added is equal to the product of the natural number power of 2 and the expected bandwidth of the token bucket, including: the number of clock cycles equal to the natural number of 2 Power, the numerical product of the token unit and the clock frequency is equal to the natural power of 2.
- the token bucket-based flow control method provided by the embodiment of the present application constrains the numerical relationship among ⁇ CycleCnt, the token unit and the clock frequency, so that the token addition number conforms to an integer multiple of a natural number power of 2.
- the number of token additions and the number of token deductions also satisfy the following relationship:
- n is the expected bandwidth
- m is the clock frequency
- N is the clock frequency
- X, Y are all natural numbers
- X ⁇ N, Y ⁇ X-3 is the expected bandwidth
- the natural number Y is obtained by the following formula:
- PktLen is the minimum length required for the packet passing through the network device each time the token deduction is performed, and the minimum length conforms to the natural power of 2.
- the natural number X is obtained by the following formula:
- the natural number N is obtained by the following formula:
- the token bucket-based flow control method provided by the embodiment of the present application further simplifies the calculation methods of the token deduction number and the token addition number based on the numerical relationship between ⁇ CycleCnt, the token unit and the clock frequency.
- the token bucket-based flow control method provided by the embodiment of the present application can perform flow control on the packets transmitted based on the cell.
- the token bucket-based flow control method provided by the embodiment of the present application can perform flow control on packets transmitted based on a series of multiples of varying lengths.
- the token bucket-based flow control method provided by the embodiment of the present application can perform flow control on packets transmitted based on any length.
- the token bucket further includes a rate class table, and the rate class table includes a token bucket parameter corresponding to at least one flow control interval, so that the network device determines the token bucket parameter according to the flow control interval to which the desired bandwidth belongs.
- the token bucket parameters included in each flow control interval include one or more of the following: token unit, number of first cycles, natural number N, natural number X, bandwidth increment granularity, and bandwidth increment granularity The corresponding minimum number of token additions, the interval of the number of token additions corresponding to the flow control interval, and the interval of the expected bandwidth corresponding to the flow control interval.
- token unit number of first cycles
- natural number N natural number
- natural number X natural number X
- bandwidth increment granularity bandwidth increment granularity
- bandwidth increment granularity bandwidth increment granularity
- an embodiment of the present application provides a network device, including a memory and a processor, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the network device executes the first aspect and its implementations method in method.
- the present application also provides a computer storage medium.
- the computer storage medium computer instructions, when executed on the computer, cause the computer to perform the methods in the above-described aspects and implementations thereof.
- the present application also provides a computer program product comprising instructions, which, when executed on a computer, cause the computer to execute the methods in the above aspects and implementations thereof.
- Figure 1 is a schematic diagram of the token bucket algorithm
- FIG. 2 is a schematic structural diagram of a network device shown in an embodiment of the present application.
- FIG. 3 is a schematic diagram of the limiting conditions for the parameters of the token bucket according to an embodiment of the present application
- Fig. 4 is the flow chart of performing normalization processing to the message length PktLen shown in the embodiment of the present application;
- FIG. 5 is a schematic diagram of cumulative message length and token deduction shown in an embodiment of the present application.
- FIG. 6 is a token bucket rate level table provided by an embodiment of the present application.
- the token bucket algorithm is a common network traffic shaping and rate limiting algorithm.
- the token bucket algorithm can evaluate the relationship between the traffic rate passing through the network device and the rated rate configured by the user, so as to realize the control of the traffic rate according to this relationship.
- FIG. 1 is a schematic diagram of the token bucket algorithm.
- the token bucket 10 can be visually regarded as a bucket for storing tokens 11.
- the network device is set to put (or fill) the tokens 11 into the bucket according to the rate configured by the user, and each token corresponds to A certain flow of 12.
- the network device deducts the token 11 corresponding to the size of the traffic 12 from the bucket. If no traffic 12 passes through for a long time, the number of tokens 11 in the bucket will reach a certain upper limit, and after that, tokens 11 put into the bucket will be discarded by the bucket.
- the network device can control the rate of traffic passing through the network device according to the rate configured by the user.
- Token 1
- Token 1
- a token is a Token
- Token the value of Token represents the number of tokens in the token bucket.
- the number of tokens is essentially the packet traffic expected to pass through the network device and the packets actually passed through the network device. difference in flow.
- Token stuffing operations can involve the following concepts:
- Freq the clock frequency for filling tokens into the token bucket
- Freq the clock frequency for filling tokens into the token bucket
- Hz Hertz
- GHz GHz
- MHz MHz
- Freq 1000MHz
- ⁇ T 3 Define a certain time interval as ⁇ T, and ⁇ T is usually in seconds (s).
- the expected bandwidth is defined as BW (BandWidth), where the expected bandwidth BW represents the rated bandwidth of the network device that the user configures according to his expectations.
- the desired bandwidth BW is usually measured in bits/s (bits per second), bps (bits per second), Kbps (kilobits per second), Mbps (megabits per second), Gbps (gigabits per second).
- TokenAdd represents the number of tokens corresponding to the packet traffic allowed to pass through the network device according to the expected bandwidth BW within a certain time interval ⁇ T.
- Token deduction operations can involve the following concepts:
- PktLen (Packet Length), where PktLen can refer to the length of the packet corresponding to one token deduction, which can be the length of one packet or the length of multiple packets, PktLen Usually in Bytes.
- TokenSub can specifically represent the number of tokens equivalent to the packet length PktLen of one token deduction.
- the working principle of the token bucket can be further expressed as: with the expected bandwidth BW, every time interval ⁇ T, the number of tokens equal to TokenAdd Fill it into the bucket; when a packet passes through, the number of tokens TokenSub corresponding to the packet length PktLen is deducted from the bucket; if the tokens in the bucket are exhausted, before new tokens are put into the bucket , the packets can no longer pass, thus achieving the goal of the token bucket algorithm: the passing rate of the control packets is the expected bandwidth BW.
- the token bucket algorithm is a common method for implementing flow control at present, and one of the most important indicators for evaluating the quality of flow control is the accuracy of flow control. If the token bucket algorithm is used, the precision here is the precision of the token bucket. In an implementation manner, the accuracy may be reflected in the form of an error rate. The higher the error rate is, the worse the accuracy is, and the higher the error rate is, the higher the accuracy is. For example: Accurate can be represented by the following formula (4):
- BW exp represents the expected bandwidth of the user
- BW cfg represents the hardware limitation of the network device, the actual bandwidth that can be configured. Due to the hardware limitation of the network device, BW exp and BW cfg may not actually be equal. Cause the value of Accurate is not 0, that is, there is an error.
- the accuracy of the token bucket may meet the requirements if the accuracy of the token bucket reaches an error of less than 0.5%.
- the accuracy of the bucket reaches 0 error, that is, the bandwidth BW cfg that can be actually configured by the network device is exactly the same as the user's expected bandwidth BW exp .
- implementing a token bucket with zero error accuracy or a high-precision token bucket needs to consume a lot of hardware resources, and requires high complexity and computing power of the hardware itself.
- the hardware of the network device can use a divider to perform a division operation according to formula 5 to accurately obtain the number of tokens to be filled, TokenAdd.
- an embodiment of the present application provides a token bucket-based flow control method, which may also be called a token bucket algorithm, which can achieve precise flow control while consuming relatively low hardware resources. For example, Error-free flow control.
- FIG. 2 is a schematic structural diagram of a network device according to an embodiment of the present application.
- the network device may specifically be a router, a switch, a network host or a server, etc., and may serve as a forwarding device to forward packets between an upstream node and a downstream node.
- the network device may include a memory 310 , a communication interface 320 and a processor 330 .
- the memory 310, the communication interface 320 and the processor 330 are coupled and connected, and program instructions are stored in the memory 310, and the processor 330 can call the program instructions in the memory 310 to make the network device perform related methods, such as parsing messages, generating messages, Packets are received and forwarded through the communication interface 320, and flow control is implemented through token buckets and the like.
- the processor 330 of the network device may include one or more processing units, such as a system on a chip (SoC), a central processing unit (CPU), a microcontroller (microcontroller, MCU), memory controller, etc.
- SoC system on a chip
- CPU central processing unit
- MCU microcontroller
- memory controller etc.
- different processing units may be independent devices, or may be integrated in one or more processors.
- the memory 310 of the network device may include one or more storage units, for example, may include a volatile memory (volatile memory), such as: dynamic random access memory (dynamic random access memory, DRAM), static random access memory Access memory (static random access memory, SRAM), etc.; can also include non-volatile memory (non-volatile memory, NVM), such as: read-only memory (read-only memory, ROM), flash memory (flash memory), etc. .
- volatile memory such as: dynamic random access memory (dynamic random access memory, DRAM), static random access memory Access memory (static random access memory, SRAM), etc.
- NVM non-volatile memory
- different storage units may be independent devices, or may be integrated or packaged in one or more processors or communication interfaces, and become part of the processors or communication interfaces.
- the communication interface 320 of the network device may include, for example, a network adapter (network adapter), a network interface card (network interface card), a local area network adapter (LAN adapter), a network interface controller (network interface controller, NIC), a modem (modem) and so on.
- the communication interface may be an independent device, and may also be partially or fully integrated or packaged in the processor to become a part of the processor.
- the structures illustrated in the embodiments of the present application do not constitute a specific limitation on network devices.
- the network device may include more or less components than shown, or combine some components, or separate some components, or arrange different components.
- the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
- the basic idea of the token bucket algorithm provided by the embodiment of the present application is to realize the division operation involved in calculating the token bucket filling amount TokenAdd and the token deduction number TokenSub in the form of a shift operation, thereby reducing the hardware requirements of the token bucket algorithm. Resource consumption, and accurate calculation of token bucket filling amount, improving flow control accuracy.
- the embodiment of the present application defines the parameters of the token bucket as follows:
- Restriction 1 The number of clock cycles ⁇ CycleCnt corresponding to the constraint time interval ⁇ T is equal to a natural number power of 2, that is, the time interval for token filling is a natural number power of 2 clock cycles.
- Constraint 3 To implement Constraint 2, TokenUnit can be a decimal to construct a natural power of 2 that is equal to TokenUnit ⁇ Freq.
- the token addition number TokenAdd and the token deduction number TokenSub are both exact integer values, and both are calculated by the hardware shift, so that the token bucket algorithm can affect the hardware resources. consumption is reduced and flow control is more precise.
- the token bucket algorithm in order for the token bucket algorithm to achieve accurate bandwidth control, the token bucket algorithm must be able to achieve accurate token filling and accurate token deduction operations for tokens according to user expectations.
- the addition and deduction of the token is realized by adding and deducting an integer number of tokens. Therefore, in order to achieve accurate token filling and accurate token deduction operations, the calculation results of formulas (2) and (3) must be Integer, that is, the number of token additions TokenAdd and the number of token deductions TokenSub must be integers.
- n and m may be natural numbers or other values, which are not limited in this embodiment of the present application.
- the control flow is 100Mbps, 500Mbps, 1Gbps, etc. Therefore, the embodiment of the present application focuses on n and m as The case of natural numbers will be expanded and explained.
- the unit of Mbps of the desired bandwidth BW and the unit of MHz of the clock frequency Freq are only used as an example here, which only means that the token bucket provided by the embodiment of the present application can be based on the clock frequency Freq of an integer MHz and an integer
- the desired bandwidth of Mbps precisely controls the traffic, and does not limit the unit of the desired bandwidth BW and the unit of the clock frequency Freq.
- the unit of the desired bandwidth BW and the unit of the clock frequency Freq can be changed according to user configuration.
- Equation (10) substituting Equation (8) and Equation (9) into Equation (5), the following Equation (10) can be obtained
- TokenUnit can be a decimal in order to ensure that 2 X can be obtained after multiplying TokenUnit by m (ie Freq).
- the embodiment of the present application also imposes some constraints on the packet length PktLen, that is, the packet length PktLen should satisfy 2 Y .
- the length PktLen of the message passing through the network device can be any value, which may be a fixed length or a variable length.
- the constraint of formula (14) is to ensure that each token deduction TokenSub is an integer. The specific implementation manner of constraining the packet length PktLen to satisfy the formula (14) will be further described in subsequent embodiments.
- N, X, Y are all natural numbers, and X ⁇ N, Y ⁇ X-3.
- N, X, Y are all natural numbers, and X ⁇ N, Y ⁇ X-3 can have innumerable combinations, which means that the number of token additions TokenAdd and the number of token deductions TokenSub There can be an infinite array of values.
- the following embodiments of the present application will exemplarily describe the values of N, X, and Y.
- Y can be obtained by constraining the packet length PktLen to be a natural number power of 2, that is, Y satisfies formula (17):
- Example: when the message length PktLen 32Byte is determined, the value of Y is determined to be 5; when the message length is 128Byte, the value of Y is determined to be 7; when the message length is 1024Byte, the value of Y is determined to be 10.
- X can be obtained by constraining the value of TokenUnit, and at the same time satisfy X ⁇ Y+3, that is, X satisfies formula (18):
- TokenUnit is an arbitrary value defined artificially, it is easy to determine a value of TokenUnit so that X satisfies the constraint.
- the corresponding X value can be the same or different, which is determined according to the value of the selected TokenUnit.
- the Y value limits the maximum value of X; for different Y values, the corresponding X value
- the value can also be the same or different, which is determined according to the value of the selected TokenUnit.
- the value of Y only limits the maximum value of X to be different.
- N can be obtained by constraining ⁇ CycleCnt to be a natural number power of 2, that is, N satisfies formula (19):
- ⁇ CycleCnt is an arbitrary value defined artificially, it is easy to determine a value of ⁇ CycleCnt so that N satisfies the constraint.
- N corresponds to the value of ⁇ CycleCnt
- a larger N means a larger time interval ⁇ T for adding tokens to the token bucket, and the corresponding token
- TokenAdd the smaller the N, the smaller the time interval ⁇ T for adding tokens to the token bucket, and the smaller the corresponding number of tokens to be added TokenAdd.
- the length of the message passing through the network device PktLen can be any value, it may be a fixed length or a variable length, and it may satisfy that PktLen is equal to a natural number power of 2, or it may not be equal to a natural number of 2. power, then, for different packet lengths PktLen, when the token bucket in this embodiment of the present application is implemented, different normalization processing methods may be used for packets.
- the packet length PktLen is divided into three implementation manners for detailed description below.
- the packet length PktLen is a fixed value, and the packet length PktLen is equal to a natural number power of 2.
- the packet length PktLen can be normalized to a fixed value.
- the storage unit cell is generally the smallest storage unit of the memory, and each storage unit cell can store one bit. Therefore, if the out-queue message is based on the granularity of the storage unit cell of the memory, it means that the message is processed with the granularity of bits. out of the queue. Since a bit is also the smallest unit of a message, an arbitrary message length PktLen can be normalized in terms of bit granularity. In order to facilitate the calculation of the token deduction, the embodiment of the present application can normalize the message length PktLen as Fixed value.
- N is greater than or less than X, that is, N ⁇ 10, and the value of ⁇ CycleCnt that can satisfy this constraint can be, for example, 1024, then according to formula (19), it can be obtained:
- N is greater than or less than X, that is, N ⁇ 10, and the value of ⁇ CycleCnt that can satisfy this constraint can be, for example, 1024, then according to formula (19), it can be obtained:
- Z is a positive integer, indicating that the length of the remaining packets is an integer multiple of the minimum value of the packet length.
- the packet length PktLen can be any value.
- the packet length PktLen of any value can be processed as 2 Y length as shown in formula (14) by normalization, or as 2 Y ⁇ Z as shown in formula (28) length.
- the third implementation manner is equivalent to the first implementation manner or the second implementation manner, and the methods in the first implementation manner and the second implementation manner can be used to determine the token addition number TokenAdd and the token deduction Number TokenSub.
- FIG. 4 is a flowchart of normalizing the packet length PktLen according to an embodiment of the present application.
- normalizing the packet length PktLen can be achieved by the following steps:
- step S1 a minimum packet length PktLen_min is defined, and the minimum packet length PktLen_min is a natural number power of 2.
- Step S2 defining the cumulative packet length QueLen of the packets entering the packet queue of the network device.
- the initial value of the cumulative packet length QueLen is 0.
- Step S3 when a message enters the message queue, the message length PktLen of the message is accumulated into QueLen, that is:
- the accumulated packet length QueLen can continue to accumulate, as shown in Figure 5, when packets 1, 2 and 3 enter the packet successively.
- the cumulative packet length QueLen can be the sum of the lengths of packet 1, packet 2, and packet 3.
- Step S4 when the number of tokens in the token bucket is greater than 0, the packet is allowed to go out of the queue.
- the cumulative packet length QueLen is greater than 0, and whenever the cumulative packet length QueLen is greater than or equal to a positive integer multiple of the minimum packet length PktLen_min, that is, QueLen ⁇ PktLen_min ⁇ Z,
- the token amount corresponding to the packet of PktLen_min ⁇ Z is deducted from the token bucket, that is, if the token amount in the token bucket is not enough to be deducted and the token amount is 0, the packet can be suspended from the queue.
- deduction is performed, and the packets are resumed out of the queue.
- Step S5 the network device records the accumulated packet length QueLen minus the remainder of PktLen_min ⁇ Z, that is:
- step S3 to step S5 are continued.
- the accumulated packet length QueLen is equal to the sum of the remaining length after the previous deduction and the length of packet 4. If the accumulated packet length is QueLen is greater than or equal to a positive integer multiple of the minimum packet length PktLen_min again, and the deduction continues.
- the token bucket in this embodiment of the present application can normalize the packet to a positive integer multiple of the minimum packet length PktLen_min, and use the minimum packet length PktLen_min as the minimum packet length PktLen_min.
- the token amount corresponding to the positive integer multiple of is deducted from the token, thereby realizing precise control of bandwidth.
- the token amount is deducted by the token amount corresponding to the positive integer multiple of the minimum packet length PktLen_min, in the implementation of the packet out-queue, you can make Before the card is deducted, the packets are dequeued with the minimum packet length PktLen_min in advance, which can reduce the delay between packets entering the queue and dequeuing.
- the embodiments of the present application may set different TokenUnit values and ⁇ CycleCnt values for different flow control intervals, to obtain token bucket parameters corresponding to different flow control intervals.
- These token bucket parameters can be stored in the network device in the form of a token bucket rate level table.
- the network device selects the corresponding token bucket parameter pair according to the flow control interval where the desired bandwidth BW is located. Precise control of flow.
- Rate class K3 0.5 Mbps to 4 Mbps, supporting a bandwidth value that increases with a granularity of 15.625 Kbps.
- Rate class K2 4 Mbps to 32 Mbps, supporting a bandwidth value incremented with a granularity of 125 Kbps.
- Rate class K1 32 Mbps to 256 Mbps, supporting a bandwidth value incremented with a granularity of 1 Mbps.
- Rate class K0 256 Mbps to 2 Gbps, supporting a bandwidth value incremented with a granularity of 8 Mbps.
- the expected bandwidth BW satisfies the following formula (30):
- TokenUnit ⁇ Freq is equal to a natural number power of 2
- ⁇ CycleCnt is equal to a natural number power of 2
- Cfg_Token represents the amount of tokens that need to be added to the token bucket according to the expected bandwidth BW every ⁇ CycleCnt clock cycles (time interval ⁇ T).
- Different Cfg_Token values correspond to different expected bandwidth BW, which are set in the token bucket rate level table.
- the rate level K0 can implement flow control with a granularity of 8 Mbps between the desired bandwidth of 256 Mbps to 2 Gbps.
- the rate level K0 can be determined according to the token bucket rate level table
- the calculation formula is as follows (31)
- Table 1 shown in this embodiment of the present application is only an example of the token bucket rate level table, and the parameters in the token bucket rate level table can be changed according to the changes of parameters such as the clock frequency Treq and the expected bandwidth BW range. , which does not constitute a specific limitation to the embodiments of the present application.
- any modification and improvement of the token bucket rate level table by those skilled in the art such as increasing or decreasing the rate level, adjusting any parameters, etc., or implementing the function of the token bucket rate level table in any other form, such as by The implementation of the hardware circuit, etc., does not exceed the protection scope of the embodiments of the present application.
- the present application also provides a computer storage medium.
- the computer storage medium computer instructions, when executed on a computer, cause the computer to perform the methods in the above-described aspects and implementations thereof.
- the present application also provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the methods of the above aspects and implementations thereof.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
本申请实施例提供了一种基于令牌桶的流量控制方法和网络设备,用于对通过网络设备的报文进行流量控制。周期性地向令牌桶中添加令牌,每周期内的令牌添加数符合 2 的自然数次幂的整数倍; 当有报文通过网络设备时,根据报文的长度从令牌桶中扣除令牌,其中,每次扣除时的令牌扣除数符合2的自然数次幂的整数倍。根据以上技术方案,令牌添加数和令牌扣除数均为 2 的自然数次幂的整数倍,因此可以通过硬件的移位运算精确得到,不需要再引入除法器,也不需要取近似值,因此本申请实施例提供的基于令牌桶的流量控制方法能够在消耗较低硬件资源的同时,实现流量的精确控制。
Description
本申请涉及网络技术领域,尤其涉及一种基于令牌桶的流量控制方法和网络设备。
令牌桶算法是一种常见的网络流量整形(traffic shaping)和速率限制(rate limiting)算法。令牌桶算法可以评估通过网络设备的流量速率与用户配置的额定速率之间的关系,从而根据这一关系实现流量速率的控制。
令牌桶可以形象地看作一个存放令牌的桶,网络设备设定按照用户配置的速率向桶中填充令牌,每个令牌对应一定的流量。当有流量通过网络设备时,网络设备将与流量大小相应的令牌从桶中扣减。当令牌桶中的令牌被耗尽时,流量不能再继续通过网络设备。
网络设备向令牌桶填充令牌的速率通过令牌添加数确定,每次扣减具体扣除多少令牌则有令牌扣除数确定。目前,网络设备需要依赖除法运算来计算令牌添加数和令牌扣除数,对于硬件来说,执行除法运算消耗的硬件资源较大,对硬件复杂程度和计算能力要求较高。
发明内容
本申请实施例提供了一种令牌桶的流量控制方法和网络设备,能够降低令牌桶算法对硬件资源的消耗,并且流量控制的精度更高。
第一方面,本申请实施例提供了一种基于令牌桶的流量控制方法,该方法用于对通过网络设备的报文进行流量控制,该方法包括:周期性地向令牌桶中添加令牌,每周期内的令牌添加数符合2的自然数次幂的整数倍;当有报文通过网络设备时,根据报文的长度从令牌桶中扣除令牌,其中,每次扣除时的令牌扣除数符合2的自然数次幂的整数倍。令牌添加数令牌扣除数。
根据以上技术方案,令牌添加数和令牌扣除数均为2的自然数次幂的整数倍,因此可以通过硬件的移位运算精确得到,不需要再引入除法器,也不需要取近似值,因此本申请实施例提供的基于令牌桶的流量控制方法能够在消耗较低硬件资源的同时,实现流量的精确控制。
在一种实现方式中,令牌添加数符合2的自然数次幂的整数倍,包括:令牌添加数等于2的自然数次幂与令牌桶的期望带宽的数值乘积,期望带宽为整数。这样,本申请实施例提供的基于令牌桶的流量控制方法能够实现整数级别的精准流量控制。
在一种实现方式中,令牌扣除数符合2的自然数次幂的整数倍,包括:令牌扣除数等于2的自然数次幂与令牌桶的时钟频率的数值乘积,时钟频率为整数。令牌扣除数。这样,本申请实施例提供的基于令牌桶的流量控制方法能够基于整数级别的时钟频率进行流量控制。
在一种实现方式中,令牌添加数通过以下公式得到:
其中:TokenAdd为令牌添加数;ΔCycleCnt为每一个添加令牌的周期对应的令牌桶的时钟周期的数量;n为期望带宽;TokenUnit为令牌桶的令牌单位,令牌单位表示每个令牌对应的报文长度;m为令牌桶的时钟频率;令牌添加数等于2的自然数次幂与令牌桶的期望带宽 的数值乘积,包括:时钟周期数等于2的自然数次幂,令牌单位与时钟频率的数值乘积等于2的自然数次幂。这样,本申请实施例提供的基于令牌桶的流量控制方法通过约束ΔCycleCnt、令牌单位和时钟频率之间的数值关系,使得令牌添加数符合2的自然数次幂的整数倍。
在一种实现方式中,令牌添加数和令牌扣除数还满足以下关系:
TokenAdd=n×2
(N-X)
TokenSub=m×2
3-X+Y
其中:n为期望带宽,m为时钟频率,N、X、Y均为自然数,并且有X≤N,Y≥X-3。
令牌扣除数令牌添加数令牌扣除数令牌添加数令牌扣除数在一种实现方式中,自然数Y通过以下公式得到:
Y=log
2PktLen
其中,PktLen为每一次执行令牌扣除时,通过网络设备的报文需满足的最小长度,最小长度符合2的自然数次幂。
在一种实现方式中,自然数X通过以下公式得到:
X=log
2(m×TokenUnit),(X≤Y+3)
在一种实现方式中,自然数N通过以下公式得到:
N=log
2ΔCycleCnt,(N≥X)
这样,本申请实施例提供的基于令牌桶的流量控制方法基于ΔCycleCnt、令牌单位和时钟频率之间的数值关系,对令牌扣除数和令牌添加数的计算方式进行进一步简化。
在一种实现方式中,如果通过网络设备的所有报文均为存储单元cell粒度的报文,则每当通过网络设备的多个报文的总长度等于最小长度时,执行一次令牌扣除。这样,本申请实施例提供的基于令牌桶的流量控制方法可以对基于cell传输的报文进行流量控制。
在一种实现方式中,如果通过网络设备的报文长度包括最小长度的报文和其他长度的报文,并且其他长度为最小长度的自然数倍,则每当有一个报文通过网络设备时,执行一次令牌扣除。这样,本申请实施例提供的基于令牌桶的流量控制方法可以对基于一系列倍数变化长度传输的报文进行流量控制。
在一种实现方式中,如果通过网络设备的报文为随机长度的报文,则统计通过网络设备的报文的累计长度;每当累计长度大于或者等于目标长度,且目标长度等于最小长度的自然数倍时,执行一次令牌扣除,并且从累计长度中扣除目标长度。这样,本申请实施例提供的基于令牌桶的流量控制方法可以对基于任意长度传输的报文进行流量控制。
在一种实现方式中,令牌桶还包括速率等级表,速率等级表包括至少一个流量控制区间对应的令牌桶参数,以使得网络设备根据期望带宽所属的流量控制区间确定令牌桶参数。
在一种实现方式中,每个流量控制区间包括的令牌桶参数包括以下一种或者多种:令牌单位、第一周期数量、自然数N、自然数X、带宽递增粒度、与带宽递增粒度相对应的最小令牌添加数、流量控制区间对应的令牌添加数的区间、流量控制区间对应的期望带宽的区间。这样,当用户配置某一期望带宽时,网络设备可以根据期望带宽所在的流量控制区间选择相应的令牌桶参数对流量进行精准控制。
第二方面,本申请实施例提供了一种网络设备,包括存储器和处理器,存储器存储有计算机程序指令,当计算机程序指令被处理器执行时,使得网络设备执行上述第一方面及其各实现方式中的方法。
第三方面,本申请还提供了一种计算机存储介质。该计算机存储介质计算机指令,当计 算机指令在计算机上运行时,使得计算机执行上述各方面及其实现方式中的方法。
第四方面,本申请还提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述各方面及其实现方式中的方法。
图1是令牌桶算法的原理图;
图2是本申请实施例示出的一种网络设备的结构示意图;
图3是本申请实施例对令牌桶的参数的限定条件示意图;
图4是本申请实施例示出的对报文长度PktLen进行归一化处理的流程图;
图5是本申请实施例示出的累计报文长度和令牌扣减的示意图;
图6是本申请实施例提供的令牌桶速率等级表。
令牌桶算法是一种常见的网络流量整形和速率限制算法。令牌桶算法可以评估通过网络设备的流量速率与用户配置的额定速率之间的关系,从而根据这一关系实现流量速率的控制。
图1是令牌桶算法的原理图。如图1所示,令牌桶10可以形象地看作一个存放令牌11的桶,网络设备设定按照用户配置的速率向桶中投放(或称填充)令牌11,每个令牌对应一定的流量12。当有流量12通过网络设备时,网络设备将与流量12大小相应的令牌11从桶中扣减。如果长时间没有流量12通过,则桶内的令牌11数量会达到一定上限,再此之后向桶内放入的令牌11都会被桶丢弃。如果通过网络设备的流量12较快,耗尽了桶内所有的令牌11,那么在新的令牌11放入桶内之前,流量12不能再继续通过网络设备,只能被缓存或者丢弃。由此可见,通过令牌桶算法,网络设备能够按照用户配置的速率对通过网络设备的流量速率进行控制。
为便于本领域技术人员后续理解本申请实施例的技术方案,下面对令牌桶算法涉及的相关概念进行列举说明:
1、令牌及令牌的度量单位。
①定义令牌为Token,一个令牌就是一个Token,因此Token的数值表示令牌桶内的令牌数目,这个令牌数目实质是期望通过网络设备的报文流量与实际通过网络设备的报文流量的差值。
②定义令牌的单位为TokenUnit,TokenUnit表示单位(每一个)令牌对应的网络设备能通过的流量的大小,TokenUnit通常以bit/Token(比特每令牌)为计量单位。例如,当TokenUnit=16bit/Token时,表示一个令牌对应网络设备能通过的流量为16bit,也即2Byte(字节)。
2、令牌填充的相关概念。
在令牌桶算法中,随着时间的推移,网络设备会周期性地向令牌桶内填充一定数量的令牌,简称令牌填充。令牌填充操作可以涉及以下概念:
①定义向令牌桶中填充令牌的时钟频率(即令牌桶的填桶时钟频率)为Freq(Frequency),Freq通常以赫兹(Hz)为单位,包括GHz、MHz等。例如,Freq=1000MHz。
②定义令牌桶的填桶时钟频率对应的周期为Period,Period通常以秒(s)为单位。可以理解Period与Freq互为倒数,例如,当Freq=1000Hz时,Period=0.001s。
③定义一定的时间间隔为ΔT,ΔT通常以秒(s)为单位。
④定义一定的时间间隔ΔT对应的时钟周期Period的数量为ΔCycleCnt。对于ΔCycleCnt,显然有公式(1):
⑤定义期望带宽为BW(BandWidth),其中期望带宽BW表示用户根据其期望配置的流量通过网络设备的额定带宽。期望带宽BW通常以bit/s(比特每秒)、bps(比特每秒)、Kbps(千比特每秒)、Mbps(兆比特每秒)、Gbps(吉比特每秒)为单位。
⑥定义在一定时间间隔ΔT内放入桶内的令牌添加数为TokenAdd,TokenAdd表示在一定的时间间隔ΔT内,按照期望带宽BW允许通过网络设备的报文流量所对应的令牌数量。对于TokenAdd,根据令牌填充的原理,显然有公式(2):
3、令牌扣减的相关概念。
在令牌桶算法中,当有报文通过网络设备时,需要从桶中扣减与报文流量向对应数量的令牌,简称令牌扣减。令牌扣减操作可以涉及以下概念:
①定义报文的长度为PktLen(Packet Length),这里的PktLen可以指的是对应一次令牌扣减的报文长度,可以是一条报文的长度,也可以是多条报文的长度,PktLen通常以Byte为单位。
②定义执行令牌扣减时的令牌扣除数TokenSub,TokenSub具体可以表示与一次令牌扣减的报文长度PktLen等价的令牌数量。
根据报文长度PktLen与令牌扣除数TokenSub等价的关系,可以得到以下公式(3):
在公式(3)中,由于报文长度PktLen的单位为Byte,TokenUnit的单位为bit/Token,两者不统一,因此可以首先将报文长度PktLen的单位转换成bit,根据1Byte=8bit,可以将PktLen乘以8,即转换成bit单位;接下来,再将PktLen×8除以TokenUnit,即得到令牌扣除数TokenSub。
基于上述1~3的相关概念说明,以及公式(1)~(3),令牌桶的工作原理可以进一步表述为:以期望带宽BW,每隔一段时间间隔ΔT,将数量为TokenAdd的令牌填充到桶内;当有报文通过时,将报文长度PktLen对应的令牌数量TokenSub从桶内扣减去;如果桶内的令牌被耗尽,在新的令牌放入桶内之前,报文不能再继续通过,从而实现了令牌桶算法的目标:控制报文的通过速率为期望带宽BW。
令牌桶算法是目前实现流量控制的常见手段,而评价流量控制好坏的一个最重要的指标就是流量控制的精度,如果采用令牌桶算法,这里的精度即为令牌桶的精度。在一种实现方式中,精度可以以误差率的方式体现,误差率越高则说明精度越差,误差率越高则说明精度越高。例如:精度Accurate可以通过以下公式(4)体现:
在公式(4)中,BW
exp表示用户的期望带宽,BW
cfg表示用于网络设备的硬件限制,实际能够配置出的带宽,由于网络设备的硬件限制,BW
exp与BW
cfg实际可能不相等,导致精度Accurate的值不为0,即存在误差。
一般来说,在对令牌桶的精度要求不高的场景中,令牌桶的精度Accurate达到误差小于 0.5%就可能已经满足需求了,但随着网络技术的发展,更多的领域要求令牌桶的精度达到0误差,即网络设备实际能够配置出的带宽BW
cfg和用户的期望带宽BW
exp完全相等。然而,基于目前已有的硬件,实现精度为0误差的令牌桶或者高精度的令牌桶需要消耗大量的硬件资源,对硬件自身复杂程度和计算能力要求也较高。
为便于理解令牌桶的精度与硬件资源消耗之间的关系,下面以目前常见的引入了除法器的令牌桶为例,进行示例性地解释说明。
以期望流量为BW=1Mbps的令牌桶为例,假定令牌单位TokenUnit为1bit/Token,时钟频率Freq=1GHz,则联立公式(1)和(2),并且代入参数值可以得到公式(5):
由此,在每个填充周期Period,网络设备的硬件可以根据公式5使用除法器执行除法运算精准得到需要填充的令牌数量TokenAdd。
然而,对于硬件来说,引入除法器会导致硬件资源消耗比较大。为了避免引入除法器,降低硬件资源的消耗,在一些精度不高的场景中,可以将公式(5)中的除法计算转换成硬件容易处理的计算,转换过程如公式(6)所示:
结合公式(6),具体来说,由于数据在硬件中是以2进制编码的形式存在,因此可以将公式(5)中的1000替换为与其数值接近的2的指数值,例如2
10=1024;这样,在硬件编码中,TokenAdd就可以通过对ΔCycleCnt执行移位运算得到,例如向左移动10个bit(即公式中的<<10)。
可以看出,借助硬件的移位运算可以避免采用除法器,降低令牌桶算法对硬件资源的消耗,然而,由于在算法的转换过程中涉及到了对数值的近似改变,因此公式(6)实际计算得到的TokenAdd并不能与用户的期望带宽BW精准对应。例如将1000转换为1024,会使得每个填充周期Period实际填充的令牌数量TokenAdd比预期的更小,因此令牌桶算法实际配置出的带宽比用户的期望带宽BW要低,具体精度可以由公式(7)计算得到:
由公式(7)可以看出,在期望流量为BW=1Mbps,令牌单位TokenUnit为1bit/Token,时钟频率Freq=1GHz的示例中,目前的移位运算方案的精度为2.34%,这个精度在流量控制精度要求不高的场景中可能能够应用,但是在流量控制精度要求较高的场景中,这个精度严重不足。
由此可见,基于目前的硬件,令牌桶算法无法在消耗较低硬件资源的同时,实现流量的精确控制。为了解决这一技术问题,本申请实施例提供了一种基于令牌桶的流量控制方法,也可以称作令牌桶算法,能够在消耗较低硬件资源的同时,实现流量的精确控制,例如无误差的流量控制。
本申请实施例提供的令牌桶可以基于硬件实现,该硬件例如可以是网络设备。图2是本申请实施例示出的一种网络设备的结构示意图。如图2所示,该网络设备具体可以是路由器、交换机、网络主机或者服务器等,可以作为转发设备在上游节点和下游节点之间转发报文。该网络设备可以包括存储器310、通信接口320和处理器330。存储器310、通信接口320和处理器330耦合连接,存储器310中存储有程序指令,处理器330可调用存储器310中的程序指令,使网络设备执行相关的方法,例如解析报文、生成报文,通过通信接口320接收和 转发报文、通过令牌桶等实现流量控制等。
本申请实施例中,网络设备的处理器330可以包括一个或者多个处理单元,例如***芯片(system on a chip,SoC)、中央处理器(central processing unit,CPU)、微控制器(microcontroller,MCU)、存储控制器等。其中,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。
本申请实施例中,网络设备的存储器310可以包括一个或者多个存储单元,例如可以包括易失性存储器(volatile memory),如:动态随机存取存储器(dynamic random access memory,DRAM)、静态随机存取存储器(static random access memory,SRAM)等;还可以包括非易失性存储器(non-volatile memory,NVM),如:只读存储器(read-only memory,ROM)、闪存(flash memory)等。其中,不同的存储单元可以是独立的器件,也可以集成或者封装在一个或者多个处理器或者通信接口中,成为处理器或者通信接口的一部分。
本申请实施例中,网络设备的通信接口320例如可以包括网络适配器(network adapter)、网卡(network interface card)、局域网接收器(LAN adapter)、网络接口控制器(network interface controller,NIC)、调制解调器(modem)等。其中,通信接口可以是独立的器件,也可以部分或者全部地集成或者封装在处理器中,成为处理器的一部分。
可以理解的是,本申请实施例示意的结构并不构成对网络设备的具体限定。在本申请另一些实施例中,网络设备可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件,软件或软件和硬件的组合实现。
本申请实施例提供的令牌桶算法的基本思路是将计算令牌桶填充量TokenAdd和令牌扣除数TokenSub涉及到的除法运算通过移位运算的形式实现,由此降低令牌桶算法对硬件资源的消耗,并且实现令牌桶填充量的精确计算,提高流量控制精度。为实现移位运算的目的,如图3所示,本申请实施例对令牌桶的参数进行了如下限定:
限定1:约束时间间隔ΔT对应的时钟周期数量ΔCycleCnt等于2的自然数次幂,即令牌填充的时间间隔为2的自然数次幂个时钟周期。
限定2:约束令牌单位TokenUnit和时钟频率Freq的乘积,即TokenUnit×Freq等于2的自然数次幂。
限定3:为实现约束2,TokenUnit可以是小数,以构造出TokenUnit×Freq等于2的自然数次幂。
限定4:将报文长度归化为2的自然数次幂。
这样,一方面,当令牌桶的参数满足限定1~3时,根据公式(5),令牌添加数TokenAdd的计算过程可以完全通过硬件的移位运算实现,并且不需要近似取值,因此降低了令牌桶算法对硬件资源的消耗,并且能够得到精确的整数值的令牌添加数TokenAdd;另一方面,当令牌桶的参数满足限定4时,根据公式(3),令牌扣除数TokenSub的计算过程可以完全通过硬件的移位运算实现,并且不需要近似取值,因此降低了令牌桶算法对硬件资源的消耗,并且能够得到精确的整数值的令牌扣除数TokenSub。由此可见,通过本申请实施例的限定,令牌添加数TokenAdd和令牌扣除数TokenSub均为精确的整数值,并且均通过硬件的移位计算得,到,使得令牌桶算法对硬件资源的消耗降低,并且流量控制更加精确。
下面将结合一些推导和演算过程、以及一些示例对限定1~4的确定依据进行具体说明:
首先,为了使令牌扣除数TokenSub的计算过程可以完全通过硬件的移位运算实现,本申请实施例设置ΔCycleCnt满足以下公式(8):
ΔCycleCnt=2
N,(N为自然数) (8)
可以理解的是,为了使令牌桶算法实现精准的带宽控制,令牌桶算法必须能够根据用户期望能够对令牌实现精准的令牌填充和精准的令牌扣减操作,由于硬件对令牌的添加和扣减是以整数个令牌添加和扣减的方式实现,因此要实现精准的令牌填充和精准的令牌扣减操作,公式(2)和(3)的计算结果就必须是整数,即令牌添加数TokenAdd和令牌扣除数TokenSub都必须为整数。
下面基于本申请实施例对ΔCycleCnt的限定,对令牌添加数TokenAdd和令牌扣除数TokenSub需要满足的条件进行进一步说明。
a.令牌添加数TokenAdd满足的条件:
首先设置令牌桶的期望带宽BW为n Mbps,设置令牌桶的时钟频率Freq为m MHz,这里n和m可以为自然数,也可以是其他数值,本申请实施例对此不做限定。一般来说,当用户需要设置网络设备进行流量控制时,从习惯上通常会对流量进行整数控制,例如控制流量为100Mbps、500Mbps、1Gbps等,因此本申请实施例此处着重针对n和m为自然数的情况进行展开说明。
那么,期望带宽BW和时钟频率Freq满足以下公式(9):
这里需要补充说明的是,期望带宽BW的单位Mbps和时钟频率Freq的单位MHz在此处仅作为一个示例,仅意味着本申请实施例提供的令牌桶可以基于整数MHz的时钟频率Freq和整数Mbps的期望带宽对流量进行精准控制,不构成对期望带宽BW的单位和时钟频率Freq的单位的限定,期望带宽BW的单位和时钟频率Freq的单位是可以根据用户配置而改变的。
接下来,将公式(8)和公式(9)代入公式(5),则可以得到以下公式(10)
从公式(10)可以看出,当ΔCycleCnt满足公式(8)时,如果要满足令牌添加数TokenAdd通过移位运算得到,那么TokenUnit×Freq可以等于2的自然数次幂。另外,若要进一步满足TokenAdd为整数,则可以有m×TokenUnit满足以下公式11:
m×TokenUnit=2
X(X≤N,且X为自然数) (11)
这样,由于2
N除以2
X等于2
(N-X),当N为自然数,X≤N,且X为自然数时,N-X为整数,同时n为整数,因此可以保证令牌添加数TokenAdd为整数。具体来说,将公式(11)代入到公式(10),可以得到令牌添加数TokenAdd满足以下公式(12):
b.令牌扣除数TokenSub满足的条件:
根据公式(11),有:
由于2
X除以m得到的不一定是整数,因此,为了确保TokenUnit与m(即Freq)相乘后能够得到2
X,TokenUnit可以是小数。
将公式(11-1)代入公式(3),可以得到令牌扣除数TokenSub满足以下公式(13):
从公式(13)可以看出,当m取值为任意自然数时,为了使令牌扣除数TokenSub为整数,PktLen×2
(3-X)也需要为自然数,并且,为了使令牌扣除数TokenSub通过硬件实现时可以通过移位运算实现,PktLen可以满足公式(14):
PktLen=2
Y(Y≥X-3,且Y为自然数) (14)
进一步地,联立公式(13)和(14),可以得到令牌扣除数TokenSub满足公式(15):
TokenSub=m×2
3-X+Y (15)
结合公式(14)可以看出,为了使令牌扣除数TokenSub满足整数要求,本申请实施例对报文长度PktLen也进行了一些约束,即报文长度PktLen应当满足2
Y。然而,一般来说,通过网络设备的报文长度PktLen可以为任意值,可能是固定长度,也可能是变化长度。因此,当通过网络设备的报文长度不满足PktLen=2
Y时,本申请实施例可以对网络设备的出队列的报文长度PktLen进行归一化处理,使其满足PktLen=2
Y,即满足公式(14)的约束,以保证每次令牌扣除数TokenSub为整数。关于将报文长度PktLen约束为满足公式(14)的具体实现方式将在后续的实施例中做进一步展开说明。
综上a和b所述,当用户配置令牌桶的期望带宽BW为n Mbps、时钟频率Freq为m MHz时,只需根据以下公式(16)(包括公式(12)和(15))即可计算出相应的令牌添加数TokenAdd和令牌扣除数TokenSub:
其中,N、X、Y均为自然数,并且有X≤N,Y≥X-3。
由公式(16)可以看出,由于n和m在期望带宽BW和时钟频率Freq设置好的状态下为常数,因此在硬件实现时,令牌添加数TokenAdd和令牌扣除数TokenSub可以是n和m通过左移或者右移的移位运算来精确实现,不需要再引入除法器,使得令牌桶算法能够在消耗较低硬件资源的同时,实现流量的精确控制。
进一步从公式(16)可以看出,N、X、Y均为自然数,并且X≤N,Y≥X-3可以存在无数种组合,也就意味着令牌添加数TokenAdd和令牌扣除数TokenSub可以有无数组取值。为便于本领域技术人员具体确定出令牌添加数TokenAdd和令牌扣除数TokenSub任意一组取值,本申请实施例以下对N、X、Y的取值方式进行示例性说明。
对于Y,根据公式(14),Y可以通过约束报文长度PktLen为2的自然数次幂得到,即Y满足公式(17):
Y=log
2PktLen (17)
由此可见,已确定了报文长度PktLen,Y的值即可以随机确定。
示例地:当确定报文长度PktLen=32Byte时,Y的值确定为5;当报文长度为128Byte时,Y的值确定为7;当报文长度为1024Byte时,Y的值确定为10。
对于X,根据公式(11),X可以通过约束TokenUnit的值得到,并且同时满足X≤Y+3,即X满足公式(18):
X=log
2(m×TokenUnit),(X≤Y+3) (18)
其中,由于m已知,并且TokenUnit是人为定义的任意值,因此很容易确定一个TokenUnit的值,使X满足约束。
示例地,当m=1000时:如果Y的值确定为5,则可以取TokenUnit=0.128,此时X=7;如果Y的值确定为5,则可以取TokenUnit=0.256,此时X=8;如果Y的值确定为7,则可以 取TokenUnit=0.128,此时X=7;如果Y的值确定为7,则可以取TokenUnit=0.256,此时X=8。
由此可见,对于相同的Y值,其对应的X值可以相同也可以不同,具体根据选取的TokenUnit的值确定,Y值限制了X的最大取值;对于不同的Y值,其对应的X值也可以相同或者不同,具体根据选取的TokenUnit的值确定,Y值仅限制了X的最大取值有所不同。
对于N,根据公式(8),N可以通过约束ΔCycleCnt为2的自然数次幂得到,即N满足公式(19):
N=log
2ΔCycleCnt,(N≥X) (19)
其中,由于ΔCycleCnt是人为定义的任意值,因此很容易确定一个ΔCycleCnt的值,使N满足约束。
示例地,当X=7时,由于N≥X,因此可以将ΔCycleCnt取值为256,此时N=8,或者将ΔCycleCnt取值为512,此时N=9,或者将ΔCycleCnt取值为1024,此时N=10。
由此可见,对于相同的X值,可以设置不同的N值,由于N对应的是ΔCycleCnt的取值,N越大意味向令牌桶中添加令牌的时间间隔ΔT越大,相应的令牌添加数TokenAdd越大,N越小意味向令牌桶中添加令牌的时间间隔ΔT越小,相应的令牌添加数TokenAdd越小。
可以理解的是,由于通过网络设备的报文长度PktLen可以为任意值,可能是固定长度,也可能是变化长度,可能满足PktLen等于2的自然数次幂,也可能不等于2的自然数次幂,那么,针对不同的报文长度PktLen,本申请实施例的令牌桶在实现时,对报文可以有不同的归一化处理方式。以下将报文长度PktLen分为三种实现方式进行具体展开说明。
第一种实现方式,报文长度PktLen为固定值,且满足报文长度PktLen等于2的自然数次幂。
一般来说,如果网络设备的出队列报文是基于存储器的存储单元cell粒度的,那么报文长度PktLen就可以归一化为固定值。存储单元cell一般是存储器的最小存储单元,每个存储单元cell可以存储一个比特bit,因此,如果出队列报文是基于存储器的存储单元cell粒度的,就说明报文是以比特bit为粒度进行出队列。由于比特bit也是报文的最小单位,因此在比特bit粒度上可以归一化出任意的报文长度PktLen,为便于计算令牌扣除数,本申请实施例可以将报文长度PktLen归一化为固定值。
对于第一种实现方式,以时钟频率Freq为1000MHz(m=1000)、期望带宽BW为1Mbps(n=1)、报文长度PktLen为固定值128Byte为例:
由公式(17)可得:
Y=log
2128=7 (20)
根据公式(18),X的取值大于或者小于Y+3,即X≤10,因此这里可以取值X=10,则进一步根据m=1000可得TokenUnit为1.024,则根据公式(18)可得:
X=log
2(1000×1.024)=10 (21)
根据公式(19),N的取值大于或者小于X,即N≥10,能够满足这一约束的ΔCycleCnt值例如可以为1024,则根据公式(19)可得:
N=log
21024=10 (22)
接下来,将确定的Y、X、N的值代入公式(16),则可得:
这样,对于时钟频率Freq=1000MHz、报文长度PktLen=128Byte(固定值)的实现方式,在配置令牌桶时,只要配置TokenUnit=1.024、ΔCycleCnt=1024,就可以得到期望带宽BW=1Mbps的精准令牌桶,实现对流量的精准控制。
第二种实现方式,报文长度PktLen可能具有多个值,其中,报文长度的最小值PktLen
min等于2的自然数次幂,即PktLen
min=2
Y,报文长度其余的值均为报文长度的最小值PktLen
min的任意整数倍。例如,报文长度的最小值PktLen
min=128Byte,其他的报文长度可以包括256Byte、384Byte、512Byte、640Byte、768Byte、896Byte、1024Byte等。
对于第二种实现方式,以时钟频率Freq为1000MHz(m=1000)、期望带宽BW为1Mbps(n=1)、报文长度的最小值PktLen
min=128Byte为例:
以报文长度的最小值PktLen
min=128Byte确定Y的取值,则根据公式19可得:
Y=log
2128=7 (24)
根据公式(18),X的取值大于或者小于Y+3,即X≤10,因此这里可以取值X=10,则进一步根据m=1000可得TokenUnit为1.024,则根据公式(18)可得:
X=log
2(1000×1.024)=10 (25)
根据公式(19),N的取值大于或者小于X,即N≥10,能够满足这一约束的ΔCycleCnt值例如可以为1024,则根据公式(19)可得:
N=log
21024=10 (26)
将确定的X、N的值代入公式(16),确定令牌添加数TokenAdd,可得:
TokenAdd=1×2
(10-10)=1 (27)
根据公式(25)-(27)可以看出,由于令牌桶的填充操作不涉及报文长度,在确定令牌添加数TokenAdd时,仅使用了X和N的取值,因此相比于第一种实现方式,第二种实现方式中的X和N的取值可以保持不变,这样令牌添加数TokenAdd也保持不变。对于令牌扣除操作,如公式(24)所示,可以根据报文长度的最小值PktLen
min确定Y的取值,令牌扣除数TokenSub则可以通过以下方式确定:
首先,对于除报文长度的最小值以外的其余报文长度PktLen,有:
PktLen=PktLen
min×Z=2
Y×Z (28)
其中,Z为正整数,表示其余报文长度是报文长度的最小值的整数倍。
将公式(28)代入公式(13),可以得到令牌扣除数TokenSub的计算公式变化为公式(29):
TokenSub=m×PktLen×2
3-X=m×Z×2
3-X+Y (29)
接下来,将确定的Y、X、N的值代入公式(29),则可得:
TokenSub=1000×Z×2
3-10+7=1000×Z (30)
通过对比公式(23)和公式(30)可以看出,当报文长度是最小值PktLen
min的整数倍时,令牌扣除数TokenSub实际上就是报文长度为最小值PktLen
min时对应的最小令牌扣除数(本示例中为1000)的整数倍,因此,只需以报文长度的最小值PktLen
min确定最小令牌扣除数,就可以很容易确定其余报文长度对应的令牌扣除数TokenSub。
第三种实现方式,报文长度PktLen可以是任意值。对于任意值的报文长度PktLen,可以通过归一化的方式将报文长度PktLen处理为如公式(14)所示的2
Y长度,或者处理为如公式(28)所示的2
Y×Z长度。这样,第三种实现方式就等效成了第一种实现方式或者第二种实现方式,可以采用第一种实现方式和第二种实现方式中的方法确定令牌添加数TokenAdd和令牌扣除数TokenSub。
图4是本申请实施例示出的对报文长度PktLen进行归一化处理的流程图。
如图4所示,对报文长度PktLen进行归一化处理可以通过以下步骤实现:
步骤S1,定义最小报文长度PktLen_min,并且最小报文长度PktLen_min为2的自然数次幂。
步骤S2,定义进入网络设备的报文队列的报文的累计报文长度QueLen。
累计报文长度QueLen的初始值为0。
步骤S3,当有报文进入报文队列时,将这个报文的报文长度PktLen累加至QueLen中,即:
QueLen=QueLen+PktLen。
可以理解的是,当持续有报文进入到报文队列时,累计报文长度QueLen可以持续累加,如图5所示,当有报文1、报文2和报文3先后进入到报文队列时,累计报文长度QueLen可以是报文1、报文2和报文3的长度之和。
步骤S4,当令牌桶中的令牌数量大于0时,报文允许出队列。在这种实现方式下,如图5所示,当累计报文长度QueLen大于0,并且每当累计报文长度QueLen大于或者等于最小报文长度PktLen_min正整数倍,即QueLen≥PktLen_min×Z时,就从令牌桶中扣减PktLen_min×Z的报文对应的令牌量,即:如果令牌桶中的令牌量不足以扣减以及令牌量为0时,可以暂停报文出队列,待令牌桶中的令牌量大于PktLen_min×Z的报文对应的令牌量之后,再进行扣减,并且恢复报文出队列。
步骤S5,网络设备记录累计报文长度QueLen扣除了PktLen_min×Z的剩余部分,即:
QueLen=QueLen-PktLen_min×Z
当后续有报文进入队列时,继续执行步骤S3-步骤S5。如图5所示,当后续又有报文4进入到报文队列时,累计报文长度QueLen等于上次扣减后的剩余长度与报文4的长度的累加之和,如果累计报文长度QueLen再一次大于或者等于最小报文长度PktLen_min正整数倍,则继续进行扣减。
这样,虽然到达网络设备的报文长度不是最小报文长度PktLen_min,但是本申请实施例的令牌桶可以将报文归一化到最小报文长度PktLen_min的正整数倍,以最小报文长度PktLen_min的正整数倍对应的令牌量进行令牌扣除,由此实现带宽的精准控制。
这里需要补充说明的是,在第三种实现方式中,虽然是以最小报文长度PktLen_min的正整数倍对应的令牌量进行令牌扣除,但是在报文出队列的实现中,可以在令牌扣减之前提前将报文以最小报文长度PktLen_min进行出队列,这样可以降低报文从进入队列到出队列之间的时延。
在另一种实现方式中,基于限定1~4,本申请实施例可以针对不同的流量控制区间设置不同的TokenUnit值、ΔCycleCnt值,得到不同流量控制区间对应的令牌桶参数。这些令牌桶参数可以采用令牌桶速率等级表的方式存储在网络设备中,当用户配置某一期望带宽BW时,网络设备根据期望带宽BW所在的流量控制区间选择相应的令牌桶参数对流量进行精准控制。
以时钟频率Freq=1000MHz为例,一种令牌桶速率等级表如图6所示:
从图6示出的令牌桶速率等级表可以看出:令牌桶的整体流量控制区间从0.5Mbps~2Gbps被划分成了四个速率等级,分别为:
速率等级K3:0.5Mbps~4Mbps,支持以15.625Kbps的粒度递增的带宽值。
速率等级K2:4Mbps~32Mbps,支持以125Kbps的粒度递增的带宽值。
速率等级K1:32Mbps~256Mbps,支持以1Mbps的粒度递增的带宽值。
速率等级K0:256Mbps~2Gbps,支持以8Mbps的粒度递增的带宽值。
根据令牌桶速率等级表,期望带宽BW满足以下公式(30):
下面以速率等级K0为例,对令牌桶速率等级表中的各个参数进行具体解释说明:
其中:
为满足TokenUnit×Freq等于2的自然数次幂,速率等级K0中确定TokenUnit=1.024。
为满足ΔCycleCnt等于2的自然数次幂,速率等级K0中确定ΔCycleCnt=128。
根据公式(8),可以确定N=7。根据公式(18)可以确定X=10,此时,X>N。根据公式(12),当X>N时,若n为任意值,则不能保证始终TokenAdd为正整数,因此可以需要对n进行限定,如果TokenAdd取最小值1,根据公式(12)可以确定n有最小值为n=8,这个n的最小值可以被称作带宽递增粒度,意味着速率等级K0对应的带宽递增粒度n=8Mbps。可见,TokenAdd=1是与带宽递增粒度n=8Mbps相对应的最小令牌添加数。
另外,Cfg_Token表示每ΔCycleCnt个时钟周期(时间间隔ΔT)按照期望带宽BW需要向令牌桶中添加的令牌量,不同的Cfg_Token的值对应不同的期望带宽BW,令牌桶速率等级表中设置Cfg_Token在32~255之间,最小值Cfg_Token_min=32,对应期望带宽的最小值BW_min=256Mbps,最大值Cfg_Token_max=255,对应期望带宽的最大值BW_max=2040Mbps。这样,速率等级K0可以实现在期望带宽256Mbps~2Gbps之间以8Mbps为粒度进行流量控制。
示例地,当期望带宽为800Mbps,根据令牌桶速率等级表可以确定采用速率等级K0,根据公式(30)可以确定Cfg_Token=100,计算公式如(31)
其他速率等级中的各个参数的确定方式与速率等级K0类似,此处不再赘述。需要补充说明书,本申请实施例示出的表1仅仅是令牌桶速率等级表的一个示例,令牌桶速率等级表中的参数可以根据时钟频率Treq、期望带宽BW范围等参数的变化而相应变化,不构成对本申请实施例的具体限定。因此,本领域技术人员对令牌桶速率等级表的任何修改和改进,例如增加或者减少速率等级、调整任意参数等,或者,将令牌桶速率等级表的功能以其他任意形式实现,例如通过硬件电路实现等,均没有超出本申请实施例的保护范围。
本申请还提供了一种计算机存储介质。该计算机存储介质计算机指令,当计算机指令在计算机上运行时,使得计算机执行上述各方面及其实现方式中的方法。
本申请还提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述各方面及其实现方式中的方法。
以上内容,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何在本申请揭露的技术范围内的变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以权利要求的保护范围为准。
Claims (13)
- 一种基于令牌桶的流量控制方法,其特征在于,用于对通过网络设备的报文进行流量控制,所述方法包括:周期性地向令牌桶中添加令牌,每周期内的令牌添加数符合2的自然数次幂的整数倍;当有报文通过网络设备时,根据所述报文的长度从令牌桶中扣除令牌,其中,每次扣除时的令牌扣除数符合2的自然数次幂的整数倍。
- 根据权利要求1所述的方法,其特征在于,所述令牌添加数符合2的自然数次幂的整数倍,包括:所述令牌添加数等于2的自然数次幂与令牌桶的期望带宽的数值乘积,所述期望带宽为整数。
- 根据权利要求2所述的方法,其特征在于,所述令牌扣除数符合2的自然数次幂的整数倍,包括:所述令牌扣除数等于2的自然数次幂与令牌桶的时钟频率的数值乘积,所述时钟频率为整数。
- 根据权利要求2-4任一项所述的方法,其特征在于,所述令牌添加数和所述令牌扣除数还满足以下关系:TokenAdd=n×2 (N-X)TokenSub=m×2 3-X+Y其中:n为期望带宽,m为时钟频率,N、X、Y均为自然数,并且有X≤N,Y≥X-3。
- 根据权利要求5所述的方法,其特征在于,所述自然数Y通过以下公式得到:Y=log 2PktLen;其中:PktLen为每一次执行令牌扣除时,通过网络设备的报文需满足的最小长度,所述最小长度符合2的自然数次幂。
- 根据权利要求5所述的方法,其特征在于,所述自然数X通过以下公式得到:X=log 2(m×TokenUnit),(X≤Y+3)。
- 根据权利要求5所述的方法,其特征在于,所述自然数N通过以下公式得到:N=log 2ΔCycleCnt,(N≥X)。
- 根据权利要求6所述的方法,其特征在于,如果通过网络设备的所有报文均为存储单元cell粒度的报文,则每当通过网络设备的多个报文的总长度等于所述最小长度时,执行一次令牌扣除。
- 根据权利要求6所述的方法,其特征在于,如果通过网络设备的报文长度包括所述最小长度的报文和其他长度的报文,并且所述其他长度为所述最小长度的自然数倍,则每当有一个报文通过网络设备时,执行一次令牌扣除。
- 根据权利要求6所述的方法,其特征在于,如果通过网络设备的报文为随机长度的报文,则统计通过网络设备的报文的累计长度;每当所述累计长度大于或者等于目标长度,且所述目标长度等于所述最小长度的自然数倍时,执行一次令牌扣除,并且从所述累计长度中扣除所述目标长度。
- 根据权利要求2-11任一项所述的方法,其特征在于,所述令牌桶还包括速率等级表,所述速率等级表包括至少一个流量控制区间对应的令牌桶参数,以使得所述网络设备根据所述期望带宽所属的流量控制区间确定所述令牌桶参数。
- 一种网络设备,其特征在于,包括存储器和处理器,所述存储器存储有计算机程序指令,当所述计算机程序指令被所述处理器执行时,使得所述网络设备执行权利要求1-12任一项所述的方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2021/091845 WO2022232976A1 (zh) | 2021-05-06 | 2021-05-06 | 一种基于令牌桶的流量控制方法和网络设备 |
CN202180089637.0A CN116686256A (zh) | 2021-05-06 | 2021-05-06 | 一种基于令牌桶的流量控制方法和网络设备 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2021/091845 WO2022232976A1 (zh) | 2021-05-06 | 2021-05-06 | 一种基于令牌桶的流量控制方法和网络设备 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022232976A1 true WO2022232976A1 (zh) | 2022-11-10 |
Family
ID=83932601
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/091845 WO2022232976A1 (zh) | 2021-05-06 | 2021-05-06 | 一种基于令牌桶的流量控制方法和网络设备 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN116686256A (zh) |
WO (1) | WO2022232976A1 (zh) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1536815A (zh) * | 2003-04-03 | 2004-10-13 | 华为技术有限公司 | 采用令牌漏桶进行报文限流的方法 |
US20060268719A1 (en) * | 2005-05-26 | 2006-11-30 | Hitachi Communication Technologies, Ltd. | Packet forwarding apparatus using token bucket algorithm and leaky bucket algorithm |
CN102739531A (zh) * | 2012-06-19 | 2012-10-17 | 华为技术有限公司 | 流量整形方法和流量整形设备 |
CN102970238A (zh) * | 2012-11-22 | 2013-03-13 | 华为技术有限公司 | 一种流量控制的方法和设备 |
CN103197754A (zh) * | 2013-04-01 | 2013-07-10 | 华为技术有限公司 | 一种降低芯片功耗的方法和装置 |
CN107204930A (zh) * | 2016-03-17 | 2017-09-26 | 深圳市中兴微电子技术有限公司 | 令牌添加方法及装置 |
-
2021
- 2021-05-06 CN CN202180089637.0A patent/CN116686256A/zh active Pending
- 2021-05-06 WO PCT/CN2021/091845 patent/WO2022232976A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1536815A (zh) * | 2003-04-03 | 2004-10-13 | 华为技术有限公司 | 采用令牌漏桶进行报文限流的方法 |
US20060268719A1 (en) * | 2005-05-26 | 2006-11-30 | Hitachi Communication Technologies, Ltd. | Packet forwarding apparatus using token bucket algorithm and leaky bucket algorithm |
CN102739531A (zh) * | 2012-06-19 | 2012-10-17 | 华为技术有限公司 | 流量整形方法和流量整形设备 |
CN102970238A (zh) * | 2012-11-22 | 2013-03-13 | 华为技术有限公司 | 一种流量控制的方法和设备 |
CN103197754A (zh) * | 2013-04-01 | 2013-07-10 | 华为技术有限公司 | 一种降低芯片功耗的方法和装置 |
CN107204930A (zh) * | 2016-03-17 | 2017-09-26 | 深圳市中兴微电子技术有限公司 | 令牌添加方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
CN116686256A (zh) | 2023-09-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Ciucu et al. | Scaling properties of statistical end-to-end bounds in the network calculus | |
Ciucu et al. | A network service curve approach for the stochastic analysis of networks | |
CN109039936B (zh) | 传输速率控制方法、装置、发送设备和接收设备 | |
JP5048184B2 (ja) | 伝送レート監視装置および伝送レート監視方法 | |
EP4024778A1 (en) | Method for determining required bandwidth for data stream transmission, and devices and system | |
TWI499242B (zh) | 用以減少封包處理線卡中之能量耗損的方法 | |
JP2005151558A (ja) | Rpr公平メカニズムを用いた帯域幅の割当方法 | |
WO2013000116A1 (zh) | 一种漏桶限速方法及装置 | |
WO2020142867A1 (zh) | 一种流量整形方法及相关设备 | |
US20220311711A1 (en) | Congestion control based on network telemetry | |
Drobisz et al. | Adaptive sampling methods to determine network traffic statistics including the hurst parameter | |
WO2022232976A1 (zh) | 一种基于令牌桶的流量控制方法和网络设备 | |
Lombardo et al. | Achieving energy savings and QoS in internet access routers | |
Sun et al. | RFBBR: a RTT faireness awared algorithm based on BBR | |
WO2021254475A1 (zh) | 一种用于调度队列的方法及装置 | |
Zhou et al. | Fast-converging congestion control in datacenter networks | |
Yang et al. | BFRP: Endpoint congestion avoidance through bilateral flow reservation | |
CN114980155A (zh) | 一种面向6TiSCH网络的周期性数据流调度聚合方法 | |
Ma et al. | An efficient resource allocation strategy for three-layer networks | |
Luo et al. | BO-ARED: A new AQM algorithm with adaptive adjustment of parameters | |
WO2022012442A1 (zh) | 一种数据采集方法及设备 | |
RU218141U1 (ru) | Устройство вычисления скорости работы модема с гарантией максимальной задержки для потока переменной битовой скорости | |
Rácz et al. | Connection admission control for flow level QoS in bufferless models | |
JP2000183880A (ja) | パケット流量監視装置 | |
Su et al. | DVPTCP: A delay-driven virtual parallel TCP for high-speed and lossy networks |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21939624 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202180089637.0 Country of ref document: CN |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21939624 Country of ref document: EP Kind code of ref document: A1 |