CN117478589A

CN117478589A - Service flow forwarding method, device and system based on DPU

Info

Publication number: CN117478589A
Application number: CN202311287228.9A
Authority: CN
Inventors: 王绍坤; 黄明亮; 鄢贵海
Original assignee: Yusur Technology Co ltd
Current assignee: Yusur Technology Co ltd
Priority date: 2023-10-07
Filing date: 2023-10-07
Publication date: 2024-01-30

Abstract

The application provides a service flow forwarding method, device and system based on a DPU, which are applied to the technical field of DPUs. The method comprises the following steps: receiving a downlink business flow message from the outside of the DPU in the DPU; if the forwarding object of the downlink service flow message is determined to be a service grid forwarding plane arranged in the DPU, the downlink service flow message is sent to a hardware protocol stack in the DPU, so that the hardware protocol stack forwards the downlink service flow message to the service grid forwarding plane based on a downlink TCP connection path constructed between the hardware protocol stack and the service grid forwarding plane in advance. The service grid service flow forwarding method and device can effectively improve the service grid service flow forwarding efficiency on the basis of not occupying host side resources, and can guarantee the reliability and the effectiveness of the service flow forwarding process.

Description

Service flow forwarding method, device and system based on DPU

Technical Field

The present disclosure relates to the field of DPU technologies, and in particular, to a service traffic forwarding method, device, and system based on a DPU.

Background

A Service Mesh (Service Mesh) is an infrastructure layer for communication between cloud native micro-services, responsible for network communication between micro-services, and is connected to each other. The centralized service grid mode is that all micro services share one service grid, so that host resources can be greatly reduced, and a part of time delay and efficiency are improved. The service grid is offloaded to the DPU for use in connection with network communications between host-side micro-services and flow control to and from the micro-services. The method does not occupy host resources and can improve the deployment quantity of micro services at the host side; more importantly, the traffic can be distributed to a certain micro service at the host side or forwarded to other devices on the DPU without being sent to the host side, and the network delay and forwarding efficiency can be greatly improved.

However, in the existing traffic forwarding method, although the service grid is offloaded onto the DPU, the base is based on a software protocol stack, such as a kernel Iptables, and the like, and a user mode software protocol stack is adopted in the prior art. Compared with the mode of unloading the service grid to the DPU, the delay forwarding capability of the method is improved, but no breakthrough is made, so that the service traffic forwarding efficiency is still limited. If the software protocol stack is directly replaced, the service grid business flow forwarding can not be reliably realized.

Based on this, there is a need to design a way to further improve the forwarding efficiency of service grid traffic and to ensure the reliability of the traffic forwarding process.

Disclosure of Invention

In view of this, embodiments of the present application provide a method, an apparatus, and a system for forwarding traffic based on a DPU, so as to eliminate or improve one or more drawbacks existing in the prior art.

One aspect of the present application provides a service traffic forwarding method based on a DPU, including:

receiving a downlink business flow message from the outside of the DPU in the DPU;

if the forwarding object of the downlink service flow message is determined to be a service grid forwarding plane arranged in the DPU, the downlink service flow message is sent to a hardware protocol stack in the DPU, so that the hardware protocol stack forwards the downlink service flow message to the service grid forwarding plane based on a downlink TCP connection path constructed between the hardware protocol stack and the service grid forwarding plane in advance.

In some embodiments of the present application, before the sending the downlink traffic packet to the hardware protocol stack in the DPU, the method further includes:

searching a corresponding forwarding object from a local forwarding table according to the attribute information corresponding to the issued service flow message, wherein the forwarding object comprises the service grid forwarding plane;

the forwarding table is received from the service grid forwarding surface in advance and stored locally, and is used for storing the corresponding relation between each attribute information and the forwarding object.

In some embodiments of the present application, if it is determined that the forwarding object of the downlink traffic packet is a service grid forwarding plane set in the DPU, the sending the downlink traffic packet to a hardware protocol stack in the DPU, so that the hardware protocol stack forwards the downlink traffic packet to the service grid forwarding plane based on a downlink TCP connection path previously constructed between the hardware protocol stack and the service grid forwarding plane, including:

if the forwarding object of the downlink service flow message is determined to be a service grid forwarding plane arranged in the DPU, the downlink service flow message is sent to a hardware protocol stack in the DPU, so that the hardware protocol stack selects one of all threads in the service grid forwarding plane as a current target thread, and forwards the downlink service flow message to the target thread in the service grid forwarding plane based on a downlink TCP connection path constructed between the service grid forwarding plane in advance.

In some embodiments of the present application, further comprising:

receiving an uplink service flow message sent by the hardware protocol stack, wherein the uplink service flow message is obtained by encapsulating service flow data received in advance based on an uplink TCP connection path by the hardware protocol stack; the uplink TCP connection path is established between the target thread and the hardware protocol stack after receiving the downlink service flow message and determining the destination endpoint information of the host side based on preset configuration rule data, and then the target thread sends the service flow data corresponding to the downlink service flow message and the destination endpoint information processed by the target thread to the hardware protocol stack based on the uplink TCP connection path;

and sending the uplink service flow message to a destination endpoint of the host side based on the destination endpoint information so that a destination container group corresponding to the destination endpoint information acquires the uplink service flow message.

In some embodiments of the present application, the sending TCP connection path and the sending TCP connection path are pre-configured by the hardware protocol stack when or after receiving a connection request signal sent by the service grid forwarding plane, and sending a TCP protocol packet of the hardware protocol stack to the service grid forwarding plane according to the connection request signal.

In some embodiments of the present application, the configuration rule data is sent to the service grid forwarding plane in advance by a service grid management plane at the host side and stored in the service grid forwarding plane.

Another aspect of the present application provides a service traffic forwarding device based on a DPU, including:

the message receiving module is used for receiving a downlink service flow message sent from the outside of the DPU in the DPU;

and the message forwarding module is used for sending the downlink service flow message to a hardware protocol stack in the DPU if the forwarding object of the downlink service flow message is determined to be a service grid forwarding plane arranged in the DPU, so that the hardware protocol stack forwards the downlink service flow message to the service grid forwarding plane based on a downlink TCP connection path constructed between the hardware protocol stack and the service grid forwarding plane in advance.

A third aspect of the present application provides a service traffic forwarding system based on a DPU, including: the system comprises a network repeater, a hardware protocol stack and a service grid forwarding plane, wherein the network repeater, the hardware protocol stack and the service grid forwarding plane are respectively arranged in a DPU and are sequentially in communication connection;

the network repeater is used for executing the service flow forwarding method based on the DPU;

The hardware protocol stack is in communication connection with the service grid forwarding plane based on a preset PCIE;

the service grid forwarding surface is in communication connection with the service grid management surface of the host side based on a preset management port, and the network forwarder is in communication connection with each container group in the container cluster of the host side based on a preset interface through network card.

In some embodiments of the present application, the hardware protocol stack includes:

the TCP connection establishment module is used for sending a TCP protocol message of the TCP connection establishment module to the service grid forwarding plane according to the connection request signal when or after receiving the connection request signal sent by the service grid forwarding plane so as to construct a corresponding transmission TCP connection path or an uplink TCP connection path;

the issuing forwarding module is used for selecting one of the threads in the service grid forwarding plane as a current target thread and forwarding the issuing service flow message to the target thread in the service grid forwarding plane based on an issuing TCP connection path constructed between the issuing service flow message and the service grid forwarding plane in advance;

and the uploading forwarding module is used for receiving the service flow data sent by the target thread in the service grid forwarding surface based on an uploading TCP connection path constructed between the service grid forwarding surface in advance, packaging the service flow data to obtain a corresponding uploading service flow message, and sending the uploading service flow message to the destination endpoint information of the host side so as to enable a destination container group corresponding to the destination endpoint information to obtain the uploading service flow message.

In some embodiments of the present application, the service grid forwarding plane includes:

the configuration receiving module is used for receiving and storing configuration rule data sent by the service grid management surface of the host side;

the TCP connection request module is used for sending a connection request signal to the hardware protocol stack so that the hardware protocol stack sends out a TCP protocol message of the hardware protocol stack according to the connection request signal to construct a corresponding transmission TCP connection path or an uplink TCP connection path with the hardware protocol stack;

the message receiving module is used for receiving the service flow issuing message forwarded by the hardware protocol stack based on a local target thread;

and the message uploading module is used for determining the destination endpoint information of the host side based on preset configuration rule data, and after an uploading TCP connection path is established between the TCP connection request module and the hardware protocol stack, the target thread sends the service flow data corresponding to the issued service flow message and the destination endpoint information which are processed by the target thread based on the uploading TCP connection path to the hardware protocol stack.

A fourth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the DPU-based traffic forwarding method when executing the computer program.

A fifth aspect of the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the described DPU-based traffic forwarding method.

The service flow forwarding method based on the DPU, provided by the application, receives a downlink service flow message sent from the outside of the DPU in the DPU; if the forwarding object of the service flow message is determined to be a service grid forwarding plane arranged in the DPU, the service flow message is sent to a hardware protocol stack in the DPU, so that the hardware protocol stack forwards the service flow message to the service grid forwarding plane based on a forwarding TCP connection path constructed between the service grid forwarding plane in advance, host side resources are not occupied, host side resources can be used for service deployment, host side resources can be greatly saved, forwarding efficiency of service grid service flow can be effectively improved, namely, delay of a forwarding process can be greatly reduced, service grids can support time delay sensitive service scenes such as financial telecommunication and the like, and reliability and effectiveness of a service flow forwarding process can be ensured.

Additional advantages, objects, and features of the application will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.

It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present application are not limited to the above-detailed description, and that the above and other objects that can be achieved with the present application will be more clearly understood from the following detailed description.

Drawings

The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this application. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the application. Corresponding parts in the drawings may be exaggerated, i.e. made larger relative to other parts in an exemplary device actually manufactured according to the present application, for convenience in showing and describing some parts of the present application. In the drawings:

fig. 1 is an exemplary schematic diagram of a conventional side car pattern service grid deployed at a host.

Fig. 2 is an exemplary schematic diagram of a service grid offloaded onto a DPU and forwarding traffic using software.

Fig. 3 is a schematic flow chart of a first service flow forwarding method based on a DPU in an embodiment of the present application.

Fig. 4 is a schematic flow chart of a second method for forwarding traffic based on a DPU in an embodiment of the present application.

Fig. 5 is a schematic architecture diagram of a DPU-based traffic forwarding device in an embodiment of the present application.

Fig. 6 is a schematic architecture diagram of a DPU-based traffic forwarding system in an embodiment of the present application.

Fig. 7 is a schematic diagram of an architecture of a hardware protocol stack according to an embodiment of the present application.

Fig. 8 is a schematic architecture diagram of a service grid forwarding plane in an embodiment of the present application.

Fig. 9 is a schematic diagram illustrating an example of communication between a DPU-side service grid forwarding plane and a host-side service grid management plane in an application example of the present application.

Fig. 10 is an exemplary schematic diagram of a service traffic forwarding process based on a DPU implemented by using the service traffic forwarding system based on a DPU in an application example of the present application.

Detailed Description

For the purposes of making the objects, technical solutions and advantages of the present application more apparent, the present application will be described in further detail with reference to the embodiments and the accompanying drawings. The exemplary embodiments of the present application and their descriptions are used herein to explain the present application, but are not intended to be limiting of the present application.

It should be noted here that, in order to avoid obscuring the present application due to unnecessary details, only structures and/or processing steps closely related to the solution according to the present application are shown in the drawings, while other details not greatly related to the present application are omitted.

It should be emphasized that the term "comprises/comprising" when used herein is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.

It is also noted herein that the term "coupled" may refer to not only a direct connection, but also an indirect connection in which an intermediate is present, unless otherwise specified.

Hereinafter, embodiments of the present application will be described with reference to the drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

In conventional micro-service systems, the service grid mostly adopts a mode of deploying a side car or a centralized service grid on the host side. Referring to fig. 1, the initial service grid adopts a side car deployment mode, namely, each micro service is configured with a side car; the method has the defects that the side car can occupy a large amount of resources of the host machine, and the micro-service time delay and the flow processing efficiency can be greatly reduced. Then, a centralized service grid mode is proposed, and all micro services share one service grid, so that host resources can be greatly reduced, and a part of time delay and efficiency are improved.

However, the service grid is taken as a side car, and a large amount of resources of the host side are occupied, including memory and a CPU, so that the number of deployable services is reduced. That is, while the service grid provides rich traffic management, it is at the expense of latency and is not suitable for low latency required traffic scenarios. Thus, a centralized service grid is derived that is accelerated by offloading the service grid to hardware.

In particular, the service grid is offloaded onto the DPU for use in charge of network communications between host-side micro-services and flow control into and out of the micro-services. The method does not occupy host resources and improves the deployment quantity of micro-services at the host side; more importantly, the traffic can be distributed to a micro service at the host side or forwarded to other devices on the DPU without being sent to the host side, so that the network delay and forwarding efficiency can be improved. For example, referring to the scheme shown in fig. 2, the Envoy serving as the Service grid forwarding plane is sunk onto the DPU card from the side of the Host side node (Service a), and can serve the Host side node (Service a) without occupying the Host side resource, so that the utilization rate of the Host side resource can be greatly improved. Meanwhile, a large part of the traffic can be directly forwarded on the DPU card without being sent to a host side node (Service A) for processing, one hop is saved, and therefore network delay can be further reduced, and forwarding efficiency is improved. Wherein, iptables is an IP packet filtering system integrated with a Linux kernel. The envoy is a communication bus and a general data plane designed for a large micro-Service Mesh architecture.

In fig. 2, an IPU work Node (IPU Worker Node) in the image processing unit IPU (Image Processing Unit) is provided with an envoy as a service grid forwarding plane, and an IPU kernel (IPU data plane) is Iptables. Virtual network devices veth are arranged between the host side node (Service A) and the kernel Iptables and between the kernel Iptables and the envoy. Istio is used as a service grid management surface, and a container cluster can adopt Kubernetes (K8 s for short), which is an open source application for managing containerization on a plurality of hosts in a cloud platform. Specifically, the service grid in fig. 2 is offloaded to the DPU, and the specific procedure for forwarding traffic using software is as follows:

1. exposing IPU computing resources as k8s working nodes;

2. modifying the business flow of k8, and injecting Envoy into the IPU working node;

3. configuring IPU data plane distribution to realize redirection of iptables flow;

4. editing tenant service;

the Istio configuration of Envoy.

However, even though the service grid is offloaded to the DPU, its base is based on a software protocol stack (as in FIG. 2, kernel iptables is used), and some techniques use a user-mode software protocol stack. The delay forwarding capability is improved, but no breakthrough is made, and the bottleneck is still formed. And thus are not suitable for use in financial, banking, etc. scenarios where extremely low latency is required.

Therefore, the designer of the application first considers replacing the existing software protocol stack with the hardware protocol stack to further improve the forwarding efficiency of the service grid service traffic. However, this direct post-replacement application manner cannot implement the service mesh forwarding plane to call the hardware protocol stack, and cannot accurately upload the traffic arriving at the DPU to the service mesh forwarding plane.

The concrete explanation is as follows:

first, envoy calls standard socket interface to connect local protocol stack when using protocol stack. Because the hardware side is not in the Envoy local, the mode of directly replacing the software protocol stack with the hardware protocol stack cannot complete the call of the protocol stack. Therefore, the designer of the application further designs an interaction mode between the Envoy side and the hardware protocol stack through creative labor, namely, a TCP connection path is constructed between the Envoy side and the service grid forwarding plane.

And, since the traffic on the host side comes out of the container group and is a TCP/IP packet, it cannot be directly determined whether the packet should be a packet sent to Envoy, and thus it cannot accurately send the traffic arriving at the DPU to the service grid forwarding plane. Based on this, the designer of the present application further designs a way to identify whether the TCP/IP packet from the container set needs to be sent to Envoy, if so, send the traffic to Envoy after the DPU hardware protocol stack is processed.

Based on this, in order to further improve the forwarding efficiency of service grid service traffic and ensure the reliability of the service traffic forwarding process, the embodiments of the present application respectively provide a service traffic forwarding method based on a DPU, a service traffic forwarding device based on a DPU and used for executing the service traffic forwarding method based on a DPU, an entity device, and a computer readable storage medium, and further provide a service traffic forwarding system based on a DPU, which can effectively improve the forwarding efficiency of service grid service traffic on the basis of not occupying host side resources, and can ensure the reliability and the effectiveness of the service traffic forwarding process.

The following examples are provided to illustrate the invention in more detail.

In one or more embodiments of the present application, DPU (Data Processing Unit) is a data processor, which is a new generation of data-centric computing oriented processor, that integrates the functionality of a complete data center into a single chip. The System On Chip (SOC) mentioned in the embodiments of the present application may be an SOC, which is a highly integrated electronic System, that integrates a plurality of functional modules into a single Chip, thereby improving System performance, reducing power consumption, and reducing physical dimensions. TCP (Transmission Control Protocol) refers to the transmission control protocol, a connection-oriented, reliable, byte stream based transport layer communication protocol, defined by IETF RFC 793. FIN (Finish) indicates closing the connection. gRPC (Google Remote Procedure Call) refers to Google remote procedure call, which is a high-performance, general open source RPC framework published by Google. PF (Physical Function) is a physical card. SR-IOV (Single Root I/O Virtualization) refers to Single Root input/output Virtualization, which is a hardware-accelerated Virtualization technology that allows multiple virtual machines to access physical devices simultaneously, thereby improving performance and reliability of the virtual machines.

Based on this, the embodiment of the application provides a service traffic forwarding method based on a DPU, which can be implemented by a service traffic forwarding device based on a DPU, referring to fig. 3, the service traffic forwarding method based on a DPU specifically includes the following contents:

step 100: and receiving a downlink business flow message sent from the outside of the DPU in the DPU.

In one or more embodiments of the present application, the functionality of the DPU-based traffic forwarding device may be implemented in a network repeater.

In step 100, an outgoing traffic message received by a traffic forwarding device based on a DPU in the DPU is sent from outside the DPU. It will be appreciated that the DPU externally comprises a host side, for example in particular a certain group Pod of containers Pod of the host side container cluster. The DPU exterior also includes other bodies located outside the DPU that are not host side. That is, after the service grid forwarding plane is unloaded to the DPU, the traffic of the host micro-service can be sent to the DPU side for processing, and the external traffic can also be directly processed and forwarded on the DPU without uploading the traffic to the host side for processing, so that one hop to the host side is saved, and the network delay can be greatly reduced by directly processing the message by the network card.

It may be understood that the downlink service flow packet and the uplink service flow packet mentioned in the subsequent embodiment may refer to a TCP/IP packet corresponding to the service flow data.

Step 200: if the forwarding object of the downlink service flow message is determined to be a service grid forwarding plane arranged in the DPU, the downlink service flow message is sent to a hardware protocol stack in the DPU, so that the hardware protocol stack forwards the downlink service flow message to the service grid forwarding plane based on a downlink TCP connection path constructed between the hardware protocol stack and the service grid forwarding plane in advance.

In step 200, the service traffic forwarding device based on the DPU sends the issued service traffic packet to the hardware protocol stack in the DPU only after determining that the forwarding object of the issued service traffic packet is the service grid forwarding plane set in the DPU, so that the traffic arriving at the DPU can be accurately sent to the service grid forwarding plane, and further the reliability and accuracy of the service traffic forwarding process can be effectively improved.

Meanwhile, a transmission TCP connection path is constructed between the hardware protocol stack and the service grid forwarding plane in advance, and then the hardware protocol stack forwards the transmission service flow message to the service grid forwarding plane based on the transmission TCP connection path, so that the service grid forwarding plane can call the hardware protocol stack, and further the reliability and the effectiveness of the service flow forwarding process can be further ensured.

Based on the above, through the improvement, the service flow forwarding method based on the DPU provided by the embodiment of the present application can implement the protocol stack processing of supporting the service grid forwarding plane by replacing the software protocol stack with the hardware protocol stack, thereby effectively playing the hardware advantage and greatly reducing the delay of the forwarding process. The service grid originally supports the internet service with large bandwidth but without time delay, but the forwarding mode provided by the embodiment of the application can greatly reduce the time delay, so that the service grid can support the service scene sensitive to the time delay, such as financial telecommunication scenes and the like.

As can be seen from the foregoing description, the service traffic forwarding method based on the DPU provided by the embodiments of the present application does not occupy the host side resources, so that the host side resources can be used for service deployment, a large amount of host side resources can be saved, the forwarding efficiency of service grid service traffic can be effectively improved, that is, the delay of the forwarding process can be greatly reduced, the service grid can support service scenarios sensitive to time delay, such as financial telecommunication, and the reliability and effectiveness of the service traffic forwarding process can be ensured.

In order to further improve the effectiveness and reliability of determining whether the traffic is processed by Envoy, in the service traffic forwarding method based on the DPU provided in the embodiment of the present application, referring to fig. 4, between step 100 and step 200 in the service traffic forwarding method based on the DPU, the following contents are specifically included:

Step 110: searching a corresponding forwarding object from a local forwarding table according to the attribute information corresponding to the issued service flow message, wherein the forwarding object comprises the service grid forwarding plane; the forwarding table is received from the service grid forwarding surface in advance and stored locally, and is used for storing the corresponding relation between each attribute information and the forwarding object.

In a specific example, the attribute information corresponding to the downlink traffic packet may be extracted from the received downlink traffic packet by the service traffic forwarding device based on the DPU, and the attribute information may be at least one of five tuples of the downlink traffic packet. It is understood that the five-tuple includes: a source IP address, a source port, a destination IP address, a destination port, and a transport layer protocol.

Based on this, the forwarding table is used to store correspondence between different attribute information and the forwarding object. For example, the forwarding table stores correspondence between each different source IP address and the forwarding object.

It may be understood that the forwarding object stored in the forwarding table may only include the service grid forwarding plane, and if the attribute information corresponding to the downlink service traffic packet is not found in the forwarding table, it may be directly determined that the forwarding object of the downlink service traffic packet is not the service grid forwarding plane, and then step 200 is not executed.

In order to further improve reliability and effectiveness of forwarding the traffic, at least two forwarding objects stored in the forwarding table include the service grid forwarding plane and at least one other main body with data forwarding capability.

Before step 100, the service grid forwarding plane sends the corresponding relationship data between each attribute information and the forwarding object to the service traffic forwarding device based on the DPU in advance, so as to store the corresponding relationship data in a local forwarding table.

In order to further improve the efficiency of service traffic forwarding, in the service traffic forwarding method based on a DPU provided in the embodiment of the present application, referring to fig. 4, step 200 in the service traffic forwarding method based on a DPU specifically includes the following contents:

step 210: if the forwarding object of the downlink service flow message is determined to be a service grid forwarding plane arranged in the DPU, the downlink service flow message is sent to a hardware protocol stack in the DPU, so that the hardware protocol stack selects one of all threads in the service grid forwarding plane as a current target thread, and forwards the downlink service flow message to the target thread in the service grid forwarding plane based on a downlink TCP connection path constructed between the service grid forwarding plane in advance.

In step 210, a specific selection manner of selecting one of the threads in the service grid forwarding plane as a current target thread may perform load balancing in a manner of polling or the like.

In order to further improve the integrity and reliability of the service traffic forwarding process based on the DPU, in the service traffic forwarding method based on the DPU provided in the embodiment of the present application, referring to fig. 4, step 200 in the service traffic forwarding method based on the DPU further specifically includes the following contents:

step 300: receiving an uplink service flow message sent by the hardware protocol stack, wherein the uplink service flow message is obtained by encapsulating service flow data received in advance based on an uplink TCP connection path by the hardware protocol stack; the target thread receives the downlink service flow message and determines the destination endpoint information of the host side based on preset configuration rule data, and then establishes the uplink TCP connection path with the hardware protocol stack, and then the target thread sends the service flow data corresponding to the downlink service flow message and the destination endpoint information after processing the uplink TCP connection path to the hardware protocol stack.

Step 400: and sending the uplink service flow message to a destination endpoint of the host side based on the destination endpoint information so that a destination container group corresponding to the destination endpoint information acquires the uplink service flow message.

In order to further improve the interaction reliability between the hardware protocol stack and the service grid forwarding plane, in the service traffic forwarding method based on the DPU provided in the embodiment of the present application, the sending TCP connection path and the sending TCP connection path are pre-configured by sending a TCP protocol packet of the hardware protocol stack to the service grid forwarding plane according to a connection request signal sent by the service grid forwarding plane when or after the hardware protocol stack receives the connection request signal.

In order to further improve efficiency and reliability of determining destination endpoint information of the host side by the service grid forwarding plane, in the service traffic forwarding method based on the DPU provided in the embodiment of the present application, the configuration rule data is sent to the service grid forwarding plane by the service grid management plane of the host side in advance and stored in the service grid forwarding plane.

In one or more embodiments of the present application, the configuration rule data may include initial configuration rule data and may also include configuration rule change data.

In order to further describe the service traffic forwarding process based on the DPU as a whole, in an example of the present application, a service traffic forwarding process based on the DPU with complete interaction between multiple ends is further provided, which specifically includes the following contents:

step 1: the service grid forwarding plane receives and stores configuration rule data sent by the service grid management plane of the host side.

Step 2: the service grid forwarding surface transmits a connection request signal for transmitting a service flow message to the hardware protocol stack.

Step 3: and the hardware protocol stack sends a TCP protocol message of the hardware protocol stack to the service grid forwarding plane according to the connection request signal when or after receiving the connection request signal for issuing the service flow message sent by the service grid forwarding plane so as to construct a corresponding issuing TCP connection path.

Step 4: service mesh forwarding sends a forwarding table to the network forwarder.

Step 5: the network forwarder receives and stores the forwarding table.

Step 6: the network repeater receives the issued service flow message sent by the host side based on the interface through network card.

Step 7: and the network forwarder searches a corresponding forwarding object from a local forwarding table according to the attribute information corresponding to the issued service flow message, wherein the forwarding object comprises the service grid forwarding plane, and if the forwarding object of the issued service flow message is determined to be the service grid forwarding plane arranged in the DPU, the issued service flow message is sent to a hardware protocol stack in the DPU.

Step 8: and selecting one of the threads in the service grid forwarding plane as a current target thread by the hardware protocol stack according to the attribute information corresponding to the issued service flow message, and forwarding the issued service flow message to the target thread in the service grid forwarding plane based on an issued TCP connection path constructed between the hardware protocol stack and the service grid forwarding plane in advance.

Step 9: the service grid forwarding plane receives the service flow message forwarded by the hardware protocol stack based on a local target thread.

Step 10: the service grid forwarding plane determines destination endpoint information of the host side based on preset configuration rule data, and sends a connection request signal for sending a service flow message to a hardware protocol stack.

Step 11: and the hardware protocol stack sends a TCP protocol message of the hardware protocol stack to the service grid forwarding plane according to the connection request signal when or after receiving the connection request signal for the uplink service flow message sent by the service grid forwarding plane so as to construct a corresponding uplink TCP connection path.

Step 12: and the target thread of the service grid forwarding plane sends the service flow data corresponding to the issued service flow message and the destination endpoint information which are processed by the target thread to the hardware protocol stack based on the uploading TCP connection path.

Step 13: and the hardware protocol stack receives the service flow data sent by the target thread in the service grid forwarding plane, encapsulates the service flow data to obtain a corresponding uplink service flow message, and then sends the uplink service flow message to a target endpoint of the host side based on the target endpoint information so as to enable a target container group corresponding to the target endpoint information to acquire the uplink service flow message.

From the software aspect, the present application further provides a service traffic forwarding device based on a DPU, which is used for executing all or part of the service traffic forwarding method based on a DPU, referring to fig. 5, where the service traffic forwarding device 1 based on a DPU specifically includes the following contents:

a message receiving module 11, configured to receive, in a DPU, a downlink service flow message sent from outside the DPU;

and the message forwarding module 12 is configured to send the downlink service flow message to a hardware protocol stack in the DPU if it is determined that the forwarding object of the downlink service flow message is a service grid forwarding plane set in the DPU, so that the hardware protocol stack forwards the downlink service flow message to the service grid forwarding plane based on a downlink TCP connection path previously constructed between the hardware protocol stack and the service grid forwarding plane.

The embodiment of the service traffic forwarding device based on the DPU provided in the present application may be specifically used to execute the processing flow of the embodiment of the service traffic forwarding method based on the DPU in the foregoing embodiment, and the functions thereof are not described herein in detail, and reference may be made to the detailed description of the embodiment of the service traffic forwarding method based on the DPU.

The part of the service flow forwarding device based on the DPU for forwarding the service flow based on the DPU can be completed in the client device. Specifically, the selection may be made according to the processing capability of the client device, and restrictions of the use scenario of the user. The present application is not limited in this regard. If all operations are performed in the client device, the client device may further include a processor for specific processing of the DPU-based traffic forwarding.

The client device may have a communication module (i.e. a communication unit) and may be connected to a remote server in a communication manner, so as to implement data transmission with the server. The server may include a server on the side of the task scheduling center, and in other implementations may include a server of an intermediate platform, such as a server of a third party server platform having a communication link with the task scheduling center server. The server may include a single computer device, a server cluster formed by a plurality of servers, or a server structure of a distributed device.

As can be seen from the foregoing description, the service traffic forwarding device based on the DPU provided in the embodiments of the present application does not occupy the host side resources, so that the host side resources can be used for service deployment, a large amount of host side resources can be saved, the forwarding efficiency of service grid service traffic can be effectively improved, that is, the delay of the forwarding process can be greatly reduced, so that the service grid can support a service scenario sensitive to time such as financial telecommunication, and the reliability and effectiveness of the service traffic forwarding process can be ensured.

The embodiment of the application further provides an electronic device, which may include a processor, a memory, a receiver, and a transmitter, where the processor is configured to perform the service traffic forwarding method based on the DPU mentioned in the foregoing embodiment, and the processor and the memory may be connected by a bus or other manners, for example, by using a bus connection. The receiver may be connected to the processor, memory, by wire or wirelessly.

The processor may be a central processing unit (Central Processing Unit, CPU). The processor may also be any other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof.

The memory, as a non-transitory computer readable storage medium, may be used to store a non-transitory software program, a non-transitory computer executable program, and a module, such as program instructions/modules corresponding to the DPU-based traffic forwarding method in the embodiments of the present application. The processor executes various functional applications and data processing of the processor by running non-transitory software programs, instructions and modules stored in the memory, that is, implements the DPU-based traffic forwarding method in the above-described method embodiments.

The memory may include a memory program area and a memory data area, wherein the memory program area may store an operating system, at least one application program required for a function; the storage data area may store data created by the processor, etc. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory may optionally include memory located remotely from the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The one or more modules are stored in the memory that, when executed by the processor, perform the DPU-based traffic forwarding method of an embodiment.

In some embodiments of the present application, the user equipment may include a processor, a memory, and a transceiver unit, where the transceiver unit may include a receiver and a transmitter, and the processor, the memory, the receiver, and the transmitter may be connected by a bus system, the memory storing computer instructions, and the processor executing the computer instructions stored in the memory to control the transceiver unit to transmit and receive signals.

As an implementation manner, the functions of the receiver and the transmitter in the present application may be considered to be implemented by a transceiver circuit or a dedicated chip for transceiver, and the processor may be considered to be implemented by a dedicated processing chip, a processing circuit or a general-purpose chip.

As another implementation manner, a manner of using a general-purpose computer may be considered to implement the server provided in the embodiments of the present application. I.e. program code for implementing the functions of the processor, the receiver and the transmitter are stored in the memory, and the general purpose processor implements the functions of the processor, the receiver and the transmitter by executing the code in the memory.

The embodiments of the present application also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the aforementioned DPU-based traffic forwarding method. The computer readable storage medium may be a tangible storage medium such as Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disk, a removable memory disk, a CD-ROM, or any other form of storage medium known in the art.

Based on the foregoing embodiment of the service traffic forwarding method based on the DPU and/or the service traffic forwarding device based on the DPU, the present application further provides an embodiment of a service traffic forwarding system based on the DPU, referring to fig. 6, where the service traffic forwarding system based on the DPU specifically includes the following contents:

a network repeater 1, a hardware protocol stack 2 and a service grid forwarding plane 3, which are respectively arranged in the DPU and are sequentially connected in a communication mode, wherein the service grid forwarding plane 3 is arranged in a system on a chip (SOC) of the DPU;

the network repeater 1 is configured to perform the service traffic forwarding method based on the DPU in the foregoing embodiment;

the hardware protocol stack 2 and the service grid forwarding plane 3 are in communication connection based on a preset PCIE;

The service grid forwarding plane 3 is in communication connection with the service grid management plane 4 of the host side based on a preset management port 5, and the network forwarder 1 is in communication connection with each container group Pod in the container cluster 6 of the host side based on a preset interface through network card 7.

In order to further improve the application reliability and effectiveness of the hardware protocol stack, in the embodiment of the DPU-based traffic forwarding system provided in the present application, referring to fig. 7, the hardware protocol stack 2 specifically includes the following contents:

the TCP connection establishment module 21 is configured to send a TCP protocol packet of itself to the service grid forwarding plane according to the connection request signal when or after receiving the connection request signal sent by the service grid forwarding plane, so as to construct a corresponding outgoing TCP connection path or outgoing TCP connection path;

the forwarding module 22 is configured to select one of the threads in the service grid forwarding plane as a current target thread according to attribute information corresponding to the forwarding service traffic message, and forward the forwarding service traffic message to the target thread in the service grid forwarding plane based on a forwarding TCP connection path constructed in advance with the service grid forwarding plane;

The forwarding module 23 is configured to receive, based on a forwarding TCP connection path previously constructed between the forwarding plane of the service mesh, service traffic data sent by the target thread in the forwarding plane of the service mesh, encapsulate the service traffic data to obtain a corresponding forwarding service traffic packet, and send the forwarding service traffic packet to the destination endpoint on the host side so that a destination container group corresponding to the destination endpoint obtains the forwarding service traffic packet.

In order to further improve the application reliability and effectiveness of the service grid forwarding plane, in the embodiment of the DPU-based service traffic forwarding system provided in the present application, referring to fig. 8, the service grid forwarding plane 3 specifically includes the following contents:

a configuration receiving module 31, configured to receive and store configuration rule data sent by a service grid management plane of a host side;

the TCP connection request module 32 is configured to send a connection request signal to the hardware protocol stack, so that the hardware protocol stack sends out a TCP protocol packet of itself according to the connection request signal to construct a corresponding downlink TCP connection path or uplink TCP connection path with the hardware protocol stack;

A message receiving module 33, configured to receive, based on a local target thread, the service flow sending message forwarded by the hardware protocol stack;

and the message uploading module 34 is configured to determine a destination endpoint on the host side based on preset configuration rule data, and after an uploading TCP connection path is established between the TCP connection request module and the hardware protocol stack, the destination thread sends the service flow data corresponding to the downlink service flow message processed by the destination thread based on the uploading TCP connection path to the hardware protocol stack.

In order to further explain the service flow forwarding scheme based on the DPU designed by the application, the application also provides a specific application example of the service flow forwarding method based on the DPU, which is realized by applying the service flow forwarding system based on the DPU, specifically, a hardware protocol stack unloading scheme is used for a service grid forwarding plane applied to the DPU field, and a four-layer TCP hardware unloading scheme is provided for a low-delay high-speed forwarding scene. The TCP protocol stack and four-layer forwarding are handled on the DPU using hardware, supporting a centralized service grid on the Soc.

Specifically, the service grid data plane in the application example is changed from the side car mode into the centralized type, and meanwhile, the centralized service grid forwarding plane is unloaded onto the DPU, firstly, a plurality of forwarding planes are changed into 1 and unloaded onto the DPU, host side resources are not occupied, the host side resources can be used for service deployment, and a large amount of host side resources are saved.

More importantly, after the service grid forwarding plane is unloaded to the DPU, the flow of the host side micro-service is directly sent to the DPU side for processing after the flow is subjected to interface direct network card technology; the external flow can be directly processed and forwarded on the DPU without uploading the processing of the host side, one hop to the host side is saved, and the network card directly processes the message to greatly reduce the network delay;

the application example can realize a hardware protocol stack, and replaces a software protocol stack to support protocol stack processing of a service grid forwarding plane, so that the hardware advantage is exerted, and the time delay can be reduced by more than half. The service grid originally supports the Internet service with large bandwidth but without time delay, and the scheme greatly reduces the time delay, so that the service grid can support service scenes sensitive to the time delay, such as financial telecom.

Referring to fig. 9, the communication procedure between the dpu side service grid forwarding plane and the host side service grid management plane is as follows:

1.1, the service grid forwarding plane is unloaded to the DPU, and can establish connection (such as gRPC connection) with the service grid management plane in the host side kubenetes cluster through a DPU card management port, and can receive configuration and configuration change information of the service grid management plane;

1.2 service grid forwarding plane uses hardware protocol stack processing protocol instead of software protocol stack. The communication problem between the forwarding plane software and the hardware is mainly required to be solved, and the communication can be completed by adopting a PCIE register writing mode.

1.3 the service grid management plane will send the traffic rule to be sent to the service grid forwarding plane to the forwarding table in the network forwarder for the subsequent traffic forwarding.

Wherein, vf-1, vf-2, vfr-1 and vfr-1 are all different interfaces and pass through the network cards. The clients pod, podA and podB all represent the container group. Kubernetes denotes a container cluster, eth0 denotes a network interface, fd1 and fd2 denote different TCP connections, NIC denotes a network card, "0.0.0.0:15001" denotes an address of a hardware protocol stack, "10.56.22.3" denotes a port of a container group client pod, "10.56.22.5" denotes a port of a container group podA, "10.56.22.6" denotes a port of a container group podB, and "server 10.86.5.100:10001" denotes an address of a downstream client.

Referring to fig. 10, the service traffic forwarding process based on the DPU, which is implemented by using the service traffic forwarding system based on the DPU, is as follows:

2.1, the flow in the pod of the host side is directly connected with a network card (such as an SR-IOV technology or a PF pass through) through an interface, and is forwarded to a corresponding module through a hardware network forwarding table by a hardware network forwarding module, and the message is forwarded to a hardware protocol stack for processing when the Envoy rule table is hit.

2.2 hardware protocol stack, according to the five-tuple of flow, selecting a certain thread (such as thread 2) of service grid forwarding plane, and establishing TCP connection (such as fd 1) between downstream client and service grid thread 2.

2.3 the service grid forwarding plane thread 2 selects an upstream destination port (endpoint) according to the configuration issued by the service grid management plane, and establishes a second TCP connection (such as fd 2) with the upstream through the hardware protocol stack according to the rule, and directly uploads the second TCP connection to the upstream destination port (endpoint) of the host side through the interface.

In summary, according to the service flow forwarding method based on the DPU implemented by the service flow forwarding system based on the DPU provided by the application example of the present application, interaction is performed between the Envoy side and the hardware protocol stack through a special channel (such as a shared memory). Envoy sends signal message to hardware protocol stack to replace Socket interface call; the hardware protocol stack sends a TCP protocol message to the opposite terminal according to the signal, and establishes TCP connection; the connection is established into a function, the Envoy sends the flow to the hardware protocol stack, and the TCP/IP header is encapsulated by the hardware protocol stack. The creation and maintenance of TCP connections is accomplished by a hardware protocol stack. The network repeater device is provided with a forwarding table, and the Envoy can send the traffic needing to be sent to the Envoy into the network forwarding table. When the traffic arrives at the network forwarder, the forwarding table is queried, the rules hit are uploaded to the hardware protocol stack, and the hardware protocol stack is uploaded to the socside Envoy through a special channel. From a latency perspective, the hardware protocol stack latency can be reduced to almost half that of the software protocol stack. Specifically, compared with the prior art, the service flow forwarding method based on the DPU, which is realized by the service flow forwarding system based on the DPU and provided by the application example of the application, has the following improvements:

1) The service grid forwarding plane is unloaded onto the DPU, and can be connected with the service grid management plane through a PCIE management port of the DPU card to acquire configuration and change.

2) The service grid forwarding plane uses a hardware protocol stack to replace a software protocol stack to process protocol layer messages and protocols.

3) The host side service directly passes the message to the DPU board card through the interface direct technology and does not issue through the host side protocol stack.

4) The DPU side hardware module (network forwarding module and hardware protocol stack module) can load balance the upstream protocol message according to a certain rule (such as using five-tuple), so that the upstream establishes TCP connection with a thread of the service grid forwarding plane.

5) A thread of the service grid forwarding plane can establish a TCP connection with a downstream through a hardware module (a network forwarding module and a hardware protocol stack module) at the DPU side.

Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or a combination of both. The particular implementation is hardware or software dependent on the specific application of the solution and the design constraints. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the present application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave.

It should be clear that the present application is not limited to the particular arrangements and processes described above and illustrated in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps, after appreciating the spirit of the present application.

The features illustrated and/or described in the context of one embodiment may be used in combination with, or instead of, the features of other embodiments in the same way or in a similar way in one or more other embodiments

The foregoing description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and variations may be made to the embodiment of the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A service traffic forwarding method based on a DPU, comprising:

2. The DPU-based traffic forwarding method of claim 1, further comprising, prior to said sending the downstream traffic packet to a hardware protocol stack in the DPU:

3. The DPU-based traffic forwarding method of claim 2, wherein if it is determined that the forwarding object of the outgoing traffic packet is a service grid forwarding plane disposed in the DPU, sending the outgoing traffic packet to a hardware protocol stack in the DPU, so that the hardware protocol stack forwards the outgoing traffic packet to the service grid forwarding plane based on an outgoing TCP connection path previously constructed between the hardware protocol stack and the service grid forwarding plane, including:

4. The DPU-based traffic forwarding method of claim 3, further comprising:

Receiving an uplink service flow message sent by the hardware protocol stack, wherein the uplink service flow message is obtained by encapsulating service flow data received in advance based on an uplink TCP connection path by the hardware protocol stack; the uplink TCP connection path is established between the target thread and the hardware protocol stack after receiving the downlink service flow message and determining target endpoint information of the host side based on preset configuration rule data, and then the target thread sends the service flow data corresponding to the downlink service flow message and the target endpoint information after processing the target thread to the hardware protocol stack based on the uplink TCP connection path;

5. The DPU-based traffic forwarding method of claim 4, wherein the downstream TCP connection path and the upstream TCP connection path are pre-constructed by the hardware protocol stack sending its own TCP protocol packet to the service mesh forwarding plane according to a connection request signal sent by the service mesh forwarding plane when or after the hardware protocol stack receives the connection request signal.

6. The DPU-based traffic forwarding method of claim 4, wherein the configuration rule data is sent to the service mesh forwarding plane by a service mesh management plane on a host side in advance and stored in the service mesh forwarding plane.

7. A DPU-based traffic forwarding device, comprising:

8. A DPU-based traffic forwarding system, comprising: the system comprises a network repeater, a hardware protocol stack and a service grid forwarding plane, wherein the network repeater, the hardware protocol stack and the service grid forwarding plane are respectively arranged in a DPU and are sequentially in communication connection;

the network repeater is configured to perform the DPU-based traffic forwarding method of any one of claims 1 to 6;

9. The DPU-based traffic forwarding system of claim 8, wherein the hardware protocol stack comprises:

10. The DPU-based traffic forwarding system of claim 8, wherein the service grid forwarding plane comprises: