WO2019134403A1 - Procédé et appareil pour envoyer un paquet de données, et support de stockage lisible par ordinateur - Google Patents

Procédé et appareil pour envoyer un paquet de données, et support de stockage lisible par ordinateur Download PDF

Info

Publication number
WO2019134403A1
WO2019134403A1 PCT/CN2018/107481 CN2018107481W WO2019134403A1 WO 2019134403 A1 WO2019134403 A1 WO 2019134403A1 CN 2018107481 W CN2018107481 W CN 2018107481W WO 2019134403 A1 WO2019134403 A1 WO 2019134403A1
Authority
WO
WIPO (PCT)
Prior art keywords
data packet
data
virtual address
transport layer
buffer
Prior art date
Application number
PCT/CN2018/107481
Other languages
English (en)
Chinese (zh)
Inventor
向光友
王英男
罗先强
张明谦
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2019134403A1 publication Critical patent/WO2019134403A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/10—Address translation
    • G06F12/109—Address translation for multiple virtual address spaces, e.g. segmentation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/90—Buffering arrangements
    • H04L49/901—Buffering arrangements using storage descriptor, e.g. read or write pointers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/90—Buffering arrangements
    • H04L49/9063—Intermediate storage in different physical parts of a node or terminal

Definitions

  • the present application relates to the field of computer storage and, more particularly, to a method, apparatus, and computer readable storage medium for transmitting data packets.
  • the communication device needs to continuously encapsulate the data packet.
  • the data packet needs to be copied multiple times during the encapsulation process.
  • a network storage device needs to send a data packet on the storage side through at least a buffer storage area to a storage protocol layer buffer, a storage protocol layer buffer to a transmission control protocol/ Multiple copies of data between buffers of the transmission control protocol/internet protocol (TCP/IP) protocol stack.
  • TCP/IP transmission control protocol/internet protocol
  • CPU central processing unit
  • the present application provides a method, an apparatus, and a computer readable storage medium for transmitting a data packet, which can reduce the occupation of CPU resources by a data packet transmission process.
  • the first aspect provides a method for sending a data packet, including: acquiring a virtual address of a first user data to be sent in a user memory; and writing a transport layer header of the first data packet in a buffer of the transport layer,
  • the first data packet includes the first user data; the virtual address of the first data packet is converted into a physical address, where the virtual address of the first data packet includes a virtual address of the first user data in a user memory, And a virtual address of the transport layer header of the first data packet in the buffer; and the first data packet is sent by the network card according to the converted physical address.
  • the virtual address of the first data packet includes a virtual address in two different storage areas of the memory: a virtual address in the user memory and a virtual address in a buffer of the transport layer, that is, the first data packet It is built and organized across memory areas, so there is no need to copy user data, which can reduce the CPU resource consumption during the packet sending process.
  • the user memory is part of a pre-allocated shared memory
  • the buffer of the transport layer is part of the pre-allocated shared memory
  • the virtual address of the first data packet is converted into a physical address.
  • the method includes: converting the virtual address of the first data packet into a physical address according to a pre-established correspondence between the virtual address and the physical address of the shared memory.
  • Both the user memory and the transport layer buffers use pre-allocated shared memory, that is, the first user data and the virtual address of the transport layer header of the first packet are both in shared memory.
  • the address translation is performed, the virtual address of the first data packet can be converted into a physical address according to the correspondence between the virtual address and the physical address of the shared memory that is established in advance.
  • the solution is capable of operating access between multiple processes and enables fast address translation.
  • the method further includes: acquiring second user data to be sent; writing a transport layer header of the second data packet in a buffer of the transport layer, the second data packet including the second User data; converting the virtual address of the second data packet into a physical address, wherein the virtual address of the second data packet includes a virtual address of the second user data in the buffer, and transmission of the second data packet The virtual address of the layer header in the buffer; according to the converted physical address, the second data packet is sent by the network card.
  • the transport layer header of the second data packet and the virtual address of the user data are virtual addresses in the transport layer buffer, and the user data indicating the second data packet is copied from the upper layer of the transport layer to the buffer of the transport layer. Area, the encapsulation and organization of the second data packet is performed in the buffer of the transport layer.
  • the buffer of the transport layer stores the first structure corresponding to the first data packet, the second structure body and the third structure body, and the first structure body corresponding to the first data packet a length of the first data packet, and a pointer of the first structure corresponding to the next data packet of the first data packet in the buffer of the transport layer, the second structure corresponding to the first data packet And a virtual address for recording the first data packet, where the third structure corresponding to the first data packet is used to record a physical address of the first data packet.
  • the first structure is responsible for organizing memory in the transport layer
  • the second structure is responsible for managing the virtual address of the data packet in the buffer
  • the third structure is for interacting with the network card, and is responsible for managing the physical address of the data packet in the buffer.
  • the method is performed by a storage server, the first user data being data requested by a client of the storage server.
  • the protocol used by the transport layer is Transmission Control Protocol/Internet Protocol TCP/IP or User Datagram Protocol UDP.
  • an apparatus for transmitting a data packet comprising means for performing the method of transmitting a data packet in any of the possible implementations of the first aspect or the first aspect.
  • a third aspect provides an apparatus for transmitting a data packet, the apparatus comprising: a memory for storing a program; a processor for executing a program, the processor is configured to perform the first aspect as described above, and when the program is executed Corresponding functions in the various implementations of the first aspect.
  • a storage system comprising: a memory for storing a program; and a controller, such as the device for transmitting a data packet in the second aspect, for executing a program in the memory.
  • a computer readable storage medium for storing computer software instructions comprising a program designed to perform the method of the first aspect described above is provided.
  • a computer program product comprising: computer program code, when the computer program code is run on a computer, causing the computer to perform any of the first aspect or the first aspect described above The method in the way.
  • FIG. 1 is a schematic diagram of a networking of a NAS storage server according to an embodiment of the present application.
  • FIG. 2 is a schematic diagram of a conventional method of transmitting a data packet.
  • FIG. 3 is a schematic flowchart of a method for sending a data packet according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a shared memory management manner according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a method for transmitting a data packet according to an embodiment of the present application.
  • FIG. 6 is a flow chart of sending a copy-free copy of the TCP/IP protocol stack in the embodiment of the present application.
  • FIG. 7 is a schematic diagram of a memory management manner according to an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a specific implementation manner of a data structure in an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a memory organization form of a single message in a copy-free scenario according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a manner of organizing a TCP/IP protocol stack buffer according to an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a memory organization form of a single packet in a copy scenario according to an embodiment of the present application.
  • FIG. 12 is a schematic diagram of an implementation of a copy-free interface according to an embodiment of the present application.
  • FIG. 13 is a schematic diagram of a sending process of a message according to an embodiment of the present application.
  • FIG. 14 is a schematic flowchart of a method for transmitting a data packet according to an embodiment of the present application.
  • FIG. 15 is a schematic block diagram of an apparatus for transmitting a data packet according to an embodiment of the present application.
  • the method for transmitting a data packet provided by the embodiment of the present invention can be applied to any computer device that needs to communicate by using a data packet.
  • the method for sending a data packet provided by the embodiment of the present invention can be applied to a storage server in a network, such as a NAS-based storage server.
  • the NAS scenario will be described as an example in conjunction with FIG. 1 .
  • FIG. 1 is a schematic diagram of an implementation of a networking of a NAS storage server according to an embodiment of the present invention.
  • NAS refers to a file-level computer data store connected to a computer network that provides data access to different types of clients over a TCP/IP network.
  • NAS usually occupies its own node on a local area network (LAN), allowing users to access data on the network without the intervention of the application server.
  • LAN local area network
  • the NAS centrally manages and processes all the data on the network, offloading the load from the application or enterprise server, effectively reducing the total cost of ownership.
  • the storage server often needs to receive the input/output (IO) request sent by the client and send the corresponding data packet to the client.
  • the storage server needs to encapsulate the data packet through a storage protocol layer, a transport layer, etc., and finally the network card uses direct memory access (DMA) according to the physical address of the data packet.
  • DMA direct memory access
  • the following describes the manner of transmitting a data packet in detail with reference to FIG. 2, taking a storage server as an example.
  • the memory in FIG. 2 is mainly used to store programs and various data, and the memory may be a hard disk, a magnetic disk, or the like.
  • the CPU can read the data from the memory and cache the data into the cache component (ie, the cache component).
  • the Cache component is a part of the memory in the user's memory and can be used to cache user data.
  • the user memory may further include a storage protocol layer buffer, and the storage protocol layer buffer is used to buffer data transferred from the cache buffer layer.
  • the transport layer can implement end-to-end transmission.
  • the protocol used by the transport layer can be TCP/IP or user datagram protocol (UDP).
  • the transport layer buffer is a part of the memory in the memory space. When there is data to be cached, the CPU requests a portion of the memory from the memory controller, and the portion of the memory can be used to cache the data transferred from the upper layer.
  • the storage protocol can handle standard storage protocol specifications and provide storage service access capabilities for different storage protocol clients.
  • the storage protocol varies according to the protocol type in the storage server.
  • the storage protocol may be, for example, a NAS protocol, a network file system (NFS) protocol, a server message block (SMB) protocol, or an Internet small computer system.
  • NFS network file system
  • SMB server message block
  • the embodiment of the present invention does not specifically limit this.
  • the embodiment of the present invention is not limited to the specific embodiment of the present invention.
  • the network card can receive the physical address of the data packet delivered by the upper layer, and send the data packet to the network in a DMA manner according to the physical address.
  • the storage server After receiving the read IO request sent by the client, the storage server sends the data packet to the client roughly through the following processes:
  • the storage protocol layer copies the data to the transport layer buffer after the data is encapsulated.
  • the transport layer After the transport layer encapsulates the data, the data is placed on the network card, and the network card is sent to the client in DMA mode.
  • the traditional way of sending data packets requires multiple copies of the data, such as a cache buffer to the storage protocol layer buffer, and a copy of the storage protocol layer buffer to the transport layer buffer. This way of sending data packets requires a lot of CPU resources and affects the performance of the CPU.
  • the foregoing describes the process of transmitting a data packet by taking the NAS scenario as an example.
  • the embodiment of the present invention is not limited thereto.
  • the embodiments of the present invention can also be applied to other scenarios, such as a common communication scenario for transmitting data packets.
  • the storage protocol layer may not be set, and the application layer directly sends the data packet to the transport layer.
  • the embodiment of the invention provides a method for sending a data packet, which can send the data packet in a copy-free manner and reduce the resource consumption of the CPU.
  • the execution body of the method for transmitting a data packet in the embodiment of the present invention may be a computer device, such as a CPU in a computer device, or a functional module in the computer device that can call a program and execute the program.
  • transport layer as the execution subject in the embodiment of the present invention means that the CPU in the computer calls the program code of the transport layer protocol to perform the corresponding function.
  • application layer as the execution subject mentioned in the embodiment of the present invention means that the CPU in the computer calls the application layer program code to execute the corresponding function.
  • transport layer and the application layer as execution subjects.
  • FIG. 3 is a schematic flowchart of a method for sending a data packet according to an embodiment of the present invention.
  • the method of FIG. 3 may include steps 310-340, which are described in detail below with respect to steps 310-340, respectively.
  • step 310 a virtual address of the first user data to be transmitted in the user memory is obtained.
  • the first user data may be data requested by the second user equipment to the first user equipment, where the first user data is located in a user memory of the first user equipment.
  • the manner of obtaining the virtual address of the first user data is not specifically limited in the embodiment of the present invention.
  • the virtual address of the first user data may be sent to the transport layer by the storage protocol layer, or the virtual address of the first user data may be sent by the application layer to the transport layer. That is to say, the virtual address of the first user data in the user memory may refer to the virtual address of the first user data in the application layer, or may also refer to the virtual address of the first user data in the storage protocol layer.
  • the protocol used by the transport layer can be TCP/IP or UDP. This embodiment of the present invention does not specifically limit this.
  • step 320 the transport layer header of the first data packet is written to a buffer of the transport layer, the first data packet including the first user data.
  • the first data packet is a data packet for transmitting the first user data.
  • the transport layer header is encapsulated for the first user data to form a first data packet.
  • the first part of the transport layer stores the active end, the port number of the destination end, the source IP address, and the destination IP address, and is used to find the application process of the sender and the receiver.
  • step 330 the virtual address of the first data packet is converted into a physical address, wherein the virtual address of the first data packet includes a virtual address of the first user data in the user memory, and the transport layer header of the first data packet is The virtual address in the buffer.
  • step 340 the first data packet is sent by the network card according to the converted physical address.
  • the transport layer sends the converted physical address to the network card, and the network card sends the first data packet to the network in a DMA manner according to the converted physical address.
  • the virtual address of the first data packet includes a virtual address in two different storage areas of the memory: a virtual address in the user memory and a virtual address in a buffer of the transport layer, that is, the first Packets are built and organized across memory regions, eliminating the need to copy user data, thereby reducing CPU resource usage during packet delivery.
  • the embodiment of the present invention does not specifically limit the virtual address of the first user data and the mapping manner of the virtual address to the physical address.
  • the shared memory controller will be specifically described below with reference to FIG.
  • Both the user memory and the buffer of the transport layer of the embodiment of the present invention may use pre-allocated shared memory.
  • the shared memory controller manages shared memory to ensure that memory is operationally accessible between components within the system.
  • the cache component can apply for shared memory through the shared memory controller
  • the TCP/IP protocol stack can implement virtual address to physical address conversion through the shared memory controller.
  • the physical address converted by the shared memory controller can be directly DMA processed by the network card without copying.
  • the operating system manages the system physical memory.
  • the shared memory controller allocates a portion of the memory from the physical memory managed by the operating system as shared memory.
  • the shared memory is managed in a secondary manner according to the layout of the storage service, and basic memory management functions such as virtual address mapping, memory pool management, and cache management are implemented and provided to the business side.
  • the memory can be released back to the operating system through the shared memory controller.
  • the mapping mode provided by the shared memory can quickly realize the mapping from the virtual address to the physical address, so that the network card can directly access the shared memory data based on the converted physical address for DMA without copying.
  • Step 310 includes obtaining a virtual address of the first user data to be sent in the shared memory.
  • the manner of obtaining the virtual address of the first user data in the shared memory is not specifically limited in the embodiment of the present invention.
  • the transport layer may receive a virtual address of the first user data transferred from the storage protocol layer in the shared memory.
  • the transport layer may receive a virtual address of the first user data passed from the application layer in the shared memory.
  • Step 320 includes writing a transport layer header of the first data packet in a buffer of the transport layer, the first data packet including first user data, wherein the buffer of the transport layer is located in the shared memory.
  • Step 330 includes converting the virtual address of the first data packet into a physical address according to a pre-established correspondence between the virtual address and the physical address of the shared memory.
  • the shared memory controller in the embodiment of the present invention can implement mapping between a shared memory virtual address and a physical address, and manage the address mapping relationship.
  • the transport layer sends the virtual address of the first data packet to the shared memory controller, and the shared memory controller converts the virtual address of the first data packet into a physical address according to the address mapping relationship.
  • the virtual address of the first data packet includes the first user data and a virtual address of the transport layer header of the first data packet in the shared memory.
  • Step 340 includes: transmitting a physical address of the first data packet to the network card to instruct the network card to send the first data packet.
  • the transport layer sends the converted physical address to the network card, and the network card sends the first data packet to the network in a DMA manner according to the converted physical address.
  • both the user memory and the buffer of the transport layer use pre-allocated shared memory, that is, the first user data and the virtual address of the transport layer header of the first data packet are both Located in shared memory.
  • the address translation is performed, the virtual address of the first data packet can be converted into a physical address according to the correspondence between the virtual address and the physical address of the shared memory that is established in advance.
  • the solution is capable of operating access between multiple processes and enables fast address translation.
  • the traditional way of sending data packets requires at least a copy of the data between the storage protocol layer and the transport layer.
  • the embodiment of the present invention adopts a self-developed transport layer protocol and a shared memory controller, so that the entire process in the sending direction is free of copy.
  • the embodiment of the present invention uses a private copy-free interface of the self-developed transport layer to input user data to the transport layer.
  • the transport layer organizes the memory through a memory mapping method and caches it in the transport layer buffer.
  • the transmission layer sending process directly uses the memory in the buffer for sending processing, and sends the memory address to the network card driver, and the network card driver sends the data packet corresponding to the memory to the network through the DMA technology.
  • the transport layer provides a complete transport layer protocol function, and supports a copy-free high-performance socket socket interface for the storage protocol to send data to the network in an efficient manner, reducing the CPU usage of the process of sending data packets.
  • the technical solution provided by the embodiment of the present invention can also perform copy processing on data.
  • the second user data Acquiring, in the copy scenario, the second user data to be sent; writing the transport layer header of the second data packet in a buffer of the transport layer, the second data packet includes the second user data; and the virtual address of the second data packet Converting into a physical address, wherein the virtual address of the second data packet includes a virtual address of the second user data in the buffer, and a virtual address of the transport layer header of the second data packet in the buffer; according to the converted physical address Send the second data packet through the network card.
  • the transport layer header of the second data packet and the virtual address of the user data are virtual addresses in the transport layer buffer, and the user data indicating the second data packet is copied from the upper layer of the transport layer to the buffer of the transport layer. Area, the encapsulation and organization of the second data packet is performed in the buffer of the transport layer.
  • both the user memory and the transport layer buffer where the second user data is located can be located in the shared memory.
  • the shared memory controller converts the virtual address of the second data packet into a physical address according to the virtual address of the second data packet in the shared memory.
  • the embodiment of the present invention does not specifically limit the manner in which the transport layer manages the address information of the first data packet, and the address information of the first data packet may be recorded by one or more structures.
  • the address information of the first data packet may be recorded by the first structure, the second structure, and the third structure corresponding to the first data packet.
  • the first structure corresponding to the first data packet is used for recording the length of the first data packet, and the pointer of the first structure corresponding to the next data packet of the first data packet in the buffer of the transport layer,
  • the second structure corresponding to the data packet is used to record the virtual address of the first data packet in the user memory, and the third structure corresponding to the first data packet is used to record the physical address of the first data packet.
  • the first structure is responsible for organizing memory in the transport layer
  • the second structure is responsible for managing the virtual address of the data packet in the buffer
  • the third structure is for interacting with the network card, and is responsible for managing the physical address of the data packet in the buffer.
  • the first structure may be a memory management unit
  • the second structure may be a data area descriptor
  • the third structure may be a message buffer.
  • the memory management unit may be, for example, an MBUF
  • the data area descriptor may be, for example, a DBD
  • the message buffer may be, for example, a PBUF.
  • the first structural body, the second structural body, and the third structural body are taken as an example to describe the organization of the memory.
  • the embodiment of the present invention is not limited thereto.
  • the first structure body, the second structure body, and the third structure body may be combined and combined into one or two structures for organizing memory.
  • the transport layer buffer is based on the message cache.
  • the copy/copy-free interface organizes the memory in units of messages and puts them into the send buffer.
  • the cache memory in the send buffer is directly used for sending in the normal scenario.
  • the transport layer uses pool-based memory management to divide a memory into a memory pool in advance in the shared memory, and the transport layer directly allocates memory from the memory pool when in use.
  • the memory management unit, data area descriptor, and message buffer are taken as an example for description.
  • the key data structure consists of a message buffer, a memory management unit, a data area descriptor, and a data payload.
  • FIG. 8 is a schematic diagram of a specific implementation manner of each structure provided by an embodiment of the present invention. The function of each structure is described below.
  • Message buffer Mainly responsible for interaction with the network card driver.
  • the message buffer itself can constitute a singly linked list, and a message buffer linked list represents a message.
  • the message buffer records the virtual address and physical address of the message.
  • the shared memory controller translates the virtual address of the packet into a physical address and sends it to the transport layer.
  • the transport layer records the physical address of the packet in the packet buffer.
  • the transport layer interacts with the NIC driver through the packet buffer to instruct the NIC to send the packet to the network in DMA mode according to the physical address of the packet.
  • Memory Management Unit responsible for memory organization within the TCP/IP protocol stack.
  • a memory management unit corresponds to a message.
  • the memory management unit records the total length of the message.
  • Data Area Descriptor The virtual address and length responsible for maintaining the data payload.
  • the data area descriptors can be composed of the data area descriptor chain and are generally maintained by the memory management unit.
  • the data area descriptor can be used to record the virtual address and length of the user data.
  • the transport layer After the transport layer obtains the virtual address of the user data, the virtual address of the user data is recorded in the data area descriptor.
  • FIG. 9 is a memory organization diagram of a single message in a copy-free scenario according to an embodiment of the present invention.
  • the transport layer After the transport layer receives the virtual address of the user data passed by the storage protocol, the virtual address of the user data is recorded in the data area descriptor.
  • the memory space of the transport layer header is not reserved in the memory passed by the storage protocol. Therefore, the transport layer will assign a new data area descriptor for a single message to manage the transport layer header. That is to say, after the transport layer generates the header of the transport layer of the message, the transport layer records the virtual address of the transport layer header in the newly allocated data area descriptor, and the other data area descriptors are used to record the user data delivered by the storage protocol. Virtual address and length.
  • the data area descriptor 1 is used to record the virtual address of the transport layer header, and the data load is the transport layer header generated by the transport layer.
  • the data area descriptor 2 and the data area descriptor 3 are used to record pointer information of a storage location of user data in the memory, and the pointer information points to a virtual address of the user data in the user memory.
  • the data area descriptor 1, the data area descriptor 2, and the data area descriptor 3 form a data area descriptor linked list, and are collectively managed by the memory management unit.
  • the transport layer When the transport layer has multiple packets to be sent, it can be managed by multiple memory management units.
  • Each memory management unit corresponds to one message, and multiple memory management units are linked together by a linked list.
  • the linked list may be a doubly linked list or a singly linked list.
  • FIG. 11 is a memory organization form of a single packet in a copy scenario according to an embodiment of the present invention.
  • the transport layer After receiving the user data, the transport layer buffers the user data at the location of the data payload 2 of the transport layer buffer, and the data region descriptor 2 records the virtual address of the user data.
  • a data area descriptor 1 is newly allocated to record the virtual address of the transport layer header, and the location of the data payload 1 records the header information of the transport layer.
  • a message can also be composed of a data area descriptor list and managed by a memory management unit.
  • the payload length of each packet is less than or equal to the fragment length of the socket, and the fragment length depends on the negotiated length of the socket and the maximum length of the hardware fragment.
  • the embodiment of the present invention exemplifies the data structure in the form of a memory chip, but the embodiment of the present invention is not limited thereto.
  • the memory occupied by each structure can also be a piece of discontinuous memory. As long as the virtual address and physical address of the message can be recorded, the memory can be organized.
  • the transport layer In addition to organizing memory, the transport layer also needs to have a socket interface that supports copy-free.
  • the implementation of the copy-free interface will be described in detail below with reference to FIG.
  • FIG. 12 is a schematic diagram of an implementation of a copy-free interface according to an embodiment of the present invention.
  • the input and output of the socket copy-free interface and the copy interface are consistent, and the internal implementation organizes the memory according to the above-mentioned memory organization.
  • the copy-free interface of the embodiment of the present invention may correspond to a standard write (writev) function.
  • the transport layer segments the IOV array according to the TCP slice length.
  • the IOV array is divided into two messages, and the data area descriptor 1, the data area descriptor 2, and the data area descriptor 3 form the first message, the data area descriptor 4, and the data area description.
  • Symbol 5 constitutes the second message.
  • the data area descriptor 1 and the data area descriptor 4 are used to record the virtual address information of the transport layer header of the first message and the second message, respectively.
  • the data area descriptor 1, the data area descriptor 2, and the data area descriptor 3 form a linked list, and are managed by the memory management unit 1. After the data area descriptor 4 and the data area descriptor 5 form a linked list, they are managed by the memory management unit 2.
  • the memory management unit 1 and the memory management unit 2 are chained together by a linked list to form a buffer queue of the transmission layer transmission buffer.
  • the transport layer records the packet pointer to be sent in the process of sending the packet.
  • the packet pointer points to a memory management unit in the send buffer. The specific implementation is shown in FIG.
  • FIG. 13 is a schematic diagram of a sending process of a message according to an embodiment of the present invention.
  • Step 1 The message pointer to be sent points to the first packet in the transport layer buffer.
  • Step 2 Obtain the memory management unit pointed to by the packet to be sent, and complete the packet header filling.
  • Step 3 According to the memory organization method described above, if the data is sent in a copy manner, the memory management unit converts to the message buffer; if the data is sent in a copy-free manner, the virtual address of the data packet is The converted physical address is recorded in the message buffer.
  • Step 4 The message pointer to be sent is moved to the next memory management unit.
  • Step 5 Repeat the above steps.
  • the embodiment of the present invention is described by taking a memory management unit, a message buffer, and a data area descriptor data area descriptor as an example, but the embodiment of the present invention is not limited thereto.
  • the message buffer body may be another structure that can interact with the driver
  • the memory management unit may also be a structure that other transport layers can support.
  • the storage server usually receives a read IO request sent by a different client, and then sends the requested data to the client according to the read IO request.
  • FIG. 14 is a schematic flowchart of a method of transmitting a data packet in the scenario.
  • the client sends an IO request to the storage device through the transport layer.
  • the storage protocol reads the IO request from the transport layer, and parses the IO request according to the storage protocol (NFS/SMB/ISCSI/FTP/HTTP, etc.) to request the client from the cache component. The data to get.
  • the cache component queries the cached data in the cache buffer. If the cached data does not exist, the shared memory controller is requested to allocate shared memory, the driver reads the data to the shared memory, and caches the read data; if the cached data exists, directly returns the cached data.
  • the cache component returns the shared memory address of the cached data to the storage protocol (no data copy).
  • the storage protocol completes the data encapsulation according to the protocol specification and pushes the data to the transport layer through the copy-free high-performance transmission mode provided by the transport layer.
  • the transport layer organizes the data in a copy-free manner. After the protocol logic processing and data encapsulation are completed, the virtual address of the data is converted into a physical address recognizable by the network card through the shared memory controller. Finally push the data to the network card.
  • the network card sends the data sent by the transport layer to the network through DMA.
  • the client receives the response from the storage server and completes an IO operation.
  • FIG. 15 is a schematic block diagram of an apparatus for transmitting a data packet according to an embodiment of the present invention.
  • the device 1500 can be a storage server or any other computer device that needs to communicate using a data packet.
  • the apparatus 1500 includes an acquisition unit 1510, a writing unit 1520, a conversion unit 1530, and a transmission unit 1540. It should be noted that, in the physical device, the processor implements the steps performed by each unit by calling a computer program in the memory. As an example, the processor can perform the corresponding functions by calling a program of the transport layer protocol.
  • the obtaining unit 1510 is configured to obtain a virtual address of the first user data to be sent in the user memory.
  • the first user data may be data requested by the second user equipment to the first user equipment, where the first user data is located in a user memory of the first user equipment.
  • the manner of obtaining the virtual address of the first user data is not specifically limited in the embodiment of the present invention.
  • the virtual address of the first user data can be sent to the transport layer by the storage protocol layer.
  • the protocol used by the transport layer can be TCP/IP or UDP. This embodiment of the present invention does not specifically limit this.
  • the writing unit 1520 writes the transport layer header of the first data packet in a buffer of the transport layer, and the first data packet includes the first user data.
  • the first data packet is a data packet for transmitting the first user data
  • the transport layer header is encapsulated for the first user data to form the first data packet.
  • the first part of the transport layer stores the active end, the port number of the destination end, the source IP address, and the destination IP address, and is used to find the application process of the sender and the receiver.
  • the converting unit 1530 converts the virtual address of the first data packet into a physical address, where the virtual address of the first data packet includes a virtual address of the first user data in the user memory, and the transport layer header of the first data packet is buffered The virtual address in the zone.
  • the sending unit 1540 sends the first data packet through the network card according to the converted physical address.
  • the transport layer sends the converted physical address to the network card, and the network card sends the first data packet to the network in a DMA manner according to the converted physical address.
  • the virtual address of the first data packet includes a virtual address in two different storage areas of the memory: a virtual address in the user memory and a virtual address in a buffer of the transport layer, that is, the first Packets are built and organized across memory regions, eliminating the need to copy user data, thereby reducing CPU resource usage during packet delivery.
  • the embodiment of the present invention does not specifically limit the virtual address of the first user data and the mapping manner of the virtual address to the physical address.
  • both the user memory and the transport layer buffers are located in pre-allocated shared memory.
  • the converting unit in the device in the embodiment of the present invention is specifically configured to convert the virtual address of the first data packet into a physical address according to a pre-established correspondence between a virtual address and a physical address of the shared memory.
  • Both the user memory and the transport layer buffers use pre-allocated shared memory, that is, the first user data and the virtual address of the transport layer header of the first packet are both in shared memory.
  • the address translation is performed, the virtual address of the first data packet can be converted into a physical address according to the correspondence between the virtual address and the physical address of the shared memory that is established in advance.
  • the solution is capable of operational access between multiple processes and enables fast address translation.
  • the apparatus for transmitting a data packet in the embodiment of the present invention may also perform copy processing on the data.
  • the specific functions of the obtaining unit 1510, the writing unit 1520, the converting unit 1530, and the transmitting unit 1540 in the device 1500 are as follows.
  • the obtaining unit 1510 is configured to acquire second user data to be sent.
  • the writing unit 1520 writes the transport layer header of the second data packet in a buffer of the transport layer, and the second data packet includes the second user data.
  • the converting unit 1530 converts the virtual address of the second data packet into a physical address, where the virtual address of the second data packet includes a virtual address of the second user data in the buffer, and the transport layer header of the second data packet is buffered The virtual address in the zone.
  • the sending unit 1540 sends the second data packet through the network card according to the converted physical address.
  • the transport layer header of the second data packet and the virtual address of the user data are virtual addresses in the transport layer buffer, and the user data indicating the second data packet is copied from the upper layer of the transport layer to the buffer of the transport layer. Area, the encapsulation and organization of the second data packet is performed in the buffer of the transport layer.
  • the embodiment of the present invention does not specifically limit the manner in which the transport layer module manages the address information of the first data packet, and the address information of the first data packet may be recorded by one or more structures.
  • the buffer of the transport layer stores the first structure corresponding to the first data packet, the second structure body and the third structure body, and the first structure body corresponding to the first data packet a length of the first data packet, and a pointer of the first structure corresponding to the next data packet of the first data packet in the buffer of the transport layer, the second structure corresponding to the first data packet And a virtual address for recording the first data packet, where the third structure corresponding to the first data packet is used to record a physical address of the first data packet.
  • the first structure is responsible for organizing memory in the transport layer
  • the second structure is responsible for managing the virtual address of the data packet in the buffer
  • the third structure is for interacting with the network card, and is responsible for managing the physical address of the data packet in the buffer.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product.
  • the technical solution of the present application which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including
  • the instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present application.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

La présente invention concerne un procédé et un appareil pour envoyer un paquet de données, et un support de stockage lisible par ordinateur. Le procédé consiste à : obtenir l'adresse virtuelle de premières données d'utilisateur à envoyer dans une mémoire d'utilisateur ; écrire l'en-tête de la couche de transmission d'un premier paquet de données dans la zone tampon de la couche de transmission, le premier paquet de données comprenant les premières données d'utilisateur ; convertir l'adresse virtuelle du premier paquet de données en une adresse physique, l'adresse virtuelle du premier paquet de données comprenant l'adresse virtuelle des premières données d'utilisateur dans la mémoire d'utilisateur et l'adresse virtuelle de l'en-tête de la couche de transmission du premier paquet de données dans la zone tampon ; et, en fonction de l'adresse physique obtenue après la conversion, envoyer le premier paquet de données au moyen d'une carte réseau. Dans la solution technique fournie par la présente invention, l'adresse virtuelle du premier paquet de données comprend des adresses virtuelles dans deux zones de stockage différentes de la mémoire, c'est-à-dire que le premier paquet de données est construit et organisé à travers une zone de mémoire, de telle sorte qu'il n'est pas nécessaire de copier des données d'utilisateur, et ainsi, l'occupation des ressources de CPU pendant le processus d'envoi du paquet de données peut être réduite.
PCT/CN2018/107481 2018-01-05 2018-09-26 Procédé et appareil pour envoyer un paquet de données, et support de stockage lisible par ordinateur WO2019134403A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810010977.XA CN110011933B (zh) 2018-01-05 2018-01-05 发送数据包的方法、装置及计算机可读存储介质
CN201810010977.X 2018-01-05

Publications (1)

Publication Number Publication Date
WO2019134403A1 true WO2019134403A1 (fr) 2019-07-11

Family

ID=67143588

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/107481 WO2019134403A1 (fr) 2018-01-05 2018-09-26 Procédé et appareil pour envoyer un paquet de données, et support de stockage lisible par ordinateur

Country Status (2)

Country Link
CN (1) CN110011933B (fr)
WO (1) WO2019134403A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117692416A (zh) * 2024-02-04 2024-03-12 苏州元脑智能科技有限公司 网络报文处理方法、装置、计算机设备和存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111506541B (zh) * 2020-06-30 2020-09-22 翱捷科技(上海)有限公司 一种嵌入式网络设备中加速网络数据包处理的方法及***
CN114095572B (zh) * 2020-07-30 2023-04-07 大唐移动通信设备有限公司 数据搬运方法及装置
CN111988394B (zh) * 2020-08-18 2022-11-01 北京金山云网络技术有限公司 一种虚拟化环境中优化p2p数据传输的方法、装置及设备
CN116959289B (zh) * 2023-09-21 2024-03-22 山东通维信息工程有限公司 一种基于车路协同技术的智慧停车***及方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000041358A2 (fr) * 1999-01-08 2000-07-13 Intel Corporation Systeme de commande de flux a base de credit sur architecture d'interface virtuelle pour reseaux de systemes
CN101150487A (zh) * 2007-11-15 2008-03-26 曙光信息产业(北京)有限公司 一种零拷贝网络报文发送方法
CN101304373A (zh) * 2008-06-25 2008-11-12 中兴通讯股份有限公司 一种实现局域网内高效传输大块数据的方法及***
CN106598752A (zh) * 2016-11-15 2017-04-26 北京大学深圳研究生院 远程零拷贝方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101340574B (zh) * 2008-08-04 2010-09-08 中兴通讯股份有限公司 一种实现零拷贝发送流媒体数据的方法及***
CN101819564B (zh) * 2009-02-26 2013-04-17 国际商业机器公司 协助在虚拟机之间进行通信的方法和装置
CN101841470B (zh) * 2010-03-29 2012-10-10 东南大学 一种基于Linux的底层数据包的高速捕获方法
CN102375789B (zh) * 2010-08-09 2014-05-28 中标软件有限公司 一种通用网卡非缓存的零拷贝方法及零拷贝***

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000041358A2 (fr) * 1999-01-08 2000-07-13 Intel Corporation Systeme de commande de flux a base de credit sur architecture d'interface virtuelle pour reseaux de systemes
CN101150487A (zh) * 2007-11-15 2008-03-26 曙光信息产业(北京)有限公司 一种零拷贝网络报文发送方法
CN101304373A (zh) * 2008-06-25 2008-11-12 中兴通讯股份有限公司 一种实现局域网内高效传输大块数据的方法及***
CN106598752A (zh) * 2016-11-15 2017-04-26 北京大学深圳研究生院 远程零拷贝方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117692416A (zh) * 2024-02-04 2024-03-12 苏州元脑智能科技有限公司 网络报文处理方法、装置、计算机设备和存储介质
CN117692416B (zh) * 2024-02-04 2024-05-03 苏州元脑智能科技有限公司 网络报文处理方法、装置、计算机设备和存储介质

Also Published As

Publication number Publication date
CN110011933B (zh) 2021-05-18
CN110011933A (zh) 2019-07-12

Similar Documents

Publication Publication Date Title
WO2019134403A1 (fr) Procédé et appareil pour envoyer un paquet de données, et support de stockage lisible par ordinateur
US11016911B2 (en) Non-volatile memory express over fabric messages between a host and a target using a burst mode
US8713180B2 (en) Zero-copy network and file offload for web and application servers
JP7001533B2 (ja) ＮＶＭｅ－ｏＦ ＳＳＤにおける低レイテンシ直接データアクセス方法、及びそのためのシステム
WO2022126919A1 (fr) Procédé et appareil de transmission de fichiers, dispositif informatique et support de données
US7496690B2 (en) Method, system, and program for managing memory for data transmission through a network
WO2020077680A1 (fr) Procédé de transmission de données, système et serveur mandataire
JP4532098B2 (ja) パックし圧縮したバッファを使用してクライアント−サーバ間の通信を向上させるシステムおよび方法
US20030145230A1 (en) System for exchanging data utilizing remote direct memory access
US20170235818A1 (en) Object-backed block-based distributed storage
US9154453B2 (en) Methods and systems for providing direct DMA
US7370174B2 (en) Method, system, and program for addressing pages of memory by an I/O device
US8898336B2 (en) Content conversion system and content conversion server
WO2017114091A1 (fr) Procédé d'accès à des données nas, système et dispositif pertinent
US8788726B2 (en) Data transmission system, storage medium and data transmission program
US11379405B2 (en) Internet small computer interface systems extension for remote direct memory access (RDMA) for distributed hyper-converged storage systems
KR20070042152A (ko) 통신 접속 수립 방법과 시스템, 데이터 전송 방법과시스템, 및 컴퓨터 판독 가능한 기록 매체
US20240039995A1 (en) Data access system and method, device, and network adapter
WO2023155526A1 (fr) Procédé de traitement de flux de données, nœud de commande de stockage et support de stockage lisible non volatil
US8185633B1 (en) Method and apparatus for offloading network processes in a computer storage system
WO2021063160A1 (fr) Procédé d'accès à un disque à circuits intégrés et dispositif de stockage
US20160275042A1 (en) Zero-copy data transmission system
WO2017032152A1 (fr) Procédé pour écrire des données dans un dispositif de stockage et dispositif de stockage
US9307025B1 (en) Optimized file creation in WAN-optimized storage
US10154079B2 (en) Pre-boot file transfer system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18898808

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18898808

Country of ref document: EP

Kind code of ref document: A1