WO2019129147A1 - 一种数据传输方法及第一设备 - Google Patents

一种数据传输方法及第一设备 Download PDF

Info

Publication number
WO2019129147A1
WO2019129147A1 PCT/CN2018/124384 CN2018124384W WO2019129147A1 WO 2019129147 A1 WO2019129147 A1 WO 2019129147A1 CN 2018124384 W CN2018124384 W CN 2018124384W WO 2019129147 A1 WO2019129147 A1 WO 2019129147A1
Authority
WO
WIPO (PCT)
Prior art keywords
rdma
data
message
psn
retransmission
Prior art date
Application number
PCT/CN2018/124384
Other languages
English (en)
French (fr)
Inventor
杨华
何春志
崔雷
苏德现
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201880083147.8A priority Critical patent/CN111512603B/zh
Priority to EP18897120.4A priority patent/EP3716546A4/en
Publication of WO2019129147A1 publication Critical patent/WO2019129147A1/zh
Priority to US16/909,414 priority patent/US11412078B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32—Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • H04L69/322—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69/324—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the data link layer [OSI layer 2], e.g. HDLC
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32—Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • H04L69/322—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69/329—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/10—Address translation
    • G06F12/1081—Address translation for peripheral access to main memory, e.g. direct memory access [DMA]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • G06F15/163—Interprocessor communication
    • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • G06F15/17306—Intercommunication techniques
    • G06F15/17331—Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/08—Arrangements for detecting or preventing errors in the information received by repeating transmission, e.g. Verdan system
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/1607—Details of the supervisory signal
    • H04L1/1614—Details of the supervisory signal using bitmaps
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1829—Arrangements specially adapted for the receiver end
    • H04L1/1858—Transmission or retransmission of more than one copy of acknowledgement message
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/46—Interconnection of networks
    • H04L12/4633—Interconnection of networks using encapsulation techniques, e.g. tunneling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
    • H04L47/2466—Traffic characterised by specific attributes, e.g. priority or QoS using signalling traffic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/32—Flow control; Congestion control by discarding or delaying data units, e.g. packets or frames
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2212/00—Encapsulation of packets

Definitions

  • the present application relates to the field of network communication technologies, and in particular, to a data transmission method and a first device.
  • PFC Priority-based flow control
  • the PFC flow control mechanism can effectively reduce the network packet loss rate, but also causes problems such as PFC congestion spread, PFC deadlock, PFC storm, and PFC transmission distance limitation.
  • the remote direct memory access (RDMA) protocol is a protocol that transfers data directly from one system to another through a network without the intervention of an operating system.
  • the RDMA protocol encapsulates the data to be transmitted into one or more RDMA messages, and sends one or more RDMA messages from the transmitting end to the receiving end, and the receiving end caches the received RDMA messages in the buffer area, when receiving When the terminal successfully receives all the RDMA messages of the data to be transmitted, the receiving end writes the buffer data in the buffer area to the memory of the receiving end.
  • the RDMA protocol uses the Go-Back-N retransmission mechanism to retransmit lost packets.
  • the Go-Back-N retransmission mechanism is a retransmission mechanism that causes network transmission efficiency to drop significantly even if the packet loss rate is very low. Therefore, the RDMA protocol is extremely sensitive to the packet loss rate.
  • the RDMA protocol is generated to solve the delay of data transmission between servers in network communication. Since there is almost no packet loss in data transmission between server terminals in network communication, the packet loss rate is extremely low. The use of the above RDMA protocol effectively solves the problem of transmission delay between server terminals in network communication.
  • the RDMA protocol can run on a variety of data link layer protocols, such as Ethernet. With the development of the RDMA protocol, the RDMA protocol is gradually being applied to Ethernet. In this case, the RDMA protocol is sensitive to the packet loss rate. It gradually emerges that when the RDMA protocol is applied to Ethernet for data transmission, the network equipment in the Ethernet needs to enable the PFC flow control mechanism to reduce the impact of network packet loss on the network transmission efficiency, but with the PFC flow control mechanism When it is turned on, the reliability of network transmission will also decrease.
  • the present application provides a data transmission method and a first device, which are used to reduce the impact of network packet loss on network transmission efficiency, and improve network transmission reliability.
  • the application provides a data transmission method, including:
  • the first device encapsulates the first target data in the N RDMA messages according to the remote direct memory access RDMA protocol, where the first target data is data that the first device needs to write into the second device for storage.
  • Each of the N RDMA messages carries a message sequence number PSN, and the N is a positive integer greater than or equal to 2;
  • the first device sequentially sends the N RDMA messages to the second device according to the PSN sequence of the N RDMA messages, where each of the N RDMA packets carries the first Data write address, the first data write address is to write data in each of the N RDMA messages to an address of the second device, so that the second device is from each
  • the first data write address in each RDMA message is directly obtained in the RDMA message, and the target data is written into the storage space corresponding to the first data write address.
  • the data transmission method of the present application has the following advantages:
  • each RDMA message in the technical solution carries a first data write address, and the first data write address is to write data in the RDMA message to an address in the second device
  • the second device Each RDMA message can be received, and the corresponding first data write address can be directly obtained from the message, and the data in the message is immediately written into the memory of the second device. It can be understood that even if there is an RDMA report If the file is lost, it will not affect the other successfully received RDMA messages to be written into the memory of the second device, thereby reducing the impact of network packet loss on the network transmission rate and avoiding the loss of one or more RDMA messages.
  • the device cannot determine the write address of other RDMA packets, thereby causing a large amount of data to be cached and reducing network transmission efficiency. Therefore, the data transmission method of the present application can ensure that the target data with successful packet loss can also be stored to the second. In the storage space of the device, there is a certain network transmission rate even if the target data transmission fails, thereby reducing network packet loss. The impact on the network transmission rate and the reliability of the network transmission are also improved.
  • the first device sends the second device to the second device according to the PSN sequence of the N RDMA messages.
  • the method includes:
  • the first device receives the first retransmission indication message sent by the second device, where the retransmission indication message is carried in the packet.
  • a retransmission PSN where the retransmission PSN is a PSN corresponding to a bit in the bitmap of the second device, and the PSN of each of the N RDMA messages corresponds to the PSN.
  • the retransmission indication message indicates the PSN of the RDMA packet that the second device receives the failure.
  • the receiving end may instruct the sender to retransmit only the lost data, so that the sender retransmits the lost packet. It will transmit other successfully received packets, which can save network transmission resources and improve network transmission efficiency.
  • the second RDMA packet corresponding to the N RDMA messages Any one of the Nth RDMA messages carries an RDMA extended transmission header, and the RDMA extended transmission header is used to indicate the first data write address.
  • the application provides a data transmission method, including:
  • the first device sends a data read request to the second device, the data read request is generated according to a remote direct memory access RDMA protocol, and carries a data read address and a second data write address, the data read The address is a destination address of the second device that stores the second target data, and the second data write address is reserved for the first device to store the information read from the second device. a destination address of the second target data, where the second target data is data that the first device needs to read from the second device;
  • each of the A RDMA messages includes the second data write address corresponding to the data of each of the packets;
  • the first device directly acquires the second data write address of each RDMA message from each of the R RDMA messages that are successfully received, where the B is a positive integer less than or equal to A;
  • the first device writes data of each RDMA message in the B RDMA messages into a storage space of the second data write address corresponding to each RDMA message.
  • the receiving, by the first device, the second device is sequentially sent according to a PSN sequence of A RDMA messages.
  • the method further includes:
  • the PSN of each RDMA message in the message corresponds to a bit in the bit map of the first device.
  • the second device is received by the first device After the A RDMA messages are sequentially sent according to the PSN sequence of the A RDMA messages, the method further includes:
  • the first device determines, in the bitmap of the first device, a PSN corresponding to a bit that has received a failure as a retransmission PSN;
  • the first device sends a retransmission indication message to the second device, where the retransmission indication message carries the retransmission PSN;
  • the first device receives the retransmission message sent by the second device according to the retransmission indication message, where the retransmission message is an RDMA message that fails to be received by the retransmission PSN.
  • the retransmission PSN is an ACK extended transmission report in a negative acknowledgement message of the RDMA protocol.
  • the header field indicates.
  • the data read request carries the second RDMA extended transmission packet header.
  • a field, the second RDMA extended transport header field is used to indicate the second data write address.
  • the third aspect of the present application provides a network device, where the network device has the function of implementing the behavior of the network device in the foregoing first aspect or the second aspect.
  • This function can be implemented in hardware or in hardware by executing the corresponding software.
  • the hardware or software includes one or more modules corresponding to the functions described above.
  • an embodiment of the present application provides a network device, including: a processor, a memory, a bus, and a communication interface; the memory is configured to store a computer execution instruction, and the processor is connected to the memory through the bus, when the network device In operation, the processor executes the computer-executable instructions stored by the memory to cause the network device to perform the data transfer method of any of the first or second aspects above.
  • an embodiment of the present application provides a computer readable storage medium, configured to store computer software instructions used by the network device, when executed on a computer, to enable the computer to perform the first aspect or the second The data transmission method of any of the aspects.
  • the embodiment of the present application provides a computer program product comprising instructions, which when executed on a computer, enable the computer to perform the data transmission method of any one of the above first aspect or the second aspect.
  • an embodiment of the present application provides a network interface processing circuit, including: a processing circuit and a communication interface circuit, where the communication interface circuit is configured to perform a data transceiving operation, where the processing circuit is configured to perform the first aspect or the second The data transmission method of any of the aspects.
  • the processing circuit may be an application specific integrated circuit (ASIC), or a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or a transistor. Logic devices, discrete hardware components.
  • an embodiment of the present application provides a network card, including: a network interface chip, a memory, and a host interface circuit, where the memory is used to store a computer operation instruction, and the host interface circuit is configured to connect the host and the network card, and the network interface chip A data transmission method for performing the above-described first aspect or the second aspect by calling the above computer operation instructions.
  • the above memory may be a cache.
  • the embodiment of the present application provides a network device, including: a network card, a host, and a memory, where the network card may be the network card described in the foregoing eighth aspect, where the memory is used to store computer operation instructions, and the network card is used to receive or The data is transmitted by the host for invoking the above-described computer operation instructions to perform the data transmission method of any one of the above first aspect or the second aspect.
  • the network device may specifically be a server.
  • FIG. 1 is a system framework diagram of a data transmission method in an embodiment of the present application.
  • FIG. 2 is a schematic diagram of an embodiment of a data transmission method according to an embodiment of the present application.
  • FIG. 3 is a schematic diagram of another embodiment of a data transmission method according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an embodiment of a first device in an embodiment of the present application.
  • FIG. 5 is a schematic diagram of another embodiment of a first device in an embodiment of the present application.
  • FIG. 6 is a schematic diagram of another embodiment of a first device in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of another embodiment of a first device in an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a hardware of a first device in an embodiment of the present application.
  • FIG. 9 is a schematic diagram of an extended structure of an RDMA packet in an embodiment of the present application.
  • FIG. 10 is a schematic diagram of an embodiment of an RDMA extended transmission packet header according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a frame of an ACK extended transmission packet header according to an embodiment of the present disclosure.
  • FIG. 12 is a schematic diagram of an embodiment of a syndrome domain in an ACK extended transmission packet header according to an embodiment of the present disclosure
  • FIG. 13 is a schematic diagram of a frame structure of an RDMA read request and an RDMA read response message in the embodiment of the present application.
  • the present application provides a data transmission method and a first device, which are used to reduce the impact of network packet loss on network transmission efficiency, and improve network transmission reliability.
  • the technical solutions in the present application are clearly and completely described in the following with reference to the drawings in the present application. It is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments.
  • the data transmission method in the embodiment of the present application is mainly applicable to an application scenario of RDMA transmission, and is particularly applicable to a network transmission scenario in which a packet loss rate is high, and a network transmission scenario in which a PFC flow control mechanism is enabled to reduce network transmission reliability, for example,
  • the data transmission method in the embodiment of the present application can effectively solve the data transmission problem in the foregoing application scenario, and improve network transmission efficiency and network transmission reliability.
  • FIG. 1 is a system frame diagram of a data transmission method according to an embodiment of the present application, where a transmission scenario of RDMA transmission is shown: the first application extracts data from the memory, generates an RDMA message, and sets the RDMA message.
  • the remote network card After being cached and sent to the local network card, and then transmitted to the remote network card through the network, the remote network card buffers the received RDMA message, and the second application program retrieves the data from the cache and writes it into the memory. Similarly, the process of the first application reading the data in the second application memory is similar to the description of the above writing process, and details are not described herein again.
  • the network card includes a host channel adapter (HCA).
  • HCA host channel adapter
  • FIG. 2 is a schematic diagram of an embodiment of a data transmission method according to an embodiment of the present application.
  • an embodiment of the data transmission method in the embodiment of the present application includes:
  • the first device encapsulates the first target data in the N RDMA messages according to the RDMA protocol, to obtain N RDMA messages.
  • the first target data is data that the first device needs to write into the memory of the second device, and each of the N RDMA packets has a corresponding packet sequence number (PSN), and N is greater than Or a positive integer equal to 2, each of the N RDMA messages carries the first data write address corresponding to the message data, so the device can directly obtain the message data directly from each RDMA message.
  • PSN packet sequence number
  • the first data write address is a type of address, and the specific value is different.
  • the data write address corresponding to the different value substantially corresponds to a different storage space, and each of the RDMA messages carries a value of the first data write address. Used to indicate a different first data write address.
  • the second data address described below is similar to the first data address described herein, and will not be described below.
  • the memory may be a memory space of an application in the second device, and the first target data may be data stored in an memory space of the application in the first device.
  • the RDMA write message may be an RDMA write message, and the RDMA write message includes the following three types: RDMA write First, RDMA write Middle, and RDMA write Last, and the RDMA extended transport header (RETH) is A field in the RDMA packet format, which carries the destination address of the packet data.
  • the base transport header (BTH) is a field in the RDMA packet format, including the PSN.
  • RDMA write First the first message in the remote direct memory access write message
  • RDMA write First has both BTH and RETH fields, therefore, RDMA write
  • the First message includes the PSN and the first data write address, and the remote direct memory access writes the intermediate message (RDMA write Middle) in the message and the last message in the remote direct memory access write message.
  • RDMA write Last There is only a BTH field in (RDMA write Last).
  • the RTU write Middle and RDMA write Last messages are added to the RDMA write Middle and RDMA write Last messages in the data transmission method of the present application (as shown in the gray part of FIG. 9), so that each of the N RDMA messages is received.
  • FIG. 9 is a schematic diagram of an extended structure of a remote direct memory access write message in the embodiment of the present application, wherein (a) of FIG. 9 shows that there is no extension.
  • the frame structure of the RDMA message, (b) in FIG. 9 shows the frame structure of the extended RDMA message, wherein the gray portion in (b) of FIG. 9 is the newly added RDMA extended transmission message header.
  • FIG. 10 is a schematic diagram of an embodiment of an RDMA extended transmission packet header in the embodiment of the present application, and FIG. 10 takes 32 bits as an example. It should be noted that the frame structure of other parts of the message in FIG. 9 is similar to the message structure in the existing RDMA protocol, and details are not described herein again.
  • the receiving device receives any one of the N RDMA messages.
  • the first data write address corresponding to the message data can be directly obtained, so that it can be directly written into the application memory space without being cached in the cache area.
  • the maximum transmission unit is the maximum packet size that can be transmitted for each RDMA transmission in the RDMA protocol.
  • the number N of the above RDMA packets can be determined according to the data size and the MTU of the first target data.
  • the first device sequentially sends N RDMA messages to the second device according to the sequence of the message sequence number PSN of the N RDMA messages.
  • the packet is an RDMA write Middle packet, and the PSNs are 1, 2, and 3, respectively, and the first device sequentially sends the five RDMA packets to the second device in the order of the PSN.
  • the second device determines, according to the receiving condition of the N RDMA messages, a receiving state of each bit in the bit chart.
  • N RDMA messages and bitmaps are mapped one by one through the PSN, and one PSN corresponds to one bit.
  • the receiving status includes receiving success or receiving failure. For example, a bit of 0 indicates that the message has failed to be received. A bit of 1 indicates that the message was successfully received.
  • the second device may specifically determine the receiving status of each bit in the bit map according to the PSN obtained in the BTH field in the RDMA write message.
  • the second device When the receiving of one or more RDMA packets fails, the second device generates a first retransmission indication message.
  • the negative acknowledgment (NACK) packet is used to notify the sender to retransmit all the packets starting from a lost packet, and it is not possible to retransmit only the lost packet. Therefore, the data transmission method of the present application re-uses the original NACK message format to newly define a selective retransmission indication message: the NACK message of the RDMA protocol includes an acknowledgment extended transport header (AETH). In the field, the definition of the selective retransmission indication message is newly added in the AETH field (as shown in FIG. 11 and FIG. 12), so that the NACK message can instruct the first device to retransmit only the received RDMA message.
  • AETH acknowledgment extended transport header
  • FIG. 11 is a schematic diagram of a frame structure of an ACK extended transmission packet header according to an embodiment of the present disclosure.
  • the ACK extended transmission packet header has a total of 32 bits, wherein 31-24 bits are confirmation indication fields.
  • the range of the confirmation indication (syndrome) field indicates information corresponding to a positive acknowledgement (ACK) or a negative acknowledgement (NACK), and the 0-23 bit is a message sequence number field, and the message sequence number (MSN) The sequence number corresponding to the message that the domain completes in the response response.
  • the embodiment of the present application adds the selective retransmission indication in the ACK extended transmission header (AETH) field, as shown in FIG. 12, FIG.
  • FIG. 12 is an ACK extension provided by the embodiment of the present application.
  • the second device sends a first retransmission indication message to the first device.
  • the second device sends a first retransmission indication message to the first device, where the first retransmission indication message carries the first retransmission PSN, and the first retransmission PSN is the PSN of the RDMA message that the second device receives the failure.
  • the first device determines, according to the first retransmission indication message, the first retransmission packet.
  • the first device parses the first retransmission indication message sent by the second device to obtain a first retransmission PSN, and second, the first device determines, according to the first retransmission PSN, that the N retransmission messages need to be retransmitted. RDMA message to get the first retransmission message.
  • the first device sends a first retransmission packet to the second device.
  • the first device sends a first retransmission message to the second device, where the first retransmission message includes only the RDMA message that the second device receives the failure.
  • each RDMA message carries a first data write address, so that the receiving end directly obtains the first data write address corresponding to the message data after receiving the RDMA message, so that The receiving end can separately store the packet data of each packet without buffering, which saves the buffer space, and the lost packet does not affect the data writing address of the successfully received packet received by the receiving end, thereby making the RDMA Data transmission is not sensitive to network packet loss rate, improve network transmission efficiency, and improve network throughput.
  • the receiving end can instruct the transmitting end to retransmit only the lost data, so that the transmitting end retransmits the lost packet and does not transmit other successfully received packets, which can save network transmission resources and improve network transmission efficiency.
  • FIG. 3 is a schematic diagram of another embodiment of a data transmission method in an embodiment of the present application.
  • another embodiment of the data transmission method in the embodiment of the present application includes:
  • the first device sends a data read request to the second device.
  • the data read request is generated by the first device according to the RDMA protocol, and the data read request includes a data read address and a second data write address, wherein the data read address is the second target data that the first device needs to read.
  • the data read request may specifically be an RDMA read request in the RDMA protocol.
  • the RETH field in the RDMA read request in the RDMA protocol includes the above data read address, but there is no second data write address, which is redesigned in this application.
  • RDMA read request message format (as shown in Figure 13), add a second RDMA extended transmission header RETH2 field in the RDMA read request message (RETH2 is used for read operation, the format can refer to RETH, and the number 2 is used to distinguish In RETH), the RETH2 field carries information with a second data write address (as carried by the virtual address field in Figure 13).
  • FIG. 13 is a schematic diagram of a frame structure of an RDMA read request and an RDMA read response message in the embodiment of the present application.
  • a RETH2 field is added to both the RDMA read request and the RDMA read response message.
  • the RETH field is the field used to represent the memory address in the RDMA protocol, that is, a new RETH2 is added.
  • the field is used to indicate the above second data write address.
  • the second data write address refers to a type of address whose specific values may be different.
  • the RETH2 field in FIG. 13 above is similar to the description in FIG. 10 above, and the related description can be referred to the description in FIG. 10 above.
  • the RDMA read response message may be an RDMA read respond message, and the RDMA read respond message may be classified into an RDMA read respond first, an RDMA read respond middle, and an RDMA read respond last according to the message position.
  • the second device extracts the second target data from the memory according to the data read request message, and encapsulates the second target data in the A RDMA messages to obtain the A RDMA messages.
  • the second device reads the second target data from the storage space corresponding to the data read address in the data read request message, and secondly, the second device encapsulates the second target data in the A RDMA message.
  • Each RDMA message in the A RDMA message includes a second data write address corresponding to the message data.
  • Each RDMA message in the A RDMA message includes a PSN, and A is greater than or equal to 2. Integer.
  • a RDMA message specifically includes three types of messages: RDMA read respond first, RDMA read respond middle, and RDMA read respond last.
  • the current three data write addresses corresponding to the message data are not included in the above three types of RDMA messages. Therefore, in the present application, the above three types of messages are redesigned, and a RETH2 field is added in three types of RDMA read respond first, RDMA read respond middle, and RDMA read respond last, so that the new RETH field is added.
  • the second data write address corresponding to the message data is carried.
  • the RDMA read respond message redesigned in this application is as shown in FIG. 13 above.
  • step 302 The other related descriptions of this step 302 are similar to the description in step 201 above, and are not described herein again.
  • the second device sequentially sends A RDMA messages to the first device according to the PSN sequence of the A RDMA messages.
  • the other three packets are RDMA read respond middle packets, and the PSNs are 1, 2, and 3, respectively, and the second device sequentially sends the five RDMA packets to the first device in the order of the PSN.
  • the first device determines, according to the receiving condition of the A RDMA messages, a receiving state of each bit in the bit chart.
  • This step 304 is similar to the above step 203, and is not described here again. The difference is that the bit chart is maintained by the first device end.
  • the first device When the receiving of one or more RDMA packets fails, the first device generates a second retransmission indication message.
  • the first device sends a second retransmission indication message to the second device.
  • the first device sends a second retransmission indication message to the second device, where the second retransmission indication message carries the second retransmission PSN, and the second retransmission PSN is the PSN of the RDMA message that the first device receives the failure.
  • the second device determines, according to the second retransmission indication message, the second retransmission message.
  • the second device parses the second retransmission indication packet sent by the first device to obtain a second retransmission PSN, and second, the second device determines, according to the second retransmission PSN, that the retransmission needs to be retransmitted in the A RDMA packet. RDMA message to get the second retransmission message.
  • the second device sends a second retransmission packet to the first device.
  • the second device sends a second retransmission message to the first device, where the second retransmission message includes only the RDMA message that the first device receives the failure.
  • an embodiment of the first device 40 in the embodiment of the present application includes:
  • the encapsulating module 401 is configured to encapsulate the first target data in the N RDMA messages according to the remote direct memory access RDMA protocol, where the first target data is data that the first device needs to write into the second device for storing, Any one of the RDMA messages carries a message sequence number PSN, where N is a positive integer greater than or equal to 2;
  • the sending module 402 is configured to sequentially send N RDMA messages to the second device according to the PSN sequence of the N RDMA messages, where each of the N RDMA messages carries a first data write address, first The data write address is to write the data in each of the N RDMA messages to the address of the second device, so that the second device directly obtains the first of each RDMA message from each RDMA message.
  • the data is written to the address, and the target data is written into the storage space corresponding to the first data write address.
  • the first device 50 further includes:
  • the receiving module 503 is configured to: if the N RDMA messages are not completely received by the second device, receive the first retransmission indication message sent by the second device, where the retransmission indication message carries the retransmission PSN, where The PSN is a PSN corresponding to the bit that receives the failure in the bit chart of the second device, and the PSN of each RDMA message in the N RDMA messages corresponds to a bit in the bitmap;
  • the sending module 502 is further configured to: send the first retransmission message to the second device, where the first retransmission message is the RDMA message that is failed to be received by the first retransmission PSN, so that the second device will be the first The data in the transmitted message is stored in the second device.
  • the RDMA extended transmission packet header is carried in any one of the second RDMA packet and the Nth RDMA packet corresponding to the N RDMA packets, where the RDMA extended transmission packet is included.
  • the header is used to indicate the first data write address.
  • an embodiment of the first device 60 in the embodiment of the present application includes: a sending module 601, configured to send a data read request message to a second device, where the data read request message is according to a remote direct memory. Accessing the RDMA protocol generation, and carrying a data read address and a second data write address, the data read address is a destination address of the second target data stored in the second device, and the second data write address is on the first device a destination address reserved for storing the second target data read from the second device, where the second target data is data that the first device needs to read from the second device;
  • the receiving module 602 is configured to receive, by the second device, A RDMA messages sequentially sent according to the PSN sequence of the A RDMA messages, where the A RDMA messages are read by the second device from the data read address.
  • the target data is encapsulated.
  • Each of the A RDMA packets carries a packet sequence number PSN, and A is a positive integer greater than or equal to 2.
  • Each of the A RDMA packets includes each one.
  • the second data write address corresponding to the data of the message;
  • the obtaining module 603 is configured to directly obtain a second data write address of each RDMA message from each of the R RDMA messages that are successfully received, where B is a positive integer less than or equal to A;
  • the data writing module 604 is configured to write data of each RDMA message in the B RDMA messages into the storage space of the second data write address corresponding to each RDMA message.
  • the network device 70 further includes:
  • the determining module 705 is configured to determine, according to the receiving condition of the A RDMA messages, a receiving state of each bit in the bit chart of the first device, where the receiving state is receiving success or receiving failure, and each RDMA packet in the A RDMA packets
  • the PSN of the text corresponds to one bit in the bit map of the first device.
  • the determination module 705 is further configured to:
  • the PSN corresponding to the bit in the bitmap of the first device whose reception status is the reception failure is determined as the second retransmission PSN;
  • the sending module 701 is further configured to: send a second retransmission indication message to the second device, where the second retransmission indication message carries the second retransmission PSN;
  • the receiving module 702 is further configured to receive a second retransmission message sent by the second device according to the second retransmission indication message, where the second retransmission message is a RDMA message corresponding to the second retransmission PSN. .
  • the retransmission PSN is indicated by an ACK Extended Transmission Header field in a Negative Acknowledgement message defined by the RDMA protocol.
  • the data read request carries a second RDMA extended transport header field, where the second RDMA extended transport header field is used to indicate the second data write address.
  • a hardware structure diagram of a first device in the embodiment of the present application includes: a receiver 801, a transmitter 802, a processor 803, a memory 804, and a bus 805.
  • the memory 804 can include read only memory and random access memory and provides instructions and data to the processor 803. A portion of the memory 804 may also include a non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • Memory 804 stores the following elements, executable modules or data structures, or subsets thereof, or their extended sets:
  • Operation instructions including various operation instructions for implementing various operations
  • Operating system Includes a variety of system programs for implementing various basic services and handling hardware-based tasks.
  • the processor 803 in the embodiment of the present application may be used to perform the operations corresponding to the first device in the foregoing embodiment corresponding to FIG. 2, and may include the following operations: encapsulating the first target data according to the remote direct memory access RDMA protocol.
  • the first target data is data that the first device needs to write to the second device for storage, and any one of the N RDMA packets carries a packet sequence number PSN, where N is a positive integer greater than or equal to 2;
  • each of the N RDMA messages carrying the first data write address, and the first data write address is The data in each of the N RDMA messages is written to the address of the second device, so that the second device directly obtains the first data write address in each RDMA message from each RDMA message, and The target data is written into the storage space corresponding to the first data write address.
  • the processor 803 is also used to perform other related operations in the embodiment corresponding to FIG. 2. For detailed description, refer to the description in the embodiment corresponding to FIG. 2, and details are not described herein again.
  • the processor 803 may be configured to perform the operations corresponding to the first device in the foregoing embodiment corresponding to FIG. 3, and may include the following operations: sending a data read request to the second device, and the data read request is based on the remote direct memory access RDMA.
  • the protocol is generated and carries a data read address and a second data write address, where the data read address is a destination address of the second target data stored in the second device, and the second data write address is reserved on the first device a destination address for storing the second target data read from the second device, where the second target data is data that the first device needs to read from the second device;
  • a RDMA messages sequentially sent according to the PSN sequence of the A RDMA messages, and the A RDMA messages are encapsulated by the second device from the second target data read from the data read address,
  • Each of the A RDMA packets carries a packet sequence number PSN, and A is a positive integer greater than or equal to 2.
  • Each of the A RDMA packets includes the data of each packet. Second data write address;
  • the data of each RDMA message in the B RDMA messages is written into the storage space of the second data write address corresponding to each RDMA message.
  • the processor 803 is further configured to perform other related operations in the embodiment corresponding to FIG. 3. For detailed description, refer to the description in the embodiment corresponding to FIG. 3, and details are not described herein again.
  • the processor 803 controls the operation of the first device, and the processor 803 may also be referred to as a central processing unit (CPU).
  • Memory 804 can include read only memory and random access memory and provides instructions and data to processor 803. A portion of the memory 804 may also include an NVRAM.
  • the components of the first device are coupled together by a bus system 805.
  • the bus system 805 may include a power bus, a control bus, a status signal bus, and the like in addition to the data bus. However, for clarity of description, various buses are labeled as bus system 805 in the figure.
  • Processor 803 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 803 or an instruction in a form of software.
  • the processor 803 may be a general-purpose processor, a digital signal processing (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or Other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field-programmable gate array
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly implemented by the hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like.
  • the storage medium is located in memory 804, and processor 803 reads the information in memory 804 and, in conjunction with its hardware, performs the steps of the above method.
  • FIG. 8 The related description of FIG. 8 can be understood by referring to the related descriptions and effects of the method parts of FIG. 2 and FIG. 3, and no further description is made here.
  • the embodiment of the present application further provides a computer storage medium for storing computer software instructions used by the first device, when the computer is running on the computer, so that the computer can execute the corresponding embodiment in FIG. 2 and FIG. A method of data transmission performed by a device.
  • the storage medium may specifically be the foregoing memory 804.
  • the embodiment of the present application further provides a computer program product comprising instructions, which when executed on a computer, enable the computer to execute the data transmission method performed by the first device.
  • the embodiment of the present application further provides a network interface processing circuit, including: a processing circuit and a communication interface circuit, where the communication interface circuit is configured to perform a data transceiving operation, and the processing circuit is configured to perform the foregoing description in FIG. 2 and FIG. Data transmission method.
  • the processing circuit may be an application specific integrated circuit (ASIC), or an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or a transistor logic device, or a discrete hardware component.
  • ASIC application specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • the embodiment of the present application further provides a network card, including: a network interface chip, a memory, and a host interface circuit, where the memory is used to store computer operation instructions, and the host interface circuit is configured to connect the host and the network card, and the network interface chip is used to pass
  • the computer operation instructions described above are invoked to perform the data transfer method described above in the FIGS. 2 and 3.
  • the above memory may be a cache.
  • the embodiment of the present application further provides a network device, including: a network card, a host, and a memory.
  • the network card may be the network card described in the foregoing embodiment, where the memory is used to store computer operation instructions, and the network card is used to receive or send data.
  • the host is configured to execute the data transfer method described in the above-described FIGS. 2 and 3 by invoking the above computer operation instructions.
  • the network device may specifically be a server.
  • the disclosed system, apparatus, and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • a computer readable storage medium A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present application.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program code. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Communication Control (AREA)
  • Detection And Prevention Of Errors In Transmission (AREA)
  • Computer And Data Communications (AREA)
  • Small-Scale Networks (AREA)

Abstract

本申请公开了一种数据传输方法及第一设备，用于降低网络丢包对网络传输效率的影响，同时提高网络传输的可靠性。本申请方法包括：第一设备根据远程直接存储器存取RDMA协议，将第一目标数据封装于N个RDMA报文中；所述第一设备根据所述N个RDMA报文的PSN顺序依次向所述第二设备发送所述N个RDMA报文，所述N个RDMA报文中每一个报文中均携带有第一数据写入地址，所述第一数据写入地址为将所述所述N个RDMA报文中每一个报文中的数据写入所述第二设备的地址，以使得所述第二设备从每一个RDMA报文中直接获取每一个RDMA报文中的第一数据写入地址，并将所述目标数据写入所述第一数据写入地址对应的存储空间中。

Description

一种数据传输方法及第一设备
本申请要求于2017年12月27日提交中国专利局、申请号为201711448880.9、发明名称为“一种数据传输方法及第一设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及网络通信技术领域，尤其涉及一种数据传输方法及第一设备。
背景技术
基于优先级的流量控制(priority-based flow control，PFC)是增强传统以太网流控机制，并避免报文丢失，提高网络吞吐率的一种流控机制。PFC流控机制可以有效降低网络丢包率，但同时也会带来PFC拥塞扩散、PFC死锁、PFC风暴和PFC传输距离限制等问题导致网络传输可靠性降低。
远程直接存储器存取(remote direct memory access，RDMA)协议是一种不需要操作***干预，通过网络直接将数据从一个***传输至另一个***的存储器中的协议。RDMA协议将待传数据封装成一个或多个RDMA报文，将一个或多个RDMA报文从发送端发送至接收端，接收端将接收到的RDMA报文进行缓存于缓存区域中，当接收端成功接收待传数据的全部RDMA报文时，接收端将缓存区域中的缓存数据写入到接收端的存储器中。
RDMA协议采用Go-Back-N重传机制对丢失报文进行重传，Go-Back-N重传机制是一种即使丢包率非常低，也会导致网络传输效率大幅度下降的重传机制，因此RDMA协议对丢包率极其敏感。RDMA协议是为了解决网络通信中服务器端之间的数据传输的延迟而产生的，由于网络通信中服务器端之间的数据传输中几乎不存在报文丢失的情况，其丢包率极低，因此，使用上述RDMA协议有效地解决了网络通信中服务器端之间的传输延迟问题。
RDMA协议可以运行在多种数据链路层协议上，例如以太网，随着RDMA协议的发展，RDMA协议逐渐被应用于以太网中，此种情况下，RDMA协议对丢包率敏感的问题被逐渐显现出来，在RDMA协议应用于以太网中进行数据传输时，以太网中的网络设备需要开启PFC流控机制，以降低网络丢包对网络传输效率的影响，但是随着PFC流控机制的开启，网络传输可靠性也会随之降低。
发明内容
本申请提供了一种数据传输方法及第一设备，用于降低网络丢包对网络传输效率的影响，同时提高网络传输的可靠性。
第一方面，本申请提供了一种数据传输方法，包括：
第一设备根据远程直接存储器存取RDMA协议，将第一目标数据封装于N个RDMA报文中，所述第一目标数据为所述第一设备需要写入第二设备中进行存储的数据，所述N个RDMA报文中的任一个报文均携带有一个报文序列号PSN，所述N为大于或等于2的正整数；
所述第一设备根据所述N个RDMA报文的PSN顺序依次向所述第二设备发送所述N个RDMA报文，所述N个RDMA报文中每一个报文中均携带有第一数据写入地址，所述第一数据写入地址为将所述N个RDMA报文中每一个报文中的数据写入所述第二设备的地址，以使得所述第二设备从每一个RDMA报文中直接获取每一个RDMA报文中的第一数据写入地址，并将所述目标数据写入所述第一数据写入地址对应的存储空间中。
从上述技术方案可以看出，本申请数据传输方法具有以下优点：
由于本技术方案的每一个RDMA报文中均携带有第一数据写入地址，并且第一数据写入地址是将RDMA报文中的数据写入第二设备中的地址，因此，第二设备可以接收到每一个RDMA报文都可以直接将从报文中获取对应的第一数据写入地址，将报文中的数据立即写入第二设备的存储器中，可以理解，这样即使存在RDMA报文丢失，也不会影响其他接收成功的RDMA报文写入到第二设备的存储器中，从而可以降低网络丢包对网络传输速率的影响，避免了其中一个或多个RDMA报文丢失第二设备无法确定其他RDMA报文的写入地址，进而使得大量数据缓存，降低网络传输效率的情况发生，从而，本申请数据传输方法可以使得存在丢包也可以保证传输成功的目标数据存储到第二设备的存储空间中，在即使有目标数据传输失败的情况下也具有一定得网络传输速率，从而降低网络丢包对网络传输速率的影响，同时使得网络传输的可靠性也得到一定的提高。
结合本申请实施例的第一方面，在第一方面的第一种可能的实现方式中，在所述第一设备根据所述N个RDMA报文的PSN顺序依次向所述第二设备发送所述N个RDMA报文之后，所述方法包括：
若所述N个RDMA报文未被所述第二设备完全接收成功，则所述第一设备接收所述第二设备发送的第一重传指示报文，所述重传指示报文中携带有重传PSN，所述重传PSN为所述第二设备的位图表中接收状态为接收失败的比特位对应的PSN，所述N个RDMA报文中每一个RDMA报文的PSN对应所述位图表中的一个比特位；
所述第一设备向所述第二设备发送重传报文，所述重传报文为所述重传PSN对应的接收失败的RDMA报文，以使得所述第二设备将所述重传报文中的数据存储于所述第二设备中。
重传指示报文指示的是第二设备接收失败的RDMA报文的PSN，在重传过程中，接收端可以指示发送端只重传丢失的数据，以使得发送端重新传输丢失的报文不会传输其他接收成功的报文，可以节约网络传输资源同时提高网络传输效率。
结合本申请实施例的第一方面或第一方面的第一种可能的实现方式，在第一方面的第二种可能的实现方式种，上述N个RDMA报文对应的第二个RDMA报文至第N个RDMA报文中任一个报文中均携带有RDMA扩展传输报文头，该RDMA扩展传输报文头用于指示上述第一数据写入地址。
第二方面，本申请提供了一种数据传输方法，包括：
第一设备向第二设备发送数据读取请求，所述数据读取请求是根据远程直接存储器存取RDMA协议生成，并且携带有数据读取地址和第二数据写入地址，所述数据读取地址为所述第二设备中存储第二目标数据的目的地址，所述第二数据写入地址为所述第一设备上 预留的用于存储从所述第二设备中读取到的所述第二目标数据的目的地址，所述第二目标数据为所述第一设备需要从所述第二设备中读取的数据；
所述第一设备接收所述第二设备根据A个RDMA报文的PSN顺序依次发送的所述A个RDMA报文，所述A个RDMA报文是由所述第二设备将从所述数据读取地址中读取到的所述第二目标数据进行封装得到，所述A个RDMA报文中每一个报文均携带有一个报文序列号PSN，所述A为大于或等于2的正整数，所述A个RDMA报文中每一个报文均包括所述每一个报文的数据对应的所述第二数据写入地址；
所述第一设备从接收成功的B个RDMA报文中每一个RDMA报文中直接获取每一个RDMA报文的所述第二数据写入地址，所述B为小于或等于A的正整数；
所述第一设备将所述B个RDMA报文中每一个RDMA报文的数据写入所述每一个RDMA报文对应的所述第二数据写入地址的存储空间中。
结合本申请的第二方面，在本申请第二方面的第一种可能的实现方式中，在所述第一设备接收所述第二设备根据A个RDMA报文的PSN顺序依次发送的所述A个RDMA报文之后，所述方法还包括：
所述第一设备根据所述A个RDMA报文的接收情况确定所述第一设备的位图表中每一个比特位的接收状态，所述接收状态为接收成功或接收失败，所述A个RDMA报文中每一个RDMA报文的PSN对应所述第一设备的位图表中的一个比特位。
结合本申请的第二方面或本申请第二方面的第一种可能的实现方式，在本申请第二方面的第二种可能的实现方式中，在所述第一设备接收所述第二设备根据A个RDMA报文的PSN顺序依次发送的所述A个RDMA报文之后，所述方法还包括：
当所述B小于所述A时，所述第一设备将所述第一设备的位图表中接收状态为接收失败的比特位对应的PSN确定为重传PSN；
所述第一设备向所述第二设备发送重传指示报文，所述重传指示报文中携带有所述重传PSN；
所述第一设备接收所述第二设备根据所述重传指示报文发送的重传报文，所述重传报文为所述重传PSN对应的接收失败的RDMA报文。
结合本申请第二方面的第二种可能的实现方式，在本申请第二方面的第三种可能的实现方式中，上述重传PSN是通过RDMA协议的否定确认报文中的ACK扩展传输报文头字段指示的。
结合本申请第二方面、第二方面的第一种可能的实现方式至第三种可能的实现方式中的任一种实现方式，上述数据读取请求中携带有第二RDMA扩展传输报文头字段，该第二RDMA扩展传输报文头字段用于指示所述第二数据写入地址。
第二方面以及其两种实现方式对应的有益效果与上述第一方面的有益效果类似，对此此处不再赘述。
第三方面本申请实施例提供一种网络设备，该网络设备具有实现上述第一方面或第二方面中网络设备行为的功能。该功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。该硬件或软件包括一个或多个与上述功能相对应的模块。
第四方面，本申请实施例提供一种网络设备，包括：处理器、存储器、总线和通信接口；该存储器用于存储计算机执行指令，该处理器与该存储器通过该总线连接，当该网络设备运行时，该处理器执行该存储器存储的该计算机执行指令，以使该网络设备执行如上述第一方面或第二方面中任意一项的数据传输方法。
第五方面，本申请实施例提供了一种计算机可读存储介质，用于储存为上述网络设备所用的计算机软件指令，当其在计算机上运行时，使得计算机可以执行上述第一方面或第二方面中任意一项的数据传输方法。
第六方面，本申请实施例提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机可以执行上述第一方面或第二方面中任意一项的数据传输方法。
第七方面，本申请实施例提供了一种网络接口处理电路，包括：处理电路和通信接口电路，该通信接口电路用于执行数据收发操作，该处理电路用于执行上述第一方面或第二方面中任意一项的数据传输方法。可选的，上述处理电路具体可以是专用集成电路(application specific integrated circuit，ASIC)，也可以是现成可编程门阵列(field－programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
第八方面，本申请实施例提供了一种网卡，包括：网络接口芯片、存储器和主机接口电路，该存储器用于存储计算机操作指令，主机接口电路用于连接主机与该网卡，该网络接口芯片用于通过调用上述计算机操作指令以执行上述第一方面或第二方面中任意一项的数据传输方法。可选的，上述存储器可以是缓存。
第九方面，本申请实施例提供了一种网络设备，包括：网卡、主机和存储器，该网卡可以是上述第八方面中所述的网卡，存储器用于存储计算机操作指令，网卡用于接收或发送数据，该主机用于通过调用上述计算机操作指令以执行上述第一方面或第二方面中任意一项的数据传输方法。可选的，该网络设备具体可以是服务器。
另外，第三方面至第九方面中任一种设计方式所带来的技术效果可参见第一方面中不同设计方式所带来的技术效果，此处不再赘述。
附图说明
图1为本申请实施例中数据传输方法的一个***框架图；
图2为本申请实施例中数据传输方法的一个实施例示意图；
图3为本申请实施例中数据传输方法的另一个实施例示意图；
图4为本申请实施例中第一设备的一个实施例示意图；
图5为本申请实施例中第一设备的另一个实施例示意图；
图6为本申请实施例中第一设备的另一个实施例示意图；
图7为本申请实施例中第一设备的另一个实施例示意图；
图8为本申请实施例中第一设备的一个硬件结构示意图；
图9为本申请实施例中RDMA报文的一个扩展结构示意图；
图10为本申请实施例中RDMA扩展传输报文头的一个实施例示意图；
图11为本申请实施例提供的ACK扩展传输报文头的一个帧结构示意图；
图12为本申请实施例提供的ACK扩展传输报文头中syndrome域的一个实施例示意图；
图13为本申请实施例中RDMA读取请求与RDMA读取响应报文的一个帧结构示意图。
具体实施方式
本申请提供了一种数据传输方法及第一设备，用于降低网络丢包对网络传输效率的影响，同时提高网络传输的可靠性。下面将结合本申请中的附图，对本申请中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、***、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
本申请实施例的数据传输方法主要适用于RDMA传输的应用场景中，尤其适用于一些丢包率较高的网络传输场景，以及开启PFC流控机制导致网络传输可靠性降低的网络传输场景，例如将RDMA协议融合到传统以太网中进行数据传输的场景，本申请实施例的数据传输方法可以有效解决上述应用场景中的数据传输问题，提高网络传输效率以及网络传输可靠性。
当一个应用程序发起RDMA读/写请求时，***并不执行数据复制动作，这就减少了处理网络通信时在内核空间和用户空间进行上下文切换的次数。在不需要任何内核内存参与的条件下，RDMA请求从运行在用户空间中的应用程序发送到本地网卡，然后经过网络传送到远端网卡，因此，RDMA传输不需要操作***参与，不会增加***负载。如图1所示为本申请实施例数据传输方法的一个***框架图，图中示出了RDMA传输的一个传输场景：第一应用程序从内存中取出数据生成RDMA报文，并将RDMA报文经缓存发送至本地网卡中，然后通过网络传输至远端网卡中，远端网卡将接收到的RDMA报文进行缓存，第二应用程序从缓存中取出数据写入到内存中。同样，第一应用程序读取第二应用程序内存中的数据的过程与上述写入过程的描述类似，此处不再赘述。另外，网卡包括主机通道适配卡(host channel adapter，HCA)。
为了便于理解本申请实施例中的数据传输方法，下面分别从数据写入以及数据读取两个方面对本申请实施例中的数据传输方法进行详细描述，具体如下：
一、基于RDMA的数据写入方法
图2为本申请实施例中数据传输方法的一个实施例示意图。
如图2所示，本申请实施例中的数据传输方法的一个实施例，包括：
201、第一设备根据RDMA协议将第一目标数据封装于N个RDMA报文中，以得到N个 RDMA报文。
第一目标数据为第一设备需要写入第二设备的存储器中的数据，N个RDMA报文中每一个报文均有一个对应的报文序列号(packet sequence number，PSN)，N为大于或等于2的正整数，N个RDMA报文中每一个报文均携带有该报文数据对应的第一数据写入地址，因此，设备可以直接从每一个RDMA报文中直接得到报文数据对应的第一数据写入地址。
第一数据写入地址是一类地址，其具体值不同，不同值对应的数据写入地址实质上对应不同的存储空间，上述每一个RDMA报文均会携带一个第一数据写入地址的值，用于指示不同的第一数据写入地址，类似的，下文中所述的第二数据地址与此处描述的第一数据地址类似，下文中将不再赘述。
存储器可以是第二设备中应用程序的内存空间，第一目标数据可以是第一设备中应用程序的内存空间中存储的数据。
N个RDMA报文具体可以是RDMA write报文，RDMA write报文包括如下三种类型：RDMA write First、RDMA write Middle和RDMA write Last，RDMA扩展传输报文头(RDMA extended transport header，RETH)是RDMA报文格式中的一个字段，该字段中携带有报文数据的目的地址，基本传输报文头(base transport header，BTH)是RDMA报文格式中的一个字段，其中包括PSN。在现有远程直接存储器存取写入(RDMA write)报文中，远程直接存储器存取写入报文中的第一个报文(RDMA write First)同时具有BTH和RETH字段，因此，RDMA write First报文中包括PSN和第一数据写入地址，而远程直接存储器存取写入报文中的中间报文(RDMA write Middle)和远程直接存储器存取写入报文中的最后一个报文(RDMA write Last)中仅仅存在BTH字段。基于上述报文格式，本申请数据传输方法中将RDMA write Middle和RDMA write Last报文中增加RETH字段(如图9中灰色部分所示)，以使得上述N个RDMA报文中每一个报文均携带有第一数据写入地址，如图9所示为本申请实施例中远程直接存储器存取写入报文的扩展结构示意图，其中，图9中的(a)示出了没有扩展的RDMA报文的帧结构，图9中的(b)示出了扩展后的RDMA报文的帧结构，其中，图9的(b)中的灰色部分为新增的RDMA扩展传输报文头。进一步，图10为本申请实施例中RDMA扩展传输报文头的一个实施例示意图，图10中以32比特位为例。需要说明的是，图9中报文的其他部分帧结构与现有RDMA协议中的报文结构类似，对此此处不再过多赘述。
由于本申请中的RDMA write First、RDMA write Middle和RDMA write Last三种RDMA write报文中均携带有RETH字段，因此，接收设备在接收到上述N个RDMA报文中的任意一个报文时，可以直接获取报文数据对应的第一数据写入地址，以使得无需在缓存区域中缓存便可以直接写入到应用程序的内存空间中。
最大传输单元(maximum transmission unit，MTU)是RDMA协议中每一次RDMA传输可以传输的最大数据包大小，上述RDMA报文的数量N可以根据第一目标数据的数据量大小和MTU确定。
202、第一设备根据N个RDMA报文的报文序列号PSN顺序依次向第二设备发送N个RDMA报文。
例如，根据上述步骤201得到5个RDMA报文，其中，第一个RDMA报文为RDMA write First报文其PSN＝0，第五个报文为RDMA write Last报文其PSN＝4，其余三个报文为RDMA write Middle报文，其PSN分别为1、2和3，则第一设备按照PSN的先后顺序依次向第二设备发送上述5个RDMA报文。
203、第二设备根据N个RDMA报文的接收情况确定位图表中每一个比特位的接收状态。
针对每一次RDMA传输，N个RDMA报文与位图表通过PSN建立一一映射关系，一个PSN对应一个比特位。接收状态包括接收成功或接收失败，例如比特位为0表示报文接收失败，比特位为1表示报文接收成功。
第二设备具体可以根据RDMA write报文中的BTH字段中得到的PSN确定位图表中每一个比特位的接收状态。
204、当一个或多个RDMA报文接收失败时，第二设备生成第一重传指示报文。
当N个报文中的一个或多个RDMA报文接收失败时，第二设备将位图表中的接收失败的比特位对应的PSN确定为第一重传PSN，其次，第二设备根据第一重传PSN生成第一重传指示报文，第一重传指示报文为一种选择性指示报文，用于指示第一设备仅仅传输未接收成功的一个或多个报文，例如，若PSN＝1和PSN＝4的报文接收失败，则第一重传指示报文用于指示第一设备只传输PSN＝1和PSN＝4两个报文，而无需从PSN＝1的报文开始传输直至传输完后续所有报文。
RDMA协议中通过否定确认(negative acknowledgment，NACK)报文通知发送端重传从某个丢失报文开始的所有报文，无法仅仅重传丢失报文。因此，本申请数据传输方法重新利用原有的NACK报文格式新定义了一种选择性重传指示报文：RDMA协议的NACK报文中包括ACK扩展传输报文头(acknowledgment extended transport header，AETH)字段，在AETH字段中新增加上述选择性重传指示报文的定义(如图11，图12所示)，以使得NACK报文可以指示第一设备只重传接收失败的RDMA报文。
图11为本申请实施例提供的ACK扩展传输报文头的一个帧结构示意图，如图11所示，ACK扩展传输报文头共32个比特位，其中，31-24比特位为确认指示域，该确认指示(syndrome)域这个范围指示的是肯定确认(ACK)或否定确认(NACK)对应的信息，0-23比特位为消息序列号域，该消息序列号(message sequence number，MSN)域在应答响应中完成传输的消息对应的序列号。进一步的，如上所述本申请实施例在ACK扩展传输报文头(AETH)字段中新增加了上述选择性重传指示，具体如图12所示，图12为本申请实施例提供的ACK扩展传输报文头中syndrome域的一个实施例示意图，与现有syndrome域相比，图12中灰色部分为新增加的选择性重传指示，即比特位(7-5)为：(0 1 0)态，可选的，选择性重传指示可以具体指示重传报文的数目以及起始重传报文序列号。除上述(0 1 0)态外为新增指示状态外，其余四种状态(000，001，011，100)均是现有的，对此本申请不做详细描述。
205、第二设备向第一设备发送第一重传指示报文。
第二设备向第一设备发送第一重传指示报文，第一重传指示报文中携带有第一重传 PSN，第一重传PSN为第二设备接收失败的RDMA报文的PSN。
206、第一设备根据第一重传指示报文确定第一重传报文。
首先，第一设备对第二设备发送的第一重传指示报文进行解析，得到第一重传PSN，其次，第一设备根据第一重传PSN确定上述N个RDMA报文中需要重传的RDMA报文，以得到第一重传报文。
207、第一设备向第二设备发送第一重传报文。
第一设备向第二设备发送第一重传报文，其中，第一重传报文中仅仅包括第二设备接收失败的RDMA报文。
本实施例中，每一个RDMA报文中均携带有一个第一数据写入地址，可以使得接收端在接收到RDMA报文之后直接获取到报文数据对应的第一数据写入地址，以使得接收端可以单独将每一个报文的报文数据进行存储，无需缓存，这样节约了缓存空间，并且丢失的报文不会影响接收端获知成功接收的报文的数据写入地址，从而使得RDMA数据传输对网络丢包率不敏感，提高网络传输效率，提升网络吞吐率。
在重传过程中，接收端可以指示发送端只重传丢失的数据，以使得发送端重新传输丢失的报文不会传输其他接收成功的报文，可以节约网络传输资源同时提高网络传输效率。
二、基于RDMA的数据读取方法
图3为本申请实施例中数据传输方法的另一个实施例示意图。
如图3所示，本申请实施例中数据传输方法的另一个实施例，包括：
301、第一设备向第二设备发送数据读取请求。
数据读取请求是第一设备根据RDMA协议生成的，数据读取请求中包括数据读取地址和第二数据写入地址，其中，数据读取地址为第一设备需要读取的第二目标数据存储于第二设备中的目的地址，第二数据写入地址为第一设备在读取到第二设备中的第二目标数据之后用于存储第二目标数据的目的地址。
数据读取请求具体可以是RDMA协议中的RDMA read request，目前RDMA协议中的RDMA read request中的RETH字段中包括上述数据读取地址，但是没有第二数据写入地址，本申请中重新设计了RDMA read request报文格式(如图13所示)，在RDMA read request报文中增加一个第二RDMA扩展传输报文头RETH2字段(RETH2用于读操作，格式可参考RETH，数字2用于区别于RETH)，该RETH2字段携带有第二数据写入地址的信息(如通过图13中的虚拟地址字段来携带)。
图13为本申请实施例中RDMA读取请求与RDMA读取响应报文的一个帧结构示意图。如图13中灰色部分所示，在RDMA读取请求中以及RDMA读取响应报文中均增加了一个RETH2字段，RETH字段是RDMA协议中用于表示内存地址的字段，即新增加了一个RETH2字段用于指示上述第二数据写入地址。第二数据写入地址指的是一类地址，其具体的值可以不同。上述图13中RETH2字段的与上述图10中的描述类似，其相关描述可参阅上述图10中的描述。可选的，上述RDMA读取响应报文具体可以是RDMA read respond报文，RDMA read respond报文根据报文位置可以划分RDMA read respond first、RDMA read respond middle和RDMA read respond last三类。
302、第二设备根据数据读取请求报文从存储器中取出第二目标数据，并将第二目标数据封装于A个RDMA报文中，以得到A个RDMA报文。
首先，第二设备从数据读取请求报文中的数据读取地址对应的存储空间中读取出第二目标数据，其次，第二设备将第二目标数据封装于A个RDMA报文中，使得A个RDMA报文中每一个RDMA报文均包括报文数据对应的第二数据写入地址，A个RDMA报文中每一个RDMA报文均包括一个PSN，A为大于或等于2的正整数。
A个RDMA报文具体包括RDMA read respond first、RDMA read respond middle和RDMA read respond last三种报文，但目前RDMA中的上述三种报文中没有该报文数据对应的第二数据写入地址，因此，在本申请中重新设计了上述三种报文，在RDMA read respond first、RDMA read respond middle和RDMA read respond last三种报文中均增加了一个RETH2字段，以使得新增的RETH字段中携带报文数据对应的第二数据写入地址。其中，本申请中重新设计的RDMA read respond报文如上述图13中所示。
本步骤302的其他相关描述与上述步骤201中的描述类似，对此此处不再赘述。
303、第二设备根据A个RDMA报文的PSN顺序依次向第一设备发送A个RDMA报文。
例如，根据上述步骤302得到5个RDMA报文，其中，第一个RDMA报文为RDMA read respond first报文其PSN＝0，第五个报文为RDMA read respond last报文其PSN＝4，其余三个报文为RDMA read respond middle报文，其PSN分别为1、2和3，则第二设备按照PSN的先后顺序依次向第一设备发送上述5个RDMA报文。
304、第一设备根据A个RDMA报文的接收情况确定位图表中每一个比特位的接收状态。
该步骤304与上述步骤203类似，此处不再赘述，区别点在于位图表是第一设备端维护的。
305、当一个或多个RDMA报文接收失败时，第一设备生成第二重传指示报文。
当A个报文中的一个或多个RDMA报文接收失败时，第一设备将位图表中的接收失败的比特位对应的PSN确定为第二重传PSN，其次，第一设备根据第二重传PSN生成第二重传指示报文，第二重传指示报文为一种选择性指示报文，用于指示第二设备仅仅传输未接收成功的一个或多个报文，例如，若PSN＝1和PSN＝4的报文接收失败，则第二重传指示报文用于指示第二设备只传输PSN＝1和PSN＝4两个报文，而无需从PSN＝1的报文开始传输直至传输完后续所有报文。
本步骤中关于选择性重传指示报文的定义及相关描述与上述步骤204中的描述类似，对此此处不再赘述。
306、第一设备向第二设备发送第二重传指示报文。
第一设备向第二设备发送第二重传指示报文，第二重传指示报文中携带有第二重传PSN，第二重传PSN为第一设备接收失败的RDMA报文的PSN。
307、第二设备根据第二重传指示报文确定第二重传报文。
首先，第二设备对第一设备发送的第二重传指示报文进行解析，得到第二重传PSN，其次，第二设备根据第二重传PSN确定上述A个RDMA报文中需要重传的RDMA报文，以得 到第二重传报文。
308、第二设备向第一设备发送第二重传报文。
第二设备向第一设备发送第二重传报文，其中，第二重传报文中仅仅包括第一设备接收失败的RDMA报文。
本实施例中，数据读取方法对应的有益效果与上述数据写入方法实施例的有益效果类似，此处不再赘述。
如图4所示，本申请实施例中第一设备40的一个实施例，包括：
封装模块401，用于根据远程直接存储器存取RDMA协议，将第一目标数据封装于N个RDMA报文中，第一目标数据为第一设备需要写入第二设备中进行存储的数据，N个RDMA报文中的任一个报文均携带有一个报文序列号PSN，N为大于或等于2的正整数；
发送模块402，用于根据N个RDMA报文的PSN顺序依次向第二设备发送N个RDMA报文，N个RDMA报文中每一个报文中均携带有第一数据写入地址，第一数据写入地址为将N个RDMA报文中每一个报文中的数据写入第二设备的地址，以使得第二设备从每一个RDMA报文中直接获取每一个RDMA报文中的第一数据写入地址，并将目标数据写入第一数据写入地址对应的存储空间中。
在一种示例中，如图5所示，第一设备50还包括：
接收模块503，用于若N个RDMA报文未被第二设备完全接收成功，则接收第二设备发送的第一重传指示报文，重传指示报文中携带有重传PSN，该重传PSN为第二设备的位图表中接收状态为接收失败的比特位对应的PSN，N个RDMA报文中每一个RDMA报文的PSN对应位图表中的一个比特位；
发送模块502，还用于：向第二设备发送第一重传报文，第一重传报文为第一重传PSN对应的接收失败的RDMA报文，以使得第二设备将第一重传报文中的数据存储于第二设备中。
在一种示例中，上述N个RDMA报文对应的第二个RDMA报文至第N个RDMA报文中任一个报文中均携带有RDMA扩展传输报文头，其中，该RDMA扩展传输报文头用于指示上述第一数据写入地址。
如图6所示，本申请实施例中第一设备60的一个实施例，包括：发送模块601，用于向第二设备发送数据读取请求报文，数据读取请求报文根据远程直接存储器存取RDMA协议生成，并且携带有数据读取地址和第二数据写入地址，数据读取地址为第二设备中存储第二目标数据的目的地址，第二数据写入地址为第一设备上预留的用于存储从第二设备中读取到的第二目标数据的目的地址，第二目标数据为第一设备需要从第二设备中读取的数据；
接收模块602，用于接收第二设备根据A个RDMA报文的PSN顺序依次发送的A个RDMA报文，A个RDMA报文由第二设备将从数据读取地址中读取到的第二目标数据进行封装得到，A个RDMA报文中每一个报文均携带有一个报文序列号PSN，A为大于或等于2的正整数，A个RDMA报文中每一个报文均包括每一个报文的数据对应的第二数据写入地址；
获取模块603，用于从接收成功的B个RDMA报文中每一个RDMA报文中直接获取每一 个RDMA报文的第二数据写入地址，B为小于或等于A的正整数；
数据写入模块604，用于将B个RDMA报文中每一个RDMA报文的数据写入每一个RDMA报文对应的第二数据写入地址的存储空间中。在一种示例中，如图7所示，网络设备70还包括：
确定模块705，用于根据A个RDMA报文的接收情况确定第一设备的位图表中每一个比特位的接收状态，接收状态为接收成功或接收失败，A个RDMA报文中每一个RDMA报文的PSN对应第一设备的位图表中的一个比特位。在另一种示例中，确定模块705还用于：
当B小于A时，将第一设备的位图表中接收状态为接收失败的比特位对应的PSN确定为第二重传PSN；
发送模块701，还用于：向第二设备发送第二重传指示报文，第二重传指示报文中携带有第二重传PSN；
接收模块702，还用于接收第二设备根据所述第二重传指示报文发送的第二重传报文，第二重传报文为第二重传PSN对应的接收失败的RDMA报文。
在一种示例中，上述重传PSN是通过所述RDMA协议定义的否定确认报文中的ACK扩展传输报文头字段指示的。
在一种示例中，上述数据读取请求中携带有第二RDMA扩展传输报文头字段，其中该第二RDMA扩展传输报文头字段用于指示上述第二数据写入地址。
本实施例中的其他相关描述及有益效果可参阅上述图2和图3对应的实施例中的对于第一设备的描述，对此此处不再赘述。
如图8所示，本申请实施例中第一设备的一个硬件结构示意图，包括：接收器801、发送器802、处理器803、存储器804和总线805。
其中，存储器804可以包括只读存储器和随机存取存储器，并向处理器803提供指令和数据。存储器804的一部分还可以包括非易失性随机存取存储器(non-volatile random access memory，NVRAM)。
存储器804存储了如下的元素，可执行模块或者数据结构，或者它们的子集，或者它们的扩展集：
操作指令：包括各种操作指令，用于实现各种操作；
操作***：包括各种***程序，用于实现各种基础业务以及处理基于硬件的任务。
其中，本申请实施例中处理器803可以用于执行上述图2对应的实施例中第一设备对应的操作，可以包括如下操作：根据远程直接存储器存取RDMA协议，将第一目标数据封装于N个RDMA报文中，第一目标数据为第一设备需要写入第二设备中进行存储的数据，N个RDMA报文中的任一个报文均携带有一个报文序列号PSN，N为大于或等于2的正整数；
根据N个RDMA报文的PSN顺序依次向第二设备发送N个RDMA报文，N个RDMA报文中每一个报文中均携带有第一数据写入地址，第一数据写入地址为将N个RDMA报文中每一个报文中的数据写入第二设备的地址，以使得第二设备从每一个RDMA报文中直接获取每一个RDMA报文中的第一数据写入地址，并将目标数据写入第一数据写入地址对应的存储空 间中。
处理器803还用于执行图2对应的实施例中的其他相关操作，详细描述可参阅图2对应的实施例中的描述，对此此处不再赘述。
另外，处理器803可以用于执行上述图3对应的实施例中第一设备对应的操作，可以包括如下操作：向第二设备发送数据读取请求，数据读取请求根据远程直接存储器存取RDMA协议生成，并且携带有数据读取地址和第二数据写入地址，数据读取地址为第二设备中存储第二目标数据的目的地址，第二数据写入地址为第一设备上预留的用于存储从第二设备中读取到的第二目标数据的目的地址，第二目标数据为第一设备需要从第二设备中读取的数据；
接收第二设备根据A个RDMA报文的PSN顺序依次发送的A个RDMA报文，A个RDMA报文由第二设备将从数据读取地址中读取到的第二目标数据进行封装得到，A个RDMA报文中每一个报文均携带有一个报文序列号PSN，A为大于或等于2的正整数，A个RDMA报文中每一个报文均包括每一个报文的数据对应的第二数据写入地址；
从接收成功的B个RDMA报文中每一个RDMA报文中直接获取每一个RDMA报文的第二数据写入地址，B为小于或等于A的正整数；
将B个RDMA报文中每一个RDMA报文的数据写入每一个RDMA报文对应的第二数据写入地址的存储空间中。处理器803还用于执行图3对应的实施例中的其他相关操作，详细描述可参阅图3对应的实施例中的描述，对此此处不再赘述。
处理器803控制第一设备的操作，处理器803还可以称为中央处理单元(central processing unit，CPU)。存储器804可以包括只读存储器和随机存取存储器，并向处理器803提供指令和数据。存储器804的一部分还可以包括NVRAM。具体的应用中，第一设备的各个组件通过总线***805耦合在一起，其中总线***805除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为总线***805。
上述本申请实施例揭示的方法可以应用于处理器803中，或者由处理器803实现。处理器803可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器803中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器803可以是通用处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现成可编程门阵列(field－programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器804，处理器803读取存储器804中的信息，结合其硬件完成上述方法的步骤。
图8的相关描述可以参阅图2和图3方法部分的相关描述和效果进行理解，此处不做过多赘述。
本申请实施例还提供了一种计算机存储介质，用于储存为上述第一设备所用的计算机软件指令，当其在计算机上运行时，使得计算机可以执行图2和图3中对应实施例中第一设备所执行的数据传输方法。其中，该存储介质具体可以为上述存储器804。
本申请实施例还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机可以执行上述第一设备所执行的数据传输方法。
本申请实施例还提供了一种网络接口处理电路，包括：处理电路和通信接口电路，该通信接口电路用于执行数据收发操作，该处理电路用于执行上述图2和图3部分所述的数据传输方法。可选的，上述处理电路具体可以是专用集成电路(ASIC)，也可以是现成可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
本申请实施例还提供了一种网卡，包括：网络接口芯片、存储器和主机接口电路，该存储器用于存储计算机操作指令，主机接口电路用于连接主机与该网卡，该网络接口芯片用于通过调用上述计算机操作指令以执行上述图2和图3部分所述的数据传输方法。可选的，上述存储器可以是缓存。
本申请实施例还提供了一种网络设备，包括：网卡、主机和存储器，该网卡可以是上述实施例中所述的网卡，存储器用于存储计算机操作指令，网卡用于接收或发送数据，该主机用于通过调用上述计算机操作指令以执行上述图2和图3部分所述的数据传输方法。可选的，该网络设备具体可以是服务器。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的***，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出 来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案范围。

Claims (16)

  1. 一种数据传输方法，其特征在于，包括：
    第一设备根据远程直接存储器存取RDMA协议，将第一目标数据封装于N个RDMA报文中，所述第一目标数据为所述第一设备需要写入第二设备中进行存储的数据，所述N个RDMA报文中的任一个报文均携带有一个报文序列号PSN，所述N为大于或等于2的正整数；
    所述第一设备根据所述N个RDMA报文的PSN顺序依次向所述第二设备发送所述N个RDMA报文，所述N个RDMA报文中每一个报文中均携带有第一数据写入地址，所述第一数据写入地址为将所述N个RDMA报文中每一个报文中的数据写入所述第二设备的地址，以使得所述第二设备从每一个RDMA报文中直接获取每一个RDMA报文中的第一数据写入地址，并将所述目标数据写入所述第一数据写入地址对应的存储空间中。
  2. 根据权利要求1所述的方法，其特征在于，在所述第一设备根据所述N个RDMA报文的PSN顺序依次向所述第二设备发送所述N个RDMA报文之后，所述方法包括：
    若所述N个RDMA报文未被所述第二设备完全接收成功，则所述第一设备接收所述第二设备发送的重传指示报文，所述重传指示报文中携带有重传PSN，所述重传PSN为所述第二设备的位图表中接收状态为接收失败的比特位对应的PSN，所述N个RDMA报文中每一个RDMA报文的PSN对应所述位图表中的一个比特位；
    所述第一设备向所述第二设备发送重传报文，所述重传报文为所述重传PSN对应的接收失败的RDMA报文，以使得所述第二设备将所述重传报文中的数据存储于所述第二设备中。
  3. 根据权利要求1或2所述的方法，其特征在于，所述N个RDMA报文对应的第二个RDMA报文至第N个RDMA报文中任一个报文中均携带有RDMA扩展传输报文头，所述RDMA扩展传输报文头用于指示所述第一数据写入地址。
  4. 一种数据传输方法，其特征在于，包括：
    第一设备向第二设备发送数据读取请求，所述数据读取请求是根据远程直接存储器存取RDMA协议生成，并且携带有数据读取地址和第二数据写入地址，所述数据读取地址为所述第二设备中存储第二目标数据的目的地址，所述第二数据写入地址为所述第一设备上预留的用于存储从所述第二设备中读取到的所述第二目标数据的目的地址，所述第二目标数据为所述第一设备需要从所述第二设备中读取的数据；
    所述第一设备接收所述第二设备根据A个RDMA报文的PSN顺序依次发送的所述A个RDMA报文，所述A个RDMA报文是由所述第二设备将从所述数据读取地址中读取到的所述第二目标数据进行封装得到，所述A个RDMA报文中每一个报文均携带有一个报文序列号PSN，所述A为大于或等于2的正整数，所述A个RDMA报文中每一个报文均包括所述每一个报文的数据对应的所述第二数据写入地址；
    所述第一设备从接收成功的B个RDMA报文中每一个RDMA报文中直接获取每一个RDMA 报文的所述第二数据写入地址，所述B为小于或等于A的正整数；
    所述第一设备将所述B个RDMA报文中每一个RDMA报文的数据写入所述每一个RDMA报文对应的所述第二数据写入地址的存储空间中。
  5. 根据权利要求4所述的方法，其特征在于，在所述第一设备接收所述第二设备根据A个RDMA报文的PSN顺序依次发送的所述A个RDMA报文之后，所述方法还包括：
    所述第一设备根据所述A个RDMA报文的接收情况确定所述第一设备的位图表中每一个比特位的接收状态，所述接收状态为接收成功或接收失败，所述A个RDMA报文中每一个RDMA报文的PSN对应所述第一设备的位图表中的一个比特位。
  6. 根据权利要求4或5所述的方法，其特征在于，在所述第一设备接收所述第二设备根据A个RDMA报文的PSN顺序依次发送的所述A个RDMA报文之后，所述方法还包括：
    当所述B小于所述A时，所述第一设备将所述第一设备的位图表中接收状态为接收失败的比特位对应的PSN确定为重传PSN；
    所述第一设备向所述第二设备发送重传指示报文，所述重传指示报文中携带有所述重传PSN；
    所述第一设备接收所述第二设备根据所述重传指示报文发送的重传报文，所述重传报文为所述重传PSN对应的接收失败的RDMA报文。
  7. 根据权利要求6所述的方法，其特征在于，所述重传PSN是通过所述RDMA协议定义的否定确认报文中的ACK扩展传输报文头字段指示的。
  8. 根据权利要求4至7中任一项所述的方法，其特征在于，所述数据读取请求中携带有第二RDMA扩展传输报文头字段，所述第二RDMA扩展传输报文头字段用于指示所述第二数据写入地址。
  9. 一种网络设备，其特征在于，所述网络设备为第一设备，包括：
    封装模块，用于根据远程直接存储器存取RDMA协议，将第一目标数据封装于N个RDMA报文中，所述第一目标数据为所述第一设备需要写入第二设备中进行存储的数据，所述N个RDMA报文中的任一个报文均携带有一个报文序列号PSN，所述N为大于或等于2的正整数；
    发送模块，用于根据所述N个RDMA报文的PSN顺序依次向所述第二设备发送所述N个RDMA报文，所述N个RDMA报文中每一个报文中均携带有第一数据写入地址，所述第一数据写入地址为将所述N个RDMA报文中每一个报文中的数据写入所述第二设备的地址，以使得所述第二设备从每一个RDMA报文中直接获取每一个RDMA报文中的第一数据写入地址，并将所述目标数据写入所述第一数据写入地址对应的存储空间中。
  10. 根据权利要求9所述的设备，其特征在于，所述网络设备还包括：
    接收模块，用于若所述N个RDMA报文未被所述第二设备完全接收成功，则接收所述第二设备发送的重传指示报文，所述重传指示报文中携带有重传PSN，所述重传PSN为所述第二设备的位图表中接收状态为接收失败的比特位对应的PSN，所述N个RDMA报文中每一个RDMA报文的PSN对应所述位图表中的一个比特位；
    所述发送模块，还用于：向所述第二设备发送重传报文，所述重传报文为所述重传PSN对应的接收失败的RDMA报文，以使得所述第二设备将所述重传报文中的数据存储于所述第二设备中。
  11. 根据权利要求9或10所述的设备，其特征在于，所述N个RDMA报文对应的第二个RDMA报文至第N个RDMA报文中任一个报文中均携带有RDMA扩展传输报文头，所述RDMA扩展传输报文头用于指示所述第一数据写入地址。
  12. 一种网络设备，其特征在于，所述网络设备为第一设备，包括：
    发送模块，用于向第二设备发送数据读取请求，所述数据读取请求根据所述远程直接存储器存取RDMA协议生成，并且携带有数据读取地址和第二数据写入地址，所述数据读取地址为所述第二设备中存储第二目标数据的目的地址，所述第二数据写入地址为所述第一设备上预留的用于存储从所述第二设备中读取到的所述第二目标数据的目的地址，所述第二目标数据为所述第一设备需要从所述第二设备中读取的数据；
    接收模块，用于接收所述第二设备根据A个RDMA报文的PSN顺序依次发送的所述A个RDMA报文，所述A个RDMA报文由所述第二设备将从所述数据读取地址中读取到的所述第二目标数据进行封装得到，所述A个RDMA报文中每一个报文均携带有一个报文序列号PSN，所述A为大于或等于2的正整数，所述A个RDMA报文中每一个报文均包括所述每一个报文的数据对应的所述第二数据写入地址；
    获取模块，用于从接收成功的B个RDMA报文中每一个RDMA报文中直接获取每一个RDMA报文的所述第二数据写入地址，所述B为小于或等于A的正整数；
    数据写入模块，用于将所述B个RDMA报文中每一个RDMA报文的数据写入所述每一个RDMA报文对应的所述第二数据写入地址的存储空间中。
  13. 根据权利要求12所述的设备，其特征在于，所述网络设备还包括：
    确定模块，用于根据所述A个RDMA报文的接收情况确定所述第一设备的位图表中每一个比特位的接收状态，所述接收状态为接收成功或接收失败，所述A个RDMA报文中每一个RDMA报文的PSN对应所述第一设备的位图表中的一个比特位。
  14. 根据权利要求12或13所述的设备，其特征在于，所述确定模块还用于：
    当所述B小于所述A时，将所述第一设备的位图表中接收状态为接收失败的比特位对应的PSN确定为重传PSN；
    所述发送模块，还用于：向所述第二设备发送重传指示报文，所述重传指示报文中携 带有所述重传PSN的信息；
    所述接收模块，还用于接收所述第二设备根据所述重传指示报文发送的重传报文，所述重传报文为所述重传PSN对应的接收失败的RDMA报文。
  15. 根据权利要求14所述的设备，其特征在于，所述重传PSN是通过所述RDMA协议定义的否定确认报文中的ACK扩展传输报文头字段指示的。
  16. 根据权利要求12至15中任一项所述的设备，其特征在于，所述数据读取请求中携带有第二RDMA扩展传输报文头字段，所述第二RDMA扩展传输报文头字段用于指示所述第二数据写入地址。
PCT/CN2018/124384 2017-12-27 2018-12-27 一种数据传输方法及第一设备 WO2019129147A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201880083147.8A CN111512603B (zh) 2017-12-27 2018-12-27 一种数据传输方法及第一设备
EP18897120.4A EP3716546A4 (en) 2017-12-27 2018-12-27 DATA TRANSMISSION PROCESS AND FIRST DEVICE
US16/909,414 US11412078B2 (en) 2017-12-27 2020-06-23 Data transmission method and first device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711448880.9 2017-12-27
CN201711448880.9A CN109981480A (zh) 2017-12-27 2017-12-27 一种数据传输方法及第一设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/909,414 Continuation US11412078B2 (en) 2017-12-27 2020-06-23 Data transmission method and first device

Publications (1)

Publication Number Publication Date
WO2019129147A1 true WO2019129147A1 (zh) 2019-07-04

Family

ID=67066598

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/124384 WO2019129147A1 (zh) 2017-12-27 2018-12-27 一种数据传输方法及第一设备

Country Status (4)

Country Link
US (1) US11412078B2 (zh)
EP (1) EP3716546A4 (zh)
CN (2) CN109981480A (zh)
WO (1) WO2019129147A1 (zh)

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11379404B2 (en) * 2018-12-18 2022-07-05 Sap Se Remote memory management
US10785306B1 (en) 2019-07-11 2020-09-22 Alibaba Group Holding Limited Data transmission and network interface controller
CN110460412B (zh) * 2019-07-11 2021-09-07 创新先进技术有限公司 用于数据传输的方法和rdma网卡
CN112448900B (zh) * 2019-09-02 2023-05-19 华为技术有限公司 一种数据传输方法及装置
WO2021203227A1 (zh) * 2020-04-07 2021-10-14 华为技术有限公司 基于内部集成电路协议的数据传输方法和传输装置
CN114090481A (zh) * 2020-07-02 2022-02-25 北京瀚海云星科技有限公司 一种发送数据的方法、接收数据的方法及相关装置
CN113886294A (zh) * 2020-07-02 2022-01-04 北京瀚海云星科技有限公司 一种基于rdma的低延时数据传输方法及相关装置
CN113965626A (zh) * 2020-07-02 2022-01-21 北京瀚海云星科技有限公司 一种发送标签数据的方法、接收标签数据的方法及相关装置
CN113965511A (zh) * 2020-07-02 2022-01-21 北京瀚海云星科技有限公司 一种基于rdma的标签数据传输方法及相关装置、***
CN112055058A (zh) * 2020-08-19 2020-12-08 广东省新一代通信与网络创新研究院 数据的存储方法、装置及计算机可读存储介质
CN114520711B (zh) * 2020-11-19 2024-05-03 迈络思科技有限公司 数据包的选择性重传
CN113422793B (zh) * 2021-02-05 2024-06-21 阿里巴巴集团控股有限公司 数据传输方法、装置、电子设备及计算机存储介质
CN113422792B (zh) * 2021-02-05 2024-06-25 阿里巴巴集团控股有限公司 数据传输方法、装置、电子设备及计算机存储介质
CN112954068B (zh) * 2021-03-09 2022-09-27 西安奥卡云数据科技有限公司 一种基于rdma的数据传输方法及装置
CN112948318B (zh) * 2021-03-09 2022-12-06 西安奥卡云数据科技有限公司 一种Linux操作***下基于RDMA的数据传输方法及装置
US11546238B1 (en) * 2021-06-14 2023-01-03 Netapp, Inc. Methods and systems for resending missing network data packets
CN113395359B (zh) * 2021-08-17 2021-10-29 苏州浪潮智能科技有限公司 基于远程直接内存访问的文件币集群数据传输方法、***
CN113784197B (zh) * 2021-08-23 2023-10-03 浙江大华技术股份有限公司 一种显示视频的方法、缓存视频帧的方法及装置
CN114090483A (zh) * 2021-09-30 2022-02-25 上海浦东发展银行股份有限公司 一种基于协程的rdma通信方法、装置及存储介质
CN115633104B (zh) * 2022-09-13 2024-02-13 江苏为是科技有限公司 数据发送方法、数据接收方法、装置及数据收发***
CN115834002B (zh) * 2022-11-16 2023-10-31 江苏为是科技有限公司 高速传输***及方法
CN116150078B (zh) * 2023-04-19 2023-07-07 湖南恩智绿源电子技术有限公司 板间数据通信传输方法、电子设备和计算机可读存储介质
CN116566921B (zh) * 2023-07-04 2024-03-22 珠海星云智联科技有限公司 远程直接内存访问读取的拥塞控制方法、***及存储介质
CN116582492B (zh) * 2023-07-14 2023-09-26 珠海星云智联科技有限公司 优化rdma读取的拥塞控制方法、***及存储介质
CN116909624B (zh) * 2023-09-12 2024-01-26 苏州浪潮智能科技有限公司 位图数据处理方法、随机存储器、电子设备和存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104484295A (zh) * 2014-12-16 2015-04-01 中国人民解放军国防科学技术大学 并行计算机***中基于接收方滑动窗口的数据传输方法
US20170324814A1 (en) * 2016-05-03 2017-11-09 Excelero Storage Ltd. System and method for providing data redundancy for remote direct memory access storage devices

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7817634B2 (en) * 2006-06-30 2010-10-19 Intel Corporation Network with a constrained usage model supporting remote direct memory access
US10223326B2 (en) * 2013-07-31 2019-03-05 Oracle International Corporation Direct access persistent memory shared storage
CN103546258B (zh) 2013-10-29 2017-01-18 华为技术有限公司 一种数据传输方法及装置
US10110518B2 (en) * 2013-12-18 2018-10-23 Mellanox Technologies, Ltd. Handling transport layer operations received out of order
CN105472023B (zh) * 2014-12-31 2018-11-20 华为技术有限公司 一种远程直接存储器存取的方法及装置
CN105577801B (zh) * 2014-12-31 2019-01-11 华为技术有限公司 一种业务加速方法及装置
US20160212214A1 (en) * 2015-01-16 2016-07-21 Avago Technologies General Ip (Singapore) Pte. Ltd. Tunneled remote direct memory access (rdma) communication
US10375167B2 (en) * 2015-11-20 2019-08-06 Microsoft Technology Licensing, Llc Low latency RDMA-based distributed storage
CN115941616A (zh) * 2017-12-15 2023-04-07 微软技术许可有限责任公司 多路径rdma传输

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104484295A (zh) * 2014-12-16 2015-04-01 中国人民解放军国防科学技术大学 并行计算机***中基于接收方滑动窗口的数据传输方法
US20170324814A1 (en) * 2016-05-03 2017-11-09 Excelero Storage Ltd. System and method for providing data redundancy for remote direct memory access storage devices

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3716546A4

Also Published As

Publication number Publication date
CN111512603A (zh) 2020-08-07
US20200322466A1 (en) 2020-10-08
CN111512603B (zh) 2022-11-11
US11412078B2 (en) 2022-08-09
EP3716546A4 (en) 2020-11-18
CN109981480A (zh) 2019-07-05
EP3716546A1 (en) 2020-09-30

Similar Documents

Publication Publication Date Title
WO2019129147A1 (zh) 一种数据传输方法及第一设备
US11153041B2 (en) Packet transmission method and user equipment
US11792114B2 (en) System and method for facilitating efficient management of non-idempotent operations in a network interface controller (NIC)
US20220014312A1 (en) Data transmission method and apparatus
US11799790B2 (en) Data transmission method, computing device, network device, and data transmission system
CN109327288B (zh) 数据传输加速方法、装置及***
US10868767B2 (en) Data transmission method and apparatus in optoelectronic hybrid network
US10430374B2 (en) Selective acknowledgement of RDMA packets
CN111786748B (zh) 数据重传方法和***、网卡、装置、服务器和存储介质
WO2018205688A1 (zh) 一种数据传输的方法、装置和***
US11381514B2 (en) Methods and apparatus for early delivery of data link layer packets
WO2022205248A1 (zh) 一种基于时间敏感传输协议的通信方法和相关装置
US10701189B2 (en) Data transmission method and apparatus
US10932159B2 (en) Data transmission method, data receiving device, and data sending device
KR20190011898A (ko) 무선 통신 시스템에서 단말, 기지국 및 이의 통신 방법
CN113973091A (zh) 一种报文处理方法、网络设备以及相关设备
WO2019015487A1 (zh) 一种数据重传处理方法、rlc实体和mac实体
WO2022056791A1 (zh) 一种报文重传方法和装置
CN117176809A (zh) 一种数据交互方法及***
CN115632748A (zh) 数据处理方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18897120

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018897120

Country of ref document: EP

Effective date: 20200625