WO2021036189A1 - Rdma数据发送及接收方法、电子设备及可读存储介质 - Google Patents
Rdma数据发送及接收方法、电子设备及可读存储介质 Download PDFInfo
- Publication number
- WO2021036189A1 WO2021036189A1 PCT/CN2020/072161 CN2020072161W WO2021036189A1 WO 2021036189 A1 WO2021036189 A1 WO 2021036189A1 CN 2020072161 W CN2020072161 W CN 2020072161W WO 2021036189 A1 WO2021036189 A1 WO 2021036189A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- original data
- rdma
- receiving end
- rdma device
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/163—Interprocessor communication
- G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
- G06F15/17306—Intercommunication techniques
- G06F15/17331—Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
Definitions
- the embodiments of this specification relate to the field of data processing technology, and in particular to a method for sending and receiving RDMA data, an electronic device, and a readable storage medium.
- RDMA technology can solve the delay of server-side data processing in network transmission, RDMA technology is receiving more and more attention in modern enterprise-level data centers and is used by more and more applications.
- RDMA is a high-throughput, low-latency network communication, especially suitable for use in large-scale parallel computer clusters.
- efficient use of network bandwidth has become a key part of its extreme performance. Therefore, a new method is urgently needed to improve the efficiency of RDMA network transmission.
- the embodiments of this specification provide a data sending and receiving method, device, electronic equipment, and readable storage medium, which can effectively improve the transmission efficiency of RDMA.
- the first aspect of the embodiments of this specification provides a method for sending RDMA data, which is applied to a sending end, and the sending end is provided with a sending end RDMA device, including:
- the sending end RDMA device obtains the original data to be sent
- the RDMA device at the transmitting end compresses the original data by using a preset compression method to obtain compressed original data
- the RDMA device at the transmitting end encapsulates the compressed original data into a data packet, and transmits the data packet to the receiving end, wherein the data packet carries a method flag corresponding to the preset compression method.
- the second aspect of the embodiments of this specification provides a method for receiving RDMA data, which is applied to a receiving end, and the receiving end is provided with a receiving end RDMA device, including:
- the RDMA device at the receiving end receives a data packet sent by the sending end, wherein the data packet includes compressed original data and a method flag corresponding to a preset compression method, and the data packet is the sending end using the preset Compression method is obtained by compressing the original data and then encapsulating;
- the receiving end RDMA device determines the preset compression method according to the method flag
- the receiving end RDMA device determines a decompression algorithm according to the preset compression method
- the RDMA device at the receiving end uses the decompression algorithm to decompress the compressed original data to obtain the original data.
- the third aspect of the embodiments of the present specification also provides a sending end, and the sending end is provided with a sending end RDMA device, including:
- An original data obtaining unit configured to obtain the original data to be sent through the RDMA device at the transmitting end
- a compression unit configured to control the RDMA device at the sending end to compress the original data using a preset compression method to obtain compressed original data
- An encapsulation unit configured to encapsulate the compressed original data into a data packet through the RDMA device, wherein the data packet carries a method flag corresponding to the preset compression method
- the data sending unit is used to transmit the data packet to the receiving end through the RDMA device.
- the fourth aspect of the embodiments of the present specification also provides a receiving end, and the receiving end RDMA device is provided in the receiving end, including:
- the data receiving unit is configured to receive the data packet sent by the sending end through the RDMA device at the receiving end, wherein the data packet includes compressed original data and a method flag corresponding to a preset compression method, and the data packet is the Obtained by the sending end after compressing the original data by using the preset compression method;
- a compression method acquisition unit configured to determine the preset compression method through the receiving end RDMA device according to the method flag;
- a decompression method acquiring unit configured to determine a decompression algorithm according to the preset compression method through the receiving end RDMA device;
- the decompression unit is configured to decompress the compressed original data by the receiving end RDMA device using the decompression algorithm to obtain the original data.
- the fifth aspect of the embodiments of this specification also provides an RDMA data transmission system, including:
- the sending end is used to obtain the original data to be sent through the sending end RDMA device; the sending end RDMA device uses a preset compression method to compress the original data to obtain compressed original data; and the sending end RDMA device compresses the original data.
- the device encapsulates the compressed original data into a data packet, and transmits the data packet to the receiving end, wherein the data packet carries a method flag corresponding to the preset compression method;
- the receiving end is configured to receive the data packet through the RDMA device at the receiving end, mark the preset compression method by the RDMA device at the receiving end according to the method, and determine the preset compression method by the RDMA device at the receiving end according to the preset Set the compression method and determine the decompression algorithm; the receiving end RDMA device uses the decompression algorithm to decompress the compressed original data to obtain the original data.
- the sixth aspect of the embodiments of this specification also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor.
- the processor implements the aforementioned data receiving and The steps of the sending method.
- the fifth aspect of the embodiments of the present specification also provides a computer-readable storage medium on which a computer program is stored.
- the program is executed by a processor, the steps of the above-mentioned data receiving and sending method are provided.
- the original data is compressed by the RDMA device, and the compressed original data is encapsulated into a data packet and then transmitted to the receiving end. Because in the process of transmitting the data packet to the receiving end, the RDMA device compresses the original data and transmits it Compared with the uncompressed original data, the compressed original data has a small amount of data, which makes the amount of data that needs to be transmitted becomes smaller. When the amount of data transmitted per unit time remains unchanged, the amount of data transmitted is As the amount of data becomes smaller, its transmission efficiency will inevitably increase, and the effect of improving the data transmission efficiency can be achieved.
- the receiving end can decompress the compressed original data according to the method flag in the received data packet, and then obtain the original data, which can ensure the reception The end can receive the original data completely.
- FIG. 1 is a system architecture diagram of the RDMA data transmission system in the embodiment of this specification
- Figure 2 is a structural diagram of a data packet in an embodiment of this specification
- FIG. 3 is a method flowchart of the RDMA data sending method in the embodiment of this specification.
- FIG. 5 is a schematic diagram of the structure of the dispensing terminal in the embodiment of the specification.
- FIG. 6 is a schematic diagram of the structure of the receiving end in the embodiment of the specification.
- FIG. 7 is a schematic diagram of the structure of the electronic device in the embodiment of the specification.
- Remote Direct Memory Access is generated in order to solve the delay of server-side data processing in network transmission; RDMA transfers data directly to the storage area of the computer through the network. Data is quickly moved from a system to a remote system memory without any impact on the operating system, so that much computer processing functions are not needed. It eliminates the overhead of external memory copy and context switching, thus freeing memory bandwidth and CPU cycles for improving application system performance.
- an embodiment of this specification provides an RDMA data transmission system, including a sending end 100 and a receiving end 200, where:
- the sending end 100 is used for the sending end RDMA device 101 to obtain the original data to be sent; the sending end RDMA device 101 uses a preset compression method to compress the original data to obtain the compressed original data; the sending end RDMA device 101 converts the original data
- the compressed original data is encapsulated into a data packet, and the data packet is transmitted to the receiving end 200, wherein the data packet carries a method flag corresponding to the preset compression method;
- the receiving end 200 is used for the receiving end RDMA device 201 to receive the data packet; the receiving end RDMA device 201 determines the preset compression method according to the method mark, and the receiving end RDMA device 201 determines the decompression method according to the preset compression method. Compression algorithm; the RDMA device 201 at the receiving end uses the decompression algorithm to decompress the compressed original data to obtain the original data.
- the sending end 100 and the receiving end 200 may be electronic devices such as notebook computers, desktop computers, smart phones, smart watches, all-in-one computers, and tablet computers; further, the sending end RDMA device 101 is set in the sending end 100, The receiving end RDMA device 201 is set in the receiving end 200, wherein the transmitting end RDMA device 101 and the receiving end RDMA device 201 can both be RDMA network cards and RDMA chips.
- the sending end RDMA device 101 compresses the original data at the hardware level and then transmits it.
- the receiving end RDMA device 201 also decompresses the compressed original data at the hardware level and then stores it directly to the receiving end. 200 storage devices.
- the sending end 100 first obtains the original data from the storage hardware of the sending end 100.
- the original data may be, for example, a 20-byte text; after obtaining the original data, the original data may be The original data is transmitted to the sending end RDMA device 101, or the sending end RDMA device 101 directly reads the original data, so that the sending end RDMA device 101 obtains the original data; the sending end RDMA device 101 obtains the original data After that, the original data is compressed using the preset compression method to obtain the compressed original data; the sending-end RDMA device 101 then encapsulates the compressed original data to obtain the data packet, and then uses the Internet Or the local area network transmits the data packet to the receiving end 200.
- the RDMA device 101 at the transmitting end compresses the original data and encapsulates the compressed original data into data packets and transmits them to the receiving end 200, since the RDMA device 101 at the transmitting end compresses the original data.
- Transmission the compressed original data is compared with the uncompressed original data.
- the compressed original data has a small amount of data, which makes the amount of data that needs to be transmitted becomes smaller.
- the data is transmitted without the amount of data transmitted per unit time. As the amount of data becomes smaller, its transmission efficiency will inevitably increase, and the effect of improving data transmission efficiency can be achieved.
- the sending end 100 after the sending end 100 obtains the original data, it calls the RDMA application interface (Verbs API) to issue a data sending command to the sending end RDMA device 101, where the data sending command carries a data compression instruction;
- the RDMA device 101 at the transmitting end first obtains the data compression instruction from the data transmission command, and then obtains the preset compression method according to the data compression instruction;
- the compression method compresses the original data to obtain the compressed original data; then encapsulates the compressed original data to obtain the data packet; and then responds to the data sending command to transfer the compressed data via the Internet or a local area network.
- the data packet is transmitted to the receiving end 200.
- the sending-end RDMA device 101 will select a compression method from the set of supported compression methods as the preset compression method.
- the set includes the string table compression algorithm (Lemple-Ziv-Welch, abbreviation: LZW) compression algorithm, (Lempel-Ziv-Oberhumer, abbreviation: LZO) compression algorithm, (Lempel-Ziv-Markov chain-Algorithm, abbreviation: LZMA) compression algorithm , LZSS compression algorithm, LZR compression algorithm, LZB compression algorithm, LZH compression algorithm, LZC compression algorithm and LZT compression algorithm and other compression algorithms.
- LZW string table compression algorithm
- LZO Low-Ziv-Oberhumer
- LZMA LZiv-Markov chain-Algorithm
- the sender RDMA device 101 selects the LZR compression algorithm from the LZW compression algorithm, the LZO compression algorithm, the LZMA compression algorithm, the LZSS compression algorithm, the LZR compression algorithm, the LZB compression algorithm, the LZH compression algorithm, the LZC compression algorithm, and the LZT compression algorithm.
- the preset compression method is used, and then the original data is compressed using the LZR compression algorithm to obtain the compressed original data.
- the sending-end RDMA device 101 encapsulates the compressed original data into the data packet, wherein the method mark is encapsulated in the transmission header of the data packet, and the user in the data packet The compressed original data is encapsulated in the data field.
- a message is a data unit exchanged and transmitted in the network, that is, a data block to be sent by a station at a time. Contains the complete data information to be sent, the length is very inconsistent, the length is not limited and variable;
- IBTA InfiniBand Trade Association, abbreviation: IBTA
- IBA InfiniBand Trade Association
- IBA is the infinite broadband technology (InfiniBand) InfiniBand all standard setters, and supplements RoCE
- the standardization of v1 and v2 explains the characteristic behaviors that should be possessed during RDMA transmission.
- IBA InfiniBand Architecture, abbreviated as: IBA
- data packet Packet, which contains the address information of the sender and receiver, and the data packet will be transmitted along different paths in one or more networks.
- the RDMA data packet includes a routing header (Routing Header), a transport header (Transport Header), a user data field (Packet Payload) and a Cyclic Redundancy Check (CRC), where ,
- the routing header contains routing information from the sender to the receiver, the transmission header contains the fields transmitted by the IBA and meets the IBTA standard, and the protocol contains reserved data bits;
- the CRC is based on network data packets or computer A hash function that generates a short fixed-digit check code for data such as files. It is used to detect or verify possible errors after data transmission or storage. The principle of division and remainder is used for error detection.
- the method mark can be encapsulated in the reserved bits of the transmission message.
- the protocol can also be reformulated, and the method mark can be encapsulated by the scheme of using non-reserved bits.
- the RDMA device 201 at the receiving end stores the original data in the storage device of the receiving end 200, where the storage device may be a mechanical hard disk, a solid state hard disk, a memory, etc. Device; enables the receiving end RDMA device 201 to directly store the original data in the sending end 100 directly into the storage device without the assistance of a processor for processing, which can effectively reduce delay and improve data transmission efficiency.
- the sender RDMA device 101 uses the LZR compression algorithm to compress the 5-byte data into the character string 21, and the data packet 30 is represented by the routing The header 31, the transmission header 32, the user data field 33 and the CRC34 are composed.
- the sender RDMA device 101 encapsulates the character string 21 into the user field 33, encapsulates the routing information of the receiver 200 into the routing header 31, and corresponds to the LZR compression algorithm
- the method mark O101 is encapsulated in the transmission header 32, and the check code 101001 is filled in the CRC34; after the sending end RDMA device 101 encapsulates the data in the data packet 30, the data packet 30 encapsulating the above data is sent to the receiving end 200 .
- the receiving end 200 is used for the receiving end RDMA network card 201 to receive the data packet 30; the receiving end RDMA network card 201 parses the data packet 30 to obtain the method mark O101, the string 21 and the check code 101001.
- the receiving The end RDMA network card 201 checks the data packet 30 according to the check code 10001. If the check passes, indicating that the data in the data packet 30 has not been tampered with, the receiving end RDMA network card 201 searches the compression algorithm index table according to the method flag 0101 The compression algorithm corresponding to 0101 is the LZR compression algorithm.
- the receiving end RDMA network card 201 can determine the decompression algorithm according to the LZR compression algorithm; the receiving end RDMA network card 201 then uses the decompression algorithm to decompress the string 21 , And then obtain the character string 20, and store the character string 20 in the storage device of the receiving end 200.
- the sending end RDMA device 101 compresses the character string 20 into a character string 21, and then encapsulates the character string 21 and the method flag 0101 into a data packet 30 and transmits it to the receiving end. 200.
- the data packet 30 transmitted to the receiving end 200 carries the character string 21.
- the data of the character string 21 is less than that of the 5-byte data.
- the data volume of the character string 20 reduces the amount of data transmitted by the sender 100 to the receiver 200. When the amount of data transmitted per unit time remains the same, the amount of data transmitted becomes smaller, and its transmission efficiency will inevitably be improved. To achieve the effect of improving the efficiency of data transmission.
- the data packet may also carry a compression mark that the original data is compressed.
- the compression mark may be encapsulated in the transmission header of the data packet, so that the data
- the transmission header of the packet encapsulates the compression mark that the original data is compressed.
- the compression mark can be encapsulated in the reserved bits of the transmission message, or the non-reserved bits can be used according to the re-developed protocol.
- the program to encapsulate the compression mark is also carry a compression mark that the original data is compressed.
- the sending end RDMA device 101 encapsulates the character string 21 into the user field 33, encapsulates the routing information of the receiving end 200 into the routing header 31, and corresponds to the LZR compression algorithm
- the method mark O101 and the compression mark 1011 are encapsulated in the transmission header 32, and the check code 101001 is filled in the CRC34; after the sending end RDMA device 101 encapsulates the data in the data packet 30, it sends the data packet 30 that encapsulates the above data To the receiving end 200.
- the receiving end 200 is used for the receiving end RDMA network card 201 to receive the data packet 30; the receiving end RDMA network card 201 parses the data packet 30 to obtain the method flag O101, the compression flag 1011, the character string 21, and the check code 101001, At this time, the receiving end RDMA network card 201 verifies the data packet 30 according to the check code 10001.
- the receiving end RDMA network card 201 first determines whether there is a compression mark 1011 , Because the data packet is encapsulated with a compression mark 1011, it is determined that there is a compression mark 1011; then the method mark 0101 is obtained, and the compression algorithm corresponding to 0101 is found from the compression algorithm index table as the LZR compression algorithm. At this time, the receiving end RDMA The network card 201 determines the decompression algorithm according to the LZR compression algorithm; the receiving end RDMA network card 201 then uses the decompression algorithm to decompress the character string 21, and then obtains the character string 20, and stores the character string 20 in the receiving end 200 In the storage device.
- the sending end RDMA device 101 compresses the character string 20 into a character string 21, and then encapsulates the character string 21 and the method flag 0101 into a data packet 30 and transmits it to the receiving end. 200.
- the data packet 30 transmitted to the receiving end 200 carries the character string 21.
- the data of the character string 21 is less than that of the 5-byte data.
- the data volume of the character string 20 reduces the amount of data transmitted by the sender 100 to the receiver 200. When the amount of data transmitted per unit time remains the same, the amount of data transmitted becomes smaller, and its transmission efficiency will inevitably be improved. To achieve the effect of improving the efficiency of data transmission.
- the receiving end RDMA network card 201 receives the data packet 30, and can determine whether the transmitted data is compressed according to the compression mark 1011. If it is compressed, the method mark 0101 is found and then decompressed. Compression, if it is not compressed, directly obtain the uncompressed data and store it in the storage device of the receiving end 200; reduce the probability of decompressing the uncompressed data when the data packet 30 transmits uncompressed data, and improve the receiving end 200 The operating performance.
- an embodiment of this specification provides an RDMA data sending method, which is applied to a sending end, and the sending end is provided with a sending end RDMA device, as shown in FIG. 3, include:
- Step S302 The RDMA device at the transmitting end obtains the original data to be transmitted;
- Step S304 The RDMA device at the transmitting end uses a preset compression method to compress the original data to obtain compressed original data;
- Step S306 The RDMA device at the transmitting end encapsulates the compressed original data into a data packet, and transmits the data packet to the receiving end, wherein the data packet carries the data packet corresponding to the preset compression method Method tag.
- the RDMA device at the transmitting end encapsulating the compressed original data into a data packet includes:
- the sending end RDMA device receives the data compression instruction sent by the RDMA application interface
- the sending end RDMA device responds to the data compression instruction to obtain the preset compression method
- the RDMA device at the transmitting end uses the preset compression method to compress the original data to obtain the compressed original data.
- the encapsulation of the compressed original data into a data packet by the RDMA device at the transmitting end includes:
- the RDMA device at the transmitting end encapsulates the compressed original data into the data packet, wherein the method mark is encapsulated in the transmission header of the data packet, and the user data field in the data packet is encapsulated The compressed original data.
- a compression mark indicating that the original data is compressed is encapsulated in the transmission header of the data packet.
- an embodiment of this specification provides an RDMA data receiving method, which is applied to a receiving end, and the receiving end is provided with a receiving end RDMA device, as shown in FIG. 4, include:
- Step S402 The RDMA device at the receiving end receives the data packet sent by the transmitting end, where the data packet includes the compressed original data and the method flag corresponding to the preset compression method, and the data packet is used by the transmitting end. Obtained by compressing the original data by the preset compression method and then encapsulating;
- Step S404 The RDMA device at the receiving end determines the preset compression method according to the method flag;
- Step S406 The RDMA device at the receiving end determines a decompression algorithm according to the preset compression method
- Step S408 The RDMA device at the receiving end uses the decompression algorithm to decompress the compressed original data to obtain the original data.
- determining the preset compression method by the RDMA device at the receiving end according to the method flag includes:
- the RDMA device at the receiving end determines the preset compression method according to the compression mark and the method mark.
- the method further includes:
- the RDMA device at the receiving end stores the original data in a storage device at the receiving end.
- an embodiment of the present specification provides a transmitting end, and the transmitting end is provided with a transmitting end RDMA device, as shown in FIG. 5, including:
- the original data obtaining unit 501 is configured to obtain the original data to be sent through the RDMA device at the transmitting end;
- the compression unit 502 is configured to control the RDMA device at the transmitting end to compress the original data using a preset compression method to obtain compressed original data;
- the encapsulation unit 503 is configured to encapsulate the compressed original data into a data packet through the RDMA device, wherein the data packet carries a method flag corresponding to the preset compression method;
- the data sending unit 504 is configured to transmit the data packet to the receiving end through the RDMA device.
- the compression unit 502 is configured to obtain the data compression instruction sent by the RDMA application interface through the sending-end RDMA device; and obtain the data compression instruction sent by the sending-end RDMA device according to the data compression instruction.
- the preset compression method; the original data is compressed using the preset compression method by the RDMA device at the transmitting end to obtain the compressed original data.
- the encapsulation unit 503 is configured to encapsulate the compressed original data into the data packet through the RDMA device at the transmitting end, wherein the transmission header of the data packet is encapsulated with According to the method mark, the compressed original data is encapsulated in the user data field in the data packet.
- a compression mark indicating that the original data is compressed is encapsulated in the transmission header of the data packet.
- an embodiment of the present specification provides a receiving end, which is provided with a receiving end RDMA device, as shown in FIG. 6, including:
- the data receiving unit 601 is configured to receive a data packet sent by the sending end through the RDMA device at the receiving end, where the data packet includes compressed original data and a method flag corresponding to a preset compression method, and the data packet is Obtained by the sending end after compressing the original data using the preset compression method;
- the compression method obtaining unit 602 is configured to determine the preset compression method by the receiving end RDMA device according to the method flag;
- the decompression method acquiring unit 603 is configured to determine a decompression algorithm according to the preset compression method through the receiving end RDMA device;
- the decompression unit 604 is configured to decompress the compressed original data by the receiving end RDMA device using the decompression algorithm to obtain the original data.
- the compression method acquiring unit 602 if the transmission header of the data packet is encapsulated with a compression mark in which the original data is compressed, it is used to pass the receiving end RDMA device according to the compression mark and The method flag determines the preset compression method.
- the receiving end further includes:
- the storage unit is configured to store the original data in the storage device of the receiving end through the receiving end RDMA device after obtaining the original data.
- an embodiment of this specification also provides an electronic device, as shown in FIG. 7, including a memory 704, a processor 702, and storage on the memory 704
- a computer program that can be run on the processor 702, and when the processor 702 executes the program, the steps of any one of the aforementioned data receiving and sending methods are implemented.
- the bus architecture (represented by the bus 700), the bus 700 can include any number of interconnected buses and bridges, and the bus 700 will include one or N processors represented by the processor 702 and the memory 704 represented by The various circuits of the memory are linked together.
- the bus 700 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, etc., which are all known in the art, and therefore, no further description is provided herein.
- the bus interface 705 provides an interface between the bus 700 and the receiver 701 and transmitter 703.
- the receiver 701 and the transmitter 703 may be the same element, that is, a transceiver, which provides a unit for communicating with various other devices on the transmission medium.
- the processor 702 is responsible for managing the bus 700 and general processing, and the memory 704 may be used to store data used by the processor 702 when performing operations.
- the embodiment of this specification also provides a computer-readable storage medium on which a computer program is stored.
- the program is executed by a processor, the foregoing The steps of any method of the data receiving and sending methods are described.
- These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
- the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
- These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
- the instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
本说明书实施例公开了一种RDMA数据发送方法,应用于发送端中,所述发送端中设置有发送端RDMA设备,所述发送端RDMA设备获取待发送的原始数据;所述发送端RDMA设备使用预设压缩方法对所述原始数据进行压缩,得到压缩后的原始数据;所述发送端RDMA设备将所述压缩后的原始数据封装成数据包,并将所述数据包传输给接收端,其中,所述数据包中携带有所述预设压缩方法对应的方法标记;如此,通过所述发送端RDMA设备在硬件层面来压缩并传输原始数据。
Description
本说明书实施例涉及数据处理技术领域,尤其涉及一种RDMA数据发送及接收方法、电子设备及可读存储介质。
由于RDMA技术能够解决网络传输中服务器端数据处理的延迟,使得在现代企业级数据中心中RDMA技术正在受到越来越多的关注,同时被越来越多的应用所使用。RDMA是高吞吐、低延迟的网络通信,尤其适合在大规模并行计算机集群中使用。但是随着网络传输数据的***式增长,网络带宽的高效利用成为发挥其极限性能的关键部分,因而,亟需一种新的方法,来提升RDMA网络传输的效率。
发明内容
本说明书实施例提供了一种数据发送及接收方法、装置、电子设备及可读存储介质,能够有效提高RDMA的传输效率。
本说明书实施例第一方面提供了一种RDMA数据发送方法,应用于发送端中,所述发送端中设置有发送端RDMA设备,包括:
所述发送端RDMA设备获取待发送的原始数据;
所述发送端RDMA设备使用预设压缩方法对所述原始数据进行压缩,得到压缩后的原始数据;
所述发送端RDMA设备将所述压缩后的原始数据封装成数据包,并将所述数据包传输给接收端,其中,所述数据包中携带有所述预设压缩方法对应的方法标记。
本说明书实施例第二方面提供了一种RDMA数据接收方法,应用于接收端中,所述接收端中设置有接收端RDMA设备,包括:
所述接收端RDMA设备接收发送端发送的数据包,其中,所述数据包包括压缩后的原始数据和预设压缩方法对应的方法标记,所述数据包是所述发送端使用所述预设压缩方法压缩所述原始数据后再进行封装得到的;
所述接收端RDMA设备根据所述方法标记,确定所述预设压缩方法;
所述接收端RDMA设备根据所述预设压缩方法,确定解压缩算法;
所述接收端RDMA设备使用所述解压缩算法对所述压缩后的原始数据进行解压缩,得到所述原始数据。
本说明书实施例第三方面还提供了一种发送端,所述发送端中设置有发送端RDMA设备,包括:
原始数据获取单元,用于通过所述发送端RDMA设备获取待发送的原始数据;
压缩单元,用于控制所述发送端RDMA设备使用预设压缩方法对所述原始数据进行压缩,得到压缩后的原始数据;
封装单元,用于通过所述RDMA设备将所述压缩后的原始数据封装成数据包,其中,所述数据包中携带有所述预设压缩方法对应的方法标记;
数据发送单元,用于通过所述RDMA设备将所述数据包传输给接收端。
本说明书实施例第四方面还提供了一种接收端,所述接收端中设置有接收端RDMA设备,包括:
数据接收单元,用于通过所述接收端RDMA设备接收发送端发送的数据包,其中,所述数据包包括压缩后的原始数据和预设压缩方法对应的方法标记,所述数据包是所述发送端使用所述预设压缩方法压缩所述原始数据后再进行封装得到的;
压缩方法获取单元,用于通过所述接收端RDMA设备根据所述方法标记,确定所述预设压缩方法;
解压缩方法获取单元,用于通过所述接收端RDMA设备根据所述预设压缩方法,确定解压缩算法;
解压缩单元,用于通过所述接收端RDMA设备使用所述解压缩算法对所述压缩后的原始数据进行解压缩,得到所述原始数据。
本说明书实施例第五方面还提供了一种RDMA数据传输***,包括:
发送端,用于通过发送端RDMA设备获取待发送的原始数据;通过所述发送端RDMA设备使用预设压缩方法对所述原始数据进行压缩,得到压缩后的原始数据;通过所述发送端RDMA设备将所述压缩后的原始数据封装成数据包,并将所述数据包传输 给接收端,其中,所述数据包中携带有所述预设压缩方法对应的方法标记;
所述接收端,用于通过接收端RDMA设备接收所述数据包,通过所述接收端RDMA设备根据所述方法标记,确定所述预设压缩方法;通过所述接收端RDMA设备根据所述预设压缩方法,确定解压缩算法;通过所述接收端RDMA设备使用所述解压缩算法对所述压缩后的原始数据进行解压缩,得到所述原始数据。
本说明书实施例第六方面还提供了一种电子设备,包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序,所述处理器执行所述程序时实现上述数据接收及发送方法的步骤。
本说明书实施例第五方面还提供了一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时上述数据接收及发送方法的步骤。
本说明书实施例的有益效果如下:
基于上述技术方案,通过RDMA设备对原始数据进行压缩,并将压缩后的原始数据封装成数据包后传输给接收端,由于在数据包传输给接收端过程中,RDMA设备将原始数据压缩后传输,压缩后的原始数据与未压缩的原始数据相比,压缩后的原始数据的数据量小,使得需要传输数据的数据量变小,在单位时间内传输数据量不变的情况下而传输数据的数据量变小,其传输效率必然会提高,进而能够实现提高数据传输效率的效果。
而且,由于传输的数据包携带有预设压缩方法对应的方法标记,使得接收端能够根据接收到数据包中的方法标记对压缩后的原始数据进行解压缩,进而得到原始数据,即能够确保接收端能够完整的接收到原始数据。
图1为本说明书实施例中RDMA数据传输***的***架构图;
图2为本说明书实施例中数据包的结构图;
图3为本说明书实施例中RDMA数据发送方法的方法流程图;
图4为本说明书实施例中RDMA数据接收方法的方法流程图;
图5为本说明书实施例中发放端的结构示意图;
图6为本说明书实施例中接收端的结构示意图;
图7为本说明书实施例中电子设备的结构示意图。
为了更好的理解上述技术方案,下面通过附图以及具体实施例对本说明书实施例的技术方案做详细的说明,应当理解本说明书实施例以及实施例中的具体特征是对本说明书实施例技术方案的详细的说明,而不是对本说明书技术方案的限定,在不冲突的情况下,本说明书实施例以及实施例中的技术特征可以相互组合。
本说明书实施例中,远程直接数据存取(Remote Direct Memory Access,简称RDMA),为了解决网络传输中服务器端数据处理的延迟而产生的;RDMA通过网络把数据直接传入计算机的存储区,将数据从一个***快速移动到远程***存储器中,而不对操作***造成任何影响,这样就不需要用到多少计算机的处理功能。它消除了外部存储器复制和上下文切换的开销,因而能解放内存带宽和CPU周期用于改进应用***性能。
第一方面,如图1所示,本说明书实施例提供一种RDMA数据传输***,包括发送端100和接收端200,其中,
发送端100,用于发送端RDMA设备101获取待发送的原始数据;发送端RDMA设备101使用预设压缩方法对所述原始数据进行压缩,得到压缩后的原始数据;发送端RDMA设备101将所述压缩后的原始数据封装成数据包,并将所述数据包传输给接收端200,其中,所述数据包中携带有所述预设压缩方法对应的方法标记;
接收端200,用于接收端RDMA设备201接收所述数据包;接收端RDMA设备201根据所述方法标记,确定所述预设压缩方法接收端RDMA设备201根据所述预设压缩方法,确定解压缩算法;接收端RDMA设备201使用所述解压缩算法对所述压缩后的原始数据进行解压缩,得到所述原始数据。
本说明书实施例中,发送端100和接收端200可以是笔记本电脑、台式电脑、智能手机、智能手表、一体机和平板电脑等电子设备;进一步,发送端RDMA设备101设置在发送端100中,接收端RDMA设备201设置在接收端200中,其中,发送端RDMA设备101和接收端RDMA设备201均可以为RDMA网卡和RDMA芯片等。
本说明书实施例中,发送端RDMA设备101是在硬件层面对原始数据进行压缩后传输,相应地,接收端RDMA设备201也是在硬件层面对压缩后的原始数据进行解压 缩后直接存储到接收端200的存储设备中。
本说明书实施例中,发送端100首先从发送端100的存储硬件中获取所述原始数据,所述原始数据例如可以是20字节的文字;在获取到所述原始数据之后,可以将所述原始数据传输给发送端RDMA设备101,或者,发送端RDMA设备101直接读取所述原始数据,从而使得发送端RDMA设备101获取到所述原始数据;发送端RDMA设备101获取到所述原始数据之后,使用所述预设压缩方法对所述原始数据进行压缩,得到所述压缩后的原始数据;发送端RDMA设备101再封装所述压缩后的原始数据,得到所述数据包,然后通过互联网或局域网将所述数据包传输给接收端200。
在此过程中,由于是发送端RDMA设备101对所述原始数据进行压缩,并将压缩后的原始数据封装成数据包后传输给接收端200,由于发送端RDMA设备101是将原始数据压缩后传输,压缩后的原始数据与未压缩的原始数据相比,压缩后的原始数据的数据量小,使得需要传输数据的数据量变小,在单位时间内传输数据量不变的情况下而传输数据的数据量变小,其传输效率必然会提高,进而能够实现提高数据传输效率的效果。
本说明书实施例中,发送端100获取所述原始数据之后,调用RDMA应用接口(Verbs API)向发送端RDMA设备101下发数据发送命令,其中,所述数据发放命令中携带有数据压缩指令;发送端RDMA设备101接收到所述数据发送命令之后,首先从所述数据发送命令中获取所述数据压缩指令,再根据所述数据压缩指令,获取所述预设压缩方法;使用所述预设压缩方法对所述原始数据进行压缩,得到所述压缩后的原始数据;再封装所述压缩后的原始数据,得到所述数据包;然后响应所述数据发送命令,通过互联网或局域网将所述数据包传输给接收端200。
本说明书实施例中,发送端RDMA设备101接收到所述数据发送命令之后,发送端RDMA设备101会从支持的压缩方法集合中选择一种压缩方法作为所述预设压缩方法,所述压缩方法集合包括串表压缩算法(Lemple-Ziv-Welch,简称:LZW)压缩算法、(Lempel-Ziv-Oberhumer,简称:LZO)压缩算法、(Lempel-Ziv-Markov chain-Algorithm,简称:LZMA)压缩算法、LZSS压缩算法、LZR压缩算法、LZB压缩算法、LZH压缩算法、LZC压缩算法和LZT压缩算法等压缩算法中一种或多种。
例如,发送端RDMA设备101从LZW压缩算法、LZO压缩算法、LZMA压缩算法、LZSS压缩算法、LZR压缩算法、LZB压缩算法、LZH压缩算法、LZC压缩算法和LZT压缩算法中选取LZR压缩算法作为所述预设压缩方法,然后使用LZR压缩算法压缩所述原始数据,得到所述压缩后的原始数据。
本说明书实施例中,发送端RDMA设备101将所述压缩后的原始数据封装成所述数据包,其中,所述数据包的传输报头中封装有所述方法标记,所述数据包中的用户数据字段中封装有所述压缩后的原始数据。
本说明书实施例中,报文是网络中交换与传输的数据单元,即站点一次性要发送的数据块。包含了将要发送的完整的数据信息,其长短很不一致,长度不限且可变;IBTA(InfiniBand Trade Association,简称:IBTA)是无限宽带技术(InfiniBand)InfiniBand的全部标准制定者,并补充了RoCE v1 v2的标准化,解释了RDMA传输过程中应具备的特性行为。IBA:(InfiniBand Architecture,简称:IBA)表示infiniband架构;数据包:包(Packet),它包含发送者和接收者的地址信息,数据包会沿着不同的路径在一个或多个网络中传输。
本说明书实施例中,在RDMA的数据包包含路由报头(Routing Header)、传输报头(Transport Header),用户数据字段(Packet Payload)和循环冗余校验(Cyclic Redundancy Check,简称:CRC),其中,所述路由报头包含发送者到接收者的路由信息,所述传输报头包含IBA传输的字段,满足IBTA的标准,其中协议中包含了保留的数据位;所述CRC是根据网络数据包或电脑文件等数据产生简短固定位数校验码的一种散列函数,用来检测或校验数据传输或者保存后可能出现的错误,利用除法及余数的原理来作错误侦测的。
本说明书实施例中,由于传输报文中当前IBTA协议中预留了6位字节,可根据压缩需要选择其中1位或多位用于表示数据压缩的方法,此时可以最多支持64种压缩算法,因此,可以将所述方法标记封装在传输报文的预留位中。当然,本说明书实施例还可以重新制定协议,采用使用非预留位的方案来封装所述方法标记。
本说明书实施例中,接收端RDMA设备201在得到所述原始数据之后,将所述原始数据存储到接收端200的存储设备中,其中,所述存储设备可以是机械硬盘、固态硬盘和内存等设备;使得接收端RDMA设备201直接将发送端100中的原始数据直接存储到所述存储设备中,不需通过处理器协助处理,能够有效降低延迟,提高数据传输效率。
例如,如图2所示,若原始数据为20字节的数据用字符串20表示,发送端RDMA设备101使用LZR压缩算法压缩成5字节的数据用字符串21表示,数据包30由路由报头31、传输报头32,用户数据字段33和CRC34组成,发送端RDMA设备101将字符串21封装到用户字段33中,将接收端200的路由信息封装到路由报头31中,将LZR 压缩算法对应的方法标记O101封装到传输报头32中,以及在CRC34中填入校验码101001;发送端RDMA设备101在数据包30中封装完数据之后,将封装上述数据的数据包30发送给接收端200。
此时,接收端200,用于接收端RDMA网卡201接收到数据包30;接收端RDMA网卡201对数据包30进行解析,得到方法标记O101、字符串21和校验码101001,此时,接收端RDMA网卡201根据校验码10001对数据包30进行校验,若校验通过,表征数据包30中的数据未被篡改,则接收端RDMA网卡201根据方法标记0101从压缩算法索引表中查找到0101对应的压缩算法为LZR压缩算法,此时,可以接收端RDMA网卡201根据LZR压缩算法,确定出解压缩算法;接收端RDMA网卡201再使用所述解压缩算法对字符串21进行解压缩,进而得到字符串20,并将字符串20存储到接收端200的存储设备中。
由上述可知,发送端100在发送字符串20过程中,发送端RDMA设备101将字符串20压缩成字符串21,然后将字符串21和方法标记0101封装到数据包30中后传输给接收端200,此时,传输给接收端200中的数据包30携带是字符串21,字符串21为5字节的数据与字符串20为20字节的数据相比,字符串21的数据量小于字符串20的数据量,使得发送端100传输给接收端200的数据量变小,在单位时间内传输数据量不变的情况下而传输数据的数据量变小,其传输效率必然会提高,进而能够实现提高数据传输效率的效果。
本说明书另一实施例,所述数据包中还可以携带有所述原始数据被压缩的压缩标记,此时,可以将所述压缩标记封装在所述数据包的传输报头中,使得所述数据包的传输报头中封装有所述原始数据被压缩的压缩标记,如此,可以将所述压缩标记封装在传输报文的预留位中,也可以根据重新制定的协议,采用使用非预留位的方案来封装所述压缩标记。
例如,继续参见图2,若压缩标记用1011表示,则发送端RDMA设备101将字符串21封装到用户字段33中,将接收端200的路由信息封装到路由报头31中,将LZR压缩算法对应的方法标记O101和压缩标记1011封装到传输报头32中,以及在CRC34中填入校验码101001;发送端RDMA设备101在数据包30中封装完数据之后,将封装上述数据的数据包30发送给接收端200。
此时,接收端200,用于接收端RDMA网卡201接收到数据包30;接收端RDMA网卡201对数据包30进行解析,得到方法标记O101和压缩标记1011、字符串21和校 验码101001,此时,接收端RDMA网卡201根据校验码10001对数据包30进行校验,若校验通过,表征数据包30中的数据未被篡改,则接收端RDMA网卡201首先确定是否存在压缩标记1011,由于数据包中封装有压缩标记1011,进而确定存在压缩标记1011;然后再获取方法标记0101,从压缩算法索引表中查找到0101对应的压缩算法为LZR压缩算法,此时,可以接收端RDMA网卡201根据LZR压缩算法,确定出解压缩算法;接收端RDMA网卡201再使用所述解压缩算法对字符串21进行解压缩,进而得到字符串20,并将字符串20存储到接收端200的存储设备中。
由上述可知,发送端100在发送字符串20过程中,发送端RDMA设备101将字符串20压缩成字符串21,然后将字符串21和方法标记0101封装到数据包30中后传输给接收端200,此时,传输给接收端200中的数据包30携带是字符串21,字符串21为5字节的数据与字符串20为20字节的数据相比,字符串21的数据量小于字符串20的数据量,使得发送端100传输给接收端200的数据量变小,在单位时间内传输数据量不变的情况下而传输数据的数据量变小,其传输效率必然会提高,进而能够实现提高数据传输效率的效果。
而且,由于数据包30中封装有压缩标记1011,使得接收端RDMA网卡201接收到数据包30,可以根据压缩标记1011来确定传输数据是否被压缩,若被压缩才查找到方法标记0101之后进行解压缩,若未被压缩,直接获取未被压缩的数据然后存储到接收端200的存储设备中;降低数据包30传输未压缩的数据时对未压缩的数据进行解压缩的概率,提高接收端200的运行性能。
第二方面,基于与第一方面的同一发明构思,本说明书实施例提供一种RDMA数据发送方法,应用于发送端中,所述发送端中设置有发送端RDMA设备,如图3所示,包括:
步骤S302、所述发送端RDMA设备获取待发送的原始数据;
步骤S304、所述发送端RDMA设备使用预设压缩方法对所述原始数据进行压缩,得到压缩后的原始数据;
步骤S306、所述发送端RDMA设备将所述压缩后的原始数据封装成数据包,并将所述数据包传输给接收端,其中,所述数据包中携带有所述预设压缩方法对应的方法标记。
在一种可选实施方式中,所述发送端RDMA设备将所述压缩后的原始数据封装成 数据包,包括:
所述发送端RDMA设备接收所述RDMA的应用接口发送的数据压缩指令;
所述发送端RDMA设备响应所述数据压缩指令,获取所述预设压缩方法;
所述发送端RDMA设备使用所述预设压缩方法对所述原始数据进行压缩,得到所述压缩后的原始数据。
在一种可选实施方式中,所述发送端RDMA设备将所述压缩后的原始数据封装成数据包,包括:
所述发送端RDMA设备将所述压缩后的原始数据封装成所述数据包,其中,所述数据包的传输报头中封装有所述方法标记,所述数据包中的用户数据字段中封装有所述压缩后的原始数据。
在一种可选实施方式中,所述数据包的传输报头中封装有所述原始数据被压缩的压缩标记。
第三方面,基于与第一方面的同一发明构思,本说明书实施例提供一种RDMA数据接收方法,应用于接收端中,所述接收端中设置有接收端RDMA设备,如图4所示,包括:
步骤S402、所述接收端RDMA设备接收发送端发送的数据包,其中,所述数据包包括压缩后的原始数据和预设压缩方法对应的方法标记,所述数据包是所述发送端使用所述预设压缩方法压缩所述原始数据后再进行封装得到的;
步骤S404、所述接收端RDMA设备根据所述方法标记,确定所述预设压缩方法;
步骤S406、所述接收端RDMA设备根据所述预设压缩方法,确定解压缩算法;
步骤S408、所述接收端RDMA设备使用所述解压缩算法对所述压缩后的原始数据进行解压缩,得到所述原始数据。
在一种可选实施方式中,所述接收端RDMA设备根据所述方法标记,确定所述预设压缩方法,包括:
若所述数据包的传输报头中封装有原始数据被压缩后的压缩标记,则所述接收端RDMA设备根据所述压缩标记和所述方法标记,确定所述预设压缩方法。
在一种可选实施方式中,在得到所述原始数据之后,所述方法还包括:
所述接收端RDMA设备将所述原始数据存储到所述接收端的存储设备中。
第四方面,基于与第二方面的同一发明构思,本说明书实施例提供一种发送端,所述发送端中设置有发送端RDMA设备,如图5所示,包括:
原始数据获取单元501,用于通过所述发送端RDMA设备获取待发送的原始数据;
压缩单元502,用于控制所述发送端RDMA设备使用预设压缩方法对所述原始数据进行压缩,得到压缩后的原始数据;
封装单元503,用于通过所述RDMA设备将所述压缩后的原始数据封装成数据包,其中,所述数据包中携带有所述预设压缩方法对应的方法标记;
数据发送单元504,用于通过所述RDMA设备将所述数据包传输给接收端。
在一种可选实施方式中,压缩单元502,用于通过所述发送端RDMA设备获取所述RDMA的应用接口发送的数据压缩指令;通过所述发送端RDMA设备根据所述数据压缩指令,获取所述预设压缩方法;通过所述发送端RDMA设备使用所述预设压缩方法对所述原始数据进行压缩,得到所述压缩后的原始数据。
在一种可选实施方式中,封装单元503,用于通过所述发送端RDMA设备将所述压缩后的原始数据封装成所述数据包,其中,所述数据包的传输报头中封装有所述方法标记,所述数据包中的用户数据字段中封装有所述压缩后的原始数据。
在一种可选实施方式中,所述数据包的传输报头中封装有所述原始数据被压缩的压缩标记。
第五方面,基于与第三方面的同一发明构思,本说明书实施例提供一种接收端,所述接收端中设置有接收端RDMA设备,如图6所示,包括:
数据接收单元601,用于通过所述接收端RDMA设备接收发送端发送的数据包,其中,所述数据包包括压缩后的原始数据和预设压缩方法对应的方法标记,所述数据包是所述发送端使用所述预设压缩方法压缩所述原始数据后再进行封装得到的;
压缩方法获取单元602,用于通过所述接收端RDMA设备根据所述方法标记,确定所述预设压缩方法;
解压缩方法获取单元603,用于通过所述接收端RDMA设备根据所述预设压缩方法,确定解压缩算法;
解压缩单元604,用于通过所述接收端RDMA设备使用所述解压缩算法对所述压 缩后的原始数据进行解压缩,得到所述原始数据。
在一种可选实施方式中,压缩方法获取单元602,若所述数据包的传输报头中封装有原始数据被压缩后的压缩标记,用于通过所述接收端RDMA设备根据所述压缩标记和所述方法标记,确定所述预设压缩方法。
在一种可选实施方式中,所述接收端还包括:
存储单元,用于在得到所述原始数据之后,通过所述接收端RDMA设备将所述原始数据存储到所述接收端的存储设备中。
第六方面,基于与前述实施例中数据接收及发送方法同样的发明构思,本说明书实施例还提供一种电子设备,如图7所示,包括存储器704、处理器702及存储在存储器704上并可在处理器702上运行的计算机程序,所述处理器702执行所述程序时实现前文所述数据接收及发送方法的任一方法的步骤。
其中,在图7中,总线架构(用总线700来代表),总线700可以包括任意数量的互联的总线和桥,总线700将包括由处理器702代表的一个或N个处理器和存储器704代表的存储器的各种电路链接在一起。总线700还可以将诸如***设备、稳压器和功率管理电路等之类的各种其他电路链接在一起,这些都是本领域所公知的,因此,本文不再对其进行进一步描述。总线接口705在总线700和接收器701和发送器703之间提供接口。接收器701和发送器703可以是同一个元件,即收发机,提供用于在传输介质上与各种其他装置通信的单元。处理器702负责管理总线700和通常的处理,而存储器704可以被用于存储处理器702在执行操作时所使用的数据。
第七方面,基于与前述实施例中数据接收及发送方法的发明构思,本说明书实施例还提供一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现前文所述数据接收及发送方法的任一方法的步骤。
本说明书是参照根据本说明书实施例的方法、设备(***)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的设备。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令设备的制造品,该指令设备实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本说明书的优选实施例,但本领域内的技术人员一旦得知了基本创造性概念,则可对这些实施例作出另外的变更和修改。所以,所附权利要求意欲解释为包括优选实施例以及落入本说明书范围的所有变更和修改。
显然,本领域的技术人员可以对本说明书进行各种改动和变型而不脱离本说明书的精神和范围。这样,倘若本说明书的这些修改和变型属于本说明书权利要求及其等同技术的范围之内,则本说明书也意图包含这些改动和变型在内。
Claims (17)
- 一种RDMA数据发送方法,应用于发送端中,所述发送端中设置有发送端RDMA设备,包括:所述发送端RDMA设备获取待发送的原始数据;所述发送端RDMA设备使用预设压缩方法对所述原始数据进行压缩,得到压缩后的原始数据;所述发送端RDMA设备将所述压缩后的原始数据封装成数据包,并将所述数据包传输给接收端,其中,所述数据包中携带有所述预设压缩方法对应的方法标记。
- 如权利要求1所述的发送方法,所述发送端RDMA设备将所述压缩后的原始数据封装成数据包,包括:所述发送端RDMA设备接收所述RDMA的应用接口发送的数据压缩指令;所述发送端RDMA设备响应所述数据压缩指令,获取所述预设压缩方法;所述发送端RDMA设备使用所述预设压缩方法对所述原始数据进行压缩,得到所述压缩后的原始数据。
- 如权利要求1所述的发送方法,所述发送端RDMA设备将所述压缩后的原始数据封装成数据包,包括:所述发送端RDMA设备将所述压缩后的原始数据封装成所述数据包,其中,所述数据包的传输报头中封装有所述方法标记,所述数据包中的用户数据字段中封装有所述压缩后的原始数据。
- 如权利要求3所述的发送方法,所述数据包的传输报头中封装有所述原始数据被压缩的压缩标记。
- 一种RDMA数据接收方法,应用于接收端中,所述接收端中设置有接收端RDMA设备,包括:所述接收端RDMA设备接收发送端发送的数据包,其中,所述数据包包括压缩后的原始数据和预设压缩方法对应的方法标记,所述数据包是所述发送端使用所述预设压缩方法压缩所述原始数据后再进行封装得到的;所述接收端RDMA设备根据所述方法标记,确定所述预设压缩方法;所述接收端RDMA设备根据所述预设压缩方法,确定解压缩算法;所述接收端RDMA设备使用所述解压缩算法对所述压缩后的原始数据进行解压缩,得到所述原始数据。
- 如权利要求5所述的接收方法,所述接收端RDMA设备根据所述方法标记,确 定所述预设压缩方法,包括:若所述数据包的传输报头中封装有原始数据被压缩后的压缩标记,则所述接收端RDMA设备根据所述压缩标记和所述方法标记,确定所述预设压缩方法。
- 如权利要求5或6所述的接收方法,在得到所述原始数据之后,所述方法还包括:所述接收端RDMA设备将所述原始数据存储到所述接收端的存储设备中。
- 一种发送端,所述发送端中设置有发送端RDMA设备,包括:原始数据获取单元,用于通过所述发送端RDMA设备获取待发送的原始数据;压缩单元,用于控制所述发送端RDMA设备使用预设压缩方法对所述原始数据进行压缩,得到压缩后的原始数据;封装单元,用于通过所述RDMA设备将所述压缩后的原始数据封装成数据包,其中,所述数据包中携带有所述预设压缩方法对应的方法标记;数据发送单元,用于通过所述RDMA设备将所述数据包传输给接收端。
- 如权利要求8所述的发送端,所述压缩单元,用于通过所述发送端RDMA设备获取所述RDMA的应用接口发送的数据压缩指令;通过所述发送端RDMA设备根据所述数据压缩指令,获取所述预设压缩方法;通过所述发送端RDMA设备使用所述预设压缩方法对所述原始数据进行压缩,得到所述压缩后的原始数据。
- 如权利要求8所述的发送端,所述封装单元,用于通过所述发送端RDMA设备将所述压缩后的原始数据封装成所述数据包,其中,所述数据包的传输报头中封装有所述方法标记,所述数据包中的用户数据字段中封装有所述压缩后的原始数据。
- 如权利要求10所述的发送端,所述数据包的传输报头中封装有所述原始数据被压缩的压缩标记。
- 一种接收端,所述接收端中设置有接收端RDMA设备,包括:数据接收单元,用于通过所述接收端RDMA设备接收发送端发送的数据包,其中,所述数据包包括压缩后的原始数据和预设压缩方法对应的方法标记,所述数据包是所述发送端使用所述预设压缩方法压缩所述原始数据后再进行封装得到的;压缩方法获取单元,用于通过所述接收端RDMA设备根据所述方法标记,确定所述预设压缩方法;解压缩方法获取单元,用于通过所述接收端RDMA设备根据所述预设压缩方法,确定解压缩算法;解压缩单元,用于通过所述接收端RDMA设备使用所述解压缩算法对所述压缩后 的原始数据进行解压缩,得到所述原始数据。
- 如权利要求12所述的接收端,所述压缩方法获取单元,若所述数据包的传输报头中封装有原始数据被压缩后的压缩标记,用于通过所述接收端RDMA设备根据所述压缩标记和所述方法标记,确定所述预设压缩方法。
- 如权利要求13所述的接收端,还包括:存储单元,用于在得到所述原始数据之后,通过所述接收端RDMA设备将所述原始数据存储到所述接收端的存储设备中。
- 一种RDMA数据传输***,包括:发送端,用于通过发送端RDMA设备获取待发送的原始数据;通过所述发送端RDMA设备使用预设压缩方法对所述原始数据进行压缩,得到压缩后的原始数据;通过所述发送端RDMA设备将所述压缩后的原始数据封装成数据包,并将所述数据包传输给接收端,其中,所述数据包中携带有所述预设压缩方法对应的方法标记;所述接收端,用于通过接收端RDMA设备接收所述数据包,通过所述接收端RDMA设备根据所述方法标记,确定所述预设压缩方法;通过所述接收端RDMA设备根据所述预设压缩方法,确定解压缩算法;通过所述接收端RDMA设备使用所述解压缩算法对所述压缩后的原始数据进行解压缩,得到所述原始数据。
- 一种电子设备,包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序,所述处理器执行所述程序时实现权利要求1-7任一项所述的方法。
- 一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现权利要求1-7任一项所述的方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/854,740 US10817460B2 (en) | 2019-08-28 | 2020-04-21 | RDMA data sending and receiving methods, electronic device, and readable storage medium |
US17/065,684 US11023412B2 (en) | 2019-08-28 | 2020-10-08 | RDMA data sending and receiving methods, electronic device, and readable storage medium |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910802307.6A CN110704361A (zh) | 2019-08-28 | 2019-08-28 | Rdma数据发送及接收方法、电子设备及可读存储介质 |
CN201910802307.6 | 2019-08-28 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/854,740 Continuation US10817460B2 (en) | 2019-08-28 | 2020-04-21 | RDMA data sending and receiving methods, electronic device, and readable storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021036189A1 true WO2021036189A1 (zh) | 2021-03-04 |
Family
ID=69193872
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/072161 WO2021036189A1 (zh) | 2019-08-28 | 2020-01-15 | Rdma数据发送及接收方法、电子设备及可读存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN110704361A (zh) |
WO (1) | WO2021036189A1 (zh) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210306166A1 (en) * | 2020-03-30 | 2021-09-30 | Mellanox Technologies Tlv Ltd. | Network information transmission systems |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105472023A (zh) * | 2014-12-31 | 2016-04-06 | 华为技术有限公司 | 一种远程直接存储器存取的方法及装置 |
US20180139281A1 (en) * | 2011-11-03 | 2018-05-17 | Netapp Inc. | Interconnect delivery process |
CN108616927A (zh) * | 2018-04-04 | 2018-10-02 | 北京锐安科技有限公司 | 一种数据发送和接收方法及装置 |
CN109547531A (zh) * | 2018-10-19 | 2019-03-29 | 华为技术有限公司 | 数据处理的方法、装置和计算设备 |
CN109587112A (zh) * | 2018-10-25 | 2019-04-05 | 华为技术有限公司 | 一种发送数据的方法、接收数据的方法、设备及*** |
CN110620762A (zh) * | 2019-08-06 | 2019-12-27 | 阿里巴巴集团控股有限公司 | 基于rdma的数据传输方法、网卡、服务器及介质 |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060274789A1 (en) * | 2005-06-07 | 2006-12-07 | Fong Pong | Apparatus and methods for a high performance hardware network protocol processing engine |
US7596144B2 (en) * | 2005-06-07 | 2009-09-29 | Broadcom Corp. | System-on-a-chip (SoC) device with integrated support for ethernet, TCP, iSCSI, RDMA, and network application acceleration |
CN102045772B (zh) * | 2009-10-21 | 2015-06-03 | 华为技术有限公司 | 一种数据传输方法及装置 |
CN102131234B (zh) * | 2010-01-18 | 2013-12-04 | 华为技术有限公司 | Ip数据包的压缩及解压缩方法和装置 |
CN101977402B (zh) * | 2010-10-19 | 2015-04-01 | 中兴通讯股份有限公司 | 鲁棒性头压缩中一种上下文重用的方法及装置 |
CN106033377B (zh) * | 2015-03-13 | 2019-06-28 | ***通信集团浙江有限公司 | 数据容灾方法及容灾服务器 |
US9936049B2 (en) * | 2015-07-29 | 2018-04-03 | Dell Products L.P. | Protocol independent way for dynamically selecting data compression methods for redirected USB devices |
-
2019
- 2019-08-28 CN CN201910802307.6A patent/CN110704361A/zh active Pending
-
2020
- 2020-01-15 WO PCT/CN2020/072161 patent/WO2021036189A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180139281A1 (en) * | 2011-11-03 | 2018-05-17 | Netapp Inc. | Interconnect delivery process |
CN105472023A (zh) * | 2014-12-31 | 2016-04-06 | 华为技术有限公司 | 一种远程直接存储器存取的方法及装置 |
CN108616927A (zh) * | 2018-04-04 | 2018-10-02 | 北京锐安科技有限公司 | 一种数据发送和接收方法及装置 |
CN109547531A (zh) * | 2018-10-19 | 2019-03-29 | 华为技术有限公司 | 数据处理的方法、装置和计算设备 |
CN109587112A (zh) * | 2018-10-25 | 2019-04-05 | 华为技术有限公司 | 一种发送数据的方法、接收数据的方法、设备及*** |
CN110620762A (zh) * | 2019-08-06 | 2019-12-27 | 阿里巴巴集团控股有限公司 | 基于rdma的数据传输方法、网卡、服务器及介质 |
Also Published As
Publication number | Publication date |
---|---|
CN110704361A (zh) | 2020-01-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11023412B2 (en) | RDMA data sending and receiving methods, electronic device, and readable storage medium | |
CN108965484B (zh) | 一种物联网数据的传输方法、***及终端 | |
JP7029471B2 (ja) | アップリンクデータ解凍、圧縮方法および装置 | |
CN109756536B (zh) | 一种数据传输的方法、装置及*** | |
CN108028833B (zh) | 一种nas数据访问的方法、***及相关设备 | |
CN109831409B (zh) | 数据传输方法、装置、计算机可读存储介质及电子设备 | |
CN101409675A (zh) | 网络分组净荷压缩 | |
US20210160353A1 (en) | Data sending method and device, data receiving method and device, and system | |
US20140269774A1 (en) | System and Method for Multi-Stream Compression and Decompression | |
CN112399479B (zh) | 用于数据传输的方法、电子设备和存储介质 | |
CN112671771B (zh) | 数据传输方法、装置、电子设备及介质 | |
US10757032B2 (en) | Methods, apparatuses and computer-readable storage mediums for communication via user services platform | |
WO2021036189A1 (zh) | Rdma数据发送及接收方法、电子设备及可读存储介质 | |
CN109688208B (zh) | 一种基于fc-ae-asm消息的数据加载方法 | |
TW201436533A (zh) | 資料流傳輸系統及方法 | |
CN107615810A (zh) | 用于在线网络代码的包头压缩***和方法 | |
CN113364790B (zh) | 数据传输方法及装置 | |
CN115499173A (zh) | 一种基于udp协议的可信通讯方法和*** | |
WO2020259704A1 (zh) | 一种用于电子设备的数据压缩、数据解压方法及电子设备 | |
CN114710426A (zh) | 一种确定读操作往返时延的方法、装置、***及相关设备 | |
US10742783B2 (en) | Data transmitting apparatus, data receiving apparatus and method thereof having encoding or decoding functionalities | |
US20190124180A1 (en) | Packet compression and decompression | |
CN101197823A (zh) | 在压缩/解压缩过程中传输解压缩信息的方法、***及装置 | |
WO2022257533A1 (zh) | 一种数据传输方法、装置、存储介质及电子装置 | |
CN114979794B (zh) | 一种数据发送方法及装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20857899 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20857899 Country of ref document: EP Kind code of ref document: A1 |