CN116594951B - FPGA-based data transmission system and method - Google Patents

FPGA-based data transmission system and method Download PDF

Info

Publication number
CN116594951B
CN116594951B CN202310854194.0A CN202310854194A CN116594951B CN 116594951 B CN116594951 B CN 116594951B CN 202310854194 A CN202310854194 A CN 202310854194A CN 116594951 B CN116594951 B CN 116594951B
Authority
CN
China
Prior art keywords
gate array
programmable gate
field programmable
data
fpga
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310854194.0A
Other languages
Chinese (zh)
Other versions
CN116594951A (en
Inventor
张斌
周勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Xintong Future Technology Development Co ltd
Original Assignee
Beijing Xintong Future Technology Development Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Xintong Future Technology Development Co ltd filed Critical Beijing Xintong Future Technology Development Co ltd
Priority to CN202310854194.0A priority Critical patent/CN116594951B/en
Publication of CN116594951A publication Critical patent/CN116594951A/en
Application granted granted Critical
Publication of CN116594951B publication Critical patent/CN116594951B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
    • G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/0026—PCI express
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The application discloses a data transmission system and a method based on an FPGA, wherein the data transmission system based on the FPGA comprises: a host device; the field programmable gate array device is loaded in the host device, is in communication connection with the host device and is connected with remote memory direct access equipment at a remote end; the field programmable gate array device is provided with a shared memory area, and the field programmable gate array device and the host device both read or write data from the shared memory area. The application can realize simple interaction between the field programmable gate array device and the host device.

Description

FPGA-based data transmission system and method
Technical Field
The application relates to the technical field of data transmission, in particular to a data transmission system and method based on an FPGA.
Background
With the continuous development of high-performance computing and big data applications, the demand for high-speed and efficient data transmission for data center networks is continuously increasing. Traditional network communication protocols (such as TCP/IP) have the problems of high delay, low throughput, low CPU utilization and the like, and cannot meet the requirements of high-performance computing and data center networks. Thus, remote direct memory access (RDMA, remote Direct Memory Access) technology is widely used, while RoCE v2 is an important RDMA technology that enables low-latency, high-throughput data transmission over ethernet. In the general RDMA network adapter field, there are general RDMA protocol stack drivers and libibverbs libraries to support RDMA programming for RDMA network programming. In the vendor level, RDMA protocol stack driver parts related to vendor hardware are added in RDMA protocol stack drivers, so that development difficulty is greatly increased. In the technical field of embedded RoCEv2, xilinx provides an IP core for implementing RoCEv2 technology and corresponding reference design and driver support in an embedded Linux system, and development of an embedded RDMA network adapter customized for a manufacturer with requirements is performed, but no complete development scheme support is provided on heterogeneous programming involving FPGA (FieldProgrammable Gate Array ) devices and host device CPUs. Although there are some general heterogeneous programming frameworks, platform adaptation and familiarity with corresponding heterogeneous programming techniques are required, which increases the difficulty and cost of technology development.
Disclosure of Invention
The present application has been made to solve the above-mentioned technical problems. The embodiment of the application provides a data transmission system and a data transmission method based on an FPGA, which can realize simple interaction between field programmable gate array equipment and host equipment.
According to one aspect of the present application, there is provided an FPGA-based data transmission system comprising: a host device; the field programmable gate array device is loaded in the host device, is in communication connection with the host device and is connected with remote memory direct access equipment at a remote end; the field programmable gate array device is provided with a shared memory area, and the field programmable gate array device and the host device both read or write data from the shared memory area.
In one embodiment, the field programmable gate array apparatus includes: the system comprises a ciphertext block link message authentication code module, a data packet and a data packet, wherein the ciphertext block link message authentication code module is connected with the Ethernet and is constructed as an outgoing Ethernet data packet; a packet filter module, coupled to the ciphertext block chaining message authentication code module, the packet filter module configured to divide the ethernet packet into a RoCE v2 protocol packet and a non-RoCEv 2 protocol packet; an ERNIC module, coupled to the packet filter module, the ERNIC module configured to parse the RoCE v2 protocol packet; an AXIDMA module, wherein the AXIDMA module is connected with the ERNIC module, and the AXIDMA module is configured to write the data parsed by the ERNIC module into the shared memory area; and the soft core processor is connected with the ciphertext block link message authentication code module.
In an embodiment, the field programmable gate array apparatus further comprises: an XDMA module, the XDMA module being coupled to the host device, the XDMA module being configured to complete control interaction and data transfer with the host device; an mb_xdma_handleshake module, the mb_xdma_handleshake module being connected to the soft core processor, the mb_xdma_handleshake module configured to implement control command communication; the mb_xdma_handleshake module comprises a block RAM, wherein the block RAM is used for forming the shared memory area; and the double-rate synchronous dynamic random access memory module forms the shared memory area.
In one embodiment, the field programmable gate array device is connected to the host device through a PCIe bus, and the field programmable gate array device is connected to the remote memory direct access device at a remote end through a QSFP interface.
According to another aspect of the present application, there is provided an FPGA-based data transmission method, which is applicable to the FPGA-based data transmission system in any one of the foregoing embodiments, wherein the FPGA-based data transmission method uses the field programmable gate array device as a client, a remote memory direct access device at a remote end as a server, and the field programmable gate array device is communicatively connected with a host device, and the FPGA-based data transmission method includes: the field programmable gate array device reads data in a current shared memory area; when the data in the shared memory area indicates that a transmission task exists, the field programmable gate array device analyzes the data in the current shared memory area and acquires an operation command and the data to be transmitted; wherein the operation command and the data to be transmitted are from the host device; according to the operation command, the field programmable gate array device sends data to be transmitted to the client; after the data transmission is completed, the field programmable gate array device writes communication end information into the shared memory area, and sends a signal of the completion of the data transmission to be transmitted to the host device.
In an embodiment, before the field programmable gate array device reads the data in the current shared memory area, the FPGA-based data transmission method further includes: the field programmable gate array device clears the data in the shared memory area.
According to another aspect of the present application, there is provided an FPGA-based data transmission method, which is applicable to the FPGA-based data transmission system described in any one of the above embodiments, wherein the FPGA-based data transmission method uses a host device as a client, and a field programmable gate array device is communicatively connected to the host device, and a remote memory direct access device at a remote end is used as a server, and the FPGA-based data transmission method includes: the host device reads data from a shared memory area of the field programmable gate array device; when the data represents that the field programmable gate array device determines to receive data to be transmitted, storing the data to be transmitted into the shared memory area of the field programmable gate array device; notifying the field programmable gate array device to read the data to be transmitted from the shared memory area; and acquiring a signal of completion of data transmission to be transmitted at the time sent by the field programmable gate array device.
According to another aspect of the present application, there is provided an FPGA-based data transmission method, which is applicable to the FPGA-based data transmission system in any one of the foregoing embodiments, where the FPGA-based data transmission method uses the field programmable gate array device as a server side, a remote memory direct access device at a remote end as a client side, and the field programmable gate array device is communicatively connected with a host device, and the FPGA-based data transmission method includes: the field programmable gate array device monitors a connection request of the client; when the client initiates a connection request, the field programmable gate array device receives client data of the client; the field programmable gate array device stores the client data into a shared memory area of the field programmable gate array device; the field programmable gate array device notifying the host device to begin receiving the client data; and when the field programmable gate array equipment receives a signal of receiving completion sent by the host equipment, continuing to monitor the connection request of the client.
In an embodiment, after the field programmable gate array device receives the client data of the client when the client initiates the connection request, the FPGA-based data transmission method further includes: the field programmable gate array device analyzes the client data to obtain an operation command; the field programmable gate array device transmits the operation command to the host device through the shared memory area; the field programmable gate array device acquires data to be transmitted stored by the host device based on the operation command from the shared memory area; and sending the data to be transmitted to the client.
According to another aspect of the present application, there is provided an FPGA-based data transmission method, which is applicable to the FPGA-based data transmission system described in any one of the above embodiments, wherein the FPGA-based data transmission method uses a host device as a server, and a field programmable gate array device is communicatively connected to the host device, and a remote memory direct access device at a remote end is used as a client, and the FPGA-based data transmission method includes: the host device reads data from a shared memory area of the field programmable gate array device; when the host equipment determines that the data to be transmitted exist in the data, the host equipment copies the data to be transmitted from the shared memory area; when copying is completed, the host device transmits a signal of completion of reception to the field programmable gate array device.
The FPGA-based data transmission system and the FPGA-based data transmission method provided by the application have the advantages that a reserved memory is opened up in the field programmable gate array equipment, the reserved memory is set in the equipment tree to serve as a shared memory area between the host equipment and the field programmable gate array equipment, and related data can be read or written from the designated position of the shared memory space when the two parties communicate, so that the control information transmission and the data transmission of the two parties are realized, the dilemma that the support is needed by using complex development tools and heterogeneous programming platforms is solved, and the simple interaction between the field programmable gate array equipment and the host equipment is realized.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing embodiments of the present application in more detail with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of embodiments of the application and are incorporated in and constitute a part of this specification, illustrate the application and together with the embodiments of the application, and not constitute a limitation to the application. In the drawings, like reference numerals generally refer to like parts or steps.
Fig. 1 is a schematic structural diagram of an FPGA-based data transmission system according to an exemplary embodiment of the present application.
Fig. 2 is a schematic diagram of a field programmable gate array device according to an exemplary embodiment of the present application.
Fig. 3 is a flowchart of an FPGA-based data transmission method according to an exemplary embodiment of the present application.
Fig. 4 is a schematic diagram of an FPGA-based data transmission method according to an exemplary embodiment of the present application.
Fig. 5 is a flowchart of a data transmission method based on FPGA according to another exemplary embodiment of the present application.
Fig. 6 is a schematic diagram of an FPGA-based data transmission method according to another exemplary embodiment of the present application.
Fig. 7 is a flowchart of a data transmission method based on FPGA according to another exemplary embodiment of the present application.
Fig. 8 is a schematic diagram of a data transmission method based on FPGA according to another exemplary embodiment of the present application.
Fig. 9 is a flowchart of a data transmission method based on FPGA according to another exemplary embodiment of the present application.
Fig. 10 is a schematic diagram of an FPGA-based data transmission method according to another exemplary embodiment of the present application.
Fig. 11 is a block diagram of an electronic device according to an exemplary embodiment of the present application.
Reference numerals illustrate: 1. a host device; 2. a field programmable gate array device; 3. remote memory direct access device at remote end; 21. a shared memory area; 22. the ciphertext grouping links the message authentication code module; 23. a packet filter module; 24. an ERNIC module; 25. an AXI DMA module; 26. a soft core processor; 27. an XDMA module; 28. an mb_xdma_handleshake module; 29. a double rate synchronous dynamic random access memory module; 30. a block RAM; 31. a hardware handshake module; 32. an AXI GPIO module; 33. a peripheral device connection module; 34. an AXI Interconnect1 module; 35. an AXI Interconnect2 module; 36. AXI Interconnect3 module.
Detailed Description
Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
Summary of the application
The embedded RoCE (RDMAover Converged Ethernet, remote memory direct access over converged ethernet) v2 technology was developed based on the RoCE v2 technology. The embedded RoCE v2 technology integrates the RoCE v2 technology into a network adapter or SoC chip, thereby achieving higher performance and lower power consumption. Compared with the traditional remote memory direct access (Remote Direct Memory Access, RDMA) technology, the embedded RoCE v2 technology does not need an additional network adapter or a hardware accelerator card and can be directly realized through hardware integration of an SoC chip, so that the cost and the power consumption are greatly reduced. In the development process of the embedded RoCE v2 technology, some related technologies, such as embedded RoCE v2 technology based on FPGA (FieldProgrammable Gate Array ), application of the embedded RoCE v2 technology in cloud computing, and the like, have also emerged. The advent of these technologies has further driven the development and application of embedded RoCE v2 technology.
However, the prior art is mainly oriented to HPC (High-performance computing) and data center, the development process is complex, the whole development process of the hardware platform needs to be very familiar, and meanwhile, manufacturers need to support different hardware platforms in an adapting way, and although the software platform and the development tool are completely unified, the software platform and the development tool are not friendly, and a large amount of adaptation and debugging work is needed between different FPGA platforms for hardware engineers and software engineers who are unfamiliar with the development tool and the process. Therefore, the invention abandons a complex technical scheme in terms of heterogeneous data transmission on the basis of realizing the RoCEv2 reference design scheme of Xilinx, and adds a new IP core in hardware on the basis of the prior general technology for communicating with a host. On the software, a scheme of sharing storage space among heterogeneous systems is adopted to carry out program control and data transmission between a host system and a device system, so that the dilemma of supporting by using complex development tools and heterogeneous programming platforms is solved.
Exemplary System
Fig. 1 is a schematic structural diagram of an FPGA-based data transmission system according to an exemplary embodiment of the present application, and as shown in fig. 1, the FPGA-based data transmission system includes: a host device 1; the field programmable gate array device 2, the field programmable gate array device 2 is connected with the host device 1 in a communication way, and the field programmable gate array device 2 is connected with the remote memory direct access device 3; the field programmable gate array device 2 is provided with a shared memory area 21, and the field programmable gate array device 2 and the host device 1 both read or write data from the shared memory area 21.
The remote memory direct access device 3 is other remote host devices loaded with remote memory direct access network cards, the remote memory direct access device 3 needs to perform data interaction with the host device 1, the field programmable gate array device 2 is used as an intermediary device to realize heterogeneous communication with the host device 1, and the field programmable gate array device 2 and the host device 1 form a server side, so that the data interaction from the client side to the server side of the remote memory direct access device 3 is simply and rapidly completed. The field programmable gate array device 2 is connected to the host device 1 through a PCIe (PeripheralComponent Interface Extend) bus, and the PCIe bus is a tree-shaped interface bus. The field programmable gate array device 2 is connected with the remote memory direct access device 3 at the far end through a QSFP (quad Small Form-factor plug) interface, and the QSFP interface can realize high-speed plug in and plug out with higher density. A reserved memory is opened up in the field programmable gate array device 2, the reserved memory is set in a device tree, and is used as a shared memory area 21 between the host device 1 and the field programmable gate array device 2, and when the two parties communicate, related data can be read or written from a designated position of the shared memory space, so that control information transmission and data transmission of the two parties are realized, and the dilemma that the two parties need to use complex development tools and heterogeneous programming platforms for supporting is solved.
Fig. 2 is a schematic structural diagram of a field programmable gate array device according to an exemplary embodiment of the present application, and as shown in fig. 2, the field programmable gate array device includes: a ciphertext block Chaining message authentication code (CMAC, cipherBlock training-Message Authentication Code) module 22, the ciphertext block Chaining message authentication code module 22 being connected to the ethernet, the ciphertext block Chaining message authentication code module 22 being configured to transmit ethernet packets; a Packet filter (Packet filter) module 23, the Packet filter module 23 being connected to the ciphertext block chaining message authentication code module 22, the Packet filter module 23 being configured to divide ethernet packets into RoCE v2 protocol packets and non-RoCEv 2 protocol packets; an ERNIC (EmbeddedRDMA enabled NIC, embedded RDMA compatible NIC) module 24, the ERNIC module 24 coupled to the packet filter module 23, the ERNIC module 24 configured to parse the RoCEv2 protocol packets; AXI DMA (DirectMemory Access ) module 25, AXIDMA module 25 is connected with ERNIC module 24, AXIDMA module 25 is configured to write the data parsed by ERNIC module 24 into the shared memory area; a soft-core processor (Microblaze processor) 26, the soft-core processor 26 being connected to the ciphertext block chaining message authentication code module 22; an XDMA module 27, the XDMA module 27 being connected to the host device, the XDMA (XilinxDirect Memory Access) module 27 being configured to complete control interaction and data transfer with the host device; an mb_xdma_handleshake module 28, the mb_xdma_handleshake module 28 being coupled to the soft core processor 26, the mb_xdma_handleshake module 28 being configured to enable control command communication; the mb_xdma_handleshare module 28 includes a Block RAM (BRAM) 30, where the block RAM30 is used to form a shared memory area; the double rate synchronous dynamic random access memory (Double Data Rate SynchronousDynamic Random Access Memory, DDR) module 29, the double rate synchronous dynamic random access memory module 29 constitutes a shared memory area.
The invention aims to realize an embedded RoCEv2 technology based on an FPGA, which consists of two parts of hardware and software. The hardware component may implement embedded RoCE v2 based on the Xilinx corporation's Virtex ™ ultrascale+ ™ FPGA VCU118 development board. The VCU118 development board includes an FPGA chip (e.g., a chip model XCU 9P-FLGA 2104-2L-e), two 28Gb/sQSFP+ connectors, a UART (UniversalAsynchronous Receiver/Transmitter ) serial port, a JTAG (JointTestActionGroup) interface, BPI16-bitFLASH, and 4Gb (256 Mbx) DDR4-2400. The VCU118 development board is plugged into the server golden finger slot through a PCIe connector. The FPGA internal design takes ERNIC (Xilinx Embedded RDMAenabled NIC) IP as a core, so that the core function of the embedded RoCEv2 is realized. On the basis, XDMA IP is used, and an mb_xdma_handleshake module is added to realize control command interaction and data transmission between a server host and an FPGA. Among these, the ERNIC IP may employ LogiCOREIP developed by Xilinx corporation, which is an implementation of NIC functionality supporting RDMAover Converged Ethernet (RoCE v 2). And a soft core processor (Microblaze processor) is adopted as a soft core CPU inside the FPGA, and a Linux operating system is operated. Microblaze controls the ERNIC module via AXI4-Lite protocol, which is mainly responsible for implementation of RoCEv2 protocol and parsing of ethernet packets. The internal system design of the field programmable gate array device is shown in fig. 2, and mainly comprises the following parts: ERNIC module 24, soft core processor 26, XDMA module 27, AXIDMA module 25, ciphertext block chaining message authentication code module 22, AXIGPIO module 32, peripheral connection module 33, AXIINTERCONNECT1 module 34, AXI Interconnect2 module 35, AXIINTERCONNECT3 module 36, double rate synchronous dynamic random memory module 29, packet filter module 23, hardware handshake module 31, mb_xdma_handle shake module 28, and block RAM30.
By using MicroBlazeIP, the CPU is realized in a soft core mode, so that a Linux operating system can run in the FPGA. The soft-core processor 26, acting as a master of the RoCEv2 system, is responsible for the print display of terminal information, the configuration of the AXI DMA module 25, the configuration of the hardware handshake module, the handling of interrupts, command interactions with the server, the initialization of the ERNIC module 24 and the configuration. Implementation of the RoCE v2 protocol is accomplished by using ERNIC IP. The ciphertext block chaining message authentication code module 22 is responsible for the physical layer and the link layer of 100Gb/s Ethernet, implemented using UltraScale+ ™ Devices Integrated 100G Ethernet IP. The packet filter module 23 is responsible for dividing the ethernet packet transmitted from the ciphertext packet chaining message authentication code module 22 into a RoCEv2 protocol packet and a non-RoCEv 2 protocol packet, and then transmitting the packets to the ERNIC module 24 via the AXI4-Stream protocol. AXIDMA module 25 is responsible for handling data parsed by ERNIC module 24 and writing data into double Rate synchronous dynamic random Access memory module 29. The hardware handshake (hard handshake) module is responsible for doorbell mechanism responses with the ERNIC module 24. The mb_xdma_handle module 28 is responsible for control command communication between the server and the soft-core processor 26. The mb_xdma_handleshake module 28 consists essentially of block RAM30 and control logic. The block RAM30 is considered a block of register space, and the transfer of control commands and parameters is accomplished by defining different address bits. The XDMA module 27 is implemented by using XDMAIP of Xilinx corporation. Is responsible for transaction processing of PCIe packets to complete control command interaction and data transfer between the server host and the field programmable gate array device. A peripheral connection (peripheral interconnect) module 33 is used to implement the connection. AXIINTERCONNEct1 module 34, AXI Interconnect2 module 35, AXIINTERCONNEct3 module 36 are used as connection modules to connect other modules in the FPGA device with the shared memory area and read/write the other modules. AXI GPIO module 32 is a functional module implemented by a user by configuring the logic resources of the chip.
In addition, the embedded RoCEv2 technology based on the FPGA is realized by matching with a software part, the software part is mainly divided into three blocks, namely a software adaptation and RDMA application program interface based on FPGA equipment (field programmable gate array equipment), a Host interface program connected with the FPGA equipment and a server/client interface program carrying a universal RDMA network card. 1. The software program inside the FPGA device runs on a Xilinx microblaze platform, and the development tool uses the petalinux tool of Xilinx corporation. The RDMA application program migration and function development are mainly based on ERNIC IP drivers, memory management programs and other drivers provided by Xilinx corporation, three basic operations of RDMA RDMASEND, RDMA READ and RDMAWRITE of RDMA are mainly realized in an FPGA, and data transmission from an FPGA device to a remote RDMA device based on RoCEv2 is realized in an application program layer. The FPGA device may be a stand-alone computing device with RoCE v2 functionality, or may be a peripheral device of the Host, typically a PCIe device. When the RDMA application program is used as a peripheral device, the RDMA application program needs to be divided into two parts, wherein one part is an RDMA program main body running in the FPGA device, and the other part is a part running in a Host device and interacting with the RDMA program in the FPGA device. For this purpose, control interaction and data transfer of Host control programs with RDMA programs in FPGA devices can be achieved through shared memory regions. 2. The Host interface program mainly provides a method at the Host end, and the method performs control interaction and data transfer with an RDMA program in the FPGA device by reading or writing data into a shared memory area of the FPGA device. After receiving the corresponding command and data, the RDMA program in the FPGA device is transmitted to a remote host device loaded with an RDMA network card through an ERNIC module after being analyzed. Thereby, the communication between the Host loaded with the FPGA device and the Host device remotely loaded with the RDMA network card is achieved. 3. In a Host for loading a general RDMA network card, an RDMA interface program matched with an RDMA program in an FPGA device is needed to realize interaction with the RDMA program in the FPGA device, and the development flow is not different from a common RDMA application program based on a C/S architecture. The FPGA equipment and the RDMA application program in the Host computer loaded with the general RDMA network card can be server programs or client programs, and the configuration is carried out according to the use situation. The three parts form an application program framework in the invention, and the application of the FPGA-based RoCE v2 technology is realized through the combination of the three parts.
Exemplary method 1
Fig. 3 is a flow chart of an FPGA-based data transmission method according to an exemplary embodiment of the present application, as shown in fig. 3, where the FPGA-based data transmission method is applicable to the FPGA-based data transmission system according to any one of the embodiments of the present application, the FPGA-based data transmission method uses a field programmable gate array device as a client, a remote memory direct access device as a server, the field programmable gate array device is communicatively connected to a host device, and the exemplary method 1 uses the field programmable gate array device as the client as an execution subject, and the FPGA-based data transmission method includes:
step 110: the field programmable gate array device reads the data in the current shared memory area.
The field programmable gate array device reads data in the shared memory area of the field programmable gate array device side, and judges whether a new transmission task exists on the Host device (Host) side from the read data.
Step 120: when the data in the shared memory area indicates that a transmission task exists, the field programmable gate array device analyzes the data in the current shared memory area and acquires an operation command and the data to be transmitted.
Wherein the operation command and the data to be transmitted come from the host device.
If there is a new transmission task, the field programmable gate array device reads data from the designated location in the shared memory area, and if there is no new transmission task, continues to read data from the shared memory area. The field programmable gate array device analyzes the read data, acquires an operation command and data to be transmitted, wherein the part of data comprises an IP address and a port number of a server side, and an address and a size of the data to be transmitted in a shared memory area of the field programmable gate array device side.
Step 130: and according to the operation command, the field programmable gate array equipment sends data to be transmitted to the client.
According to the analyzed operation command, an RDMA communication module at the field programmable gate array device side is called, and remote memory direct access data is sent to remote memory direct access equipment through the RDMA communication module.
Step 140: after the data transmission is completed, the field programmable gate array device writes communication end information into the shared memory area and sends a signal of completing the data transmission to be transmitted to the host device.
After the data is sent, writing communication end information into a designated position in a shared memory area of the field programmable gate array device, informing the host device that one RDMA transmission is completed, and enabling the host device to perform data transmission preparation of the next round.
In an embodiment, before the step 110, the data transmission method based on FPGA may further include: the field programmable gate array device clears the data in the shared memory area.
And when the program is started, the shared memory area at the field programmable gate array equipment side is emptied, so that the influence of participation information is prevented.
Fig. 4 is a schematic diagram of a data transmission method based on an FPGA according to an exemplary embodiment of the present application, as shown in fig. 4, in which a field programmable gate array device side is used as a client, a remote direct access device is used as a server, when a program is started, a shared memory area on the field programmable gate array device side is emptied (S11), the influence of participation information is prevented, data is read from the shared memory (S12), whether a new transmission task exists on the host device side is determined from the read data, if the new transmission task exists, whether the data can be acquired from the shared memory area is determined (S13), if the data cannot be acquired, S12 is executed, and if the data can be acquired, the data is read from a designated location in the shared memory area (S14). After reading the data, the read data is parsed, and the operation command and the data to be transmitted are acquired (S15). And calling an RDMA communication module at the side of the field programmable gate array device, sending data to be transmitted to a remote memory direct access device at a far end through the RDMA communication module (S16), writing communication ending information to a designated position in a shared memory area of the field programmable gate array device when communication is ended, informing a host device that one RDMA transmission is finished (S17), and then continuing to execute S12.
Exemplary method 2
Fig. 5 is a flowchart of an FPGA-based data transmission method according to an exemplary embodiment of the present application, as shown in fig. 5, where the FPGA-based data transmission method is applicable to the FPGA-based data transmission system according to any one of the embodiments of the present application, the FPGA-based data transmission method uses a host device as a client (an RDMA client is implemented in a field programmable gate array device and is used with a client interface in the field programmable gate array device), and the field programmable gate array device is communicatively connected to the host device, a remote direct memory access device is used as a server, and the exemplary method 2 uses the host device as a client as an execution subject, and the FPGA-based data transmission method includes:
step 210: the host device reads data from the shared memory area of the field programmable gate array device.
The host device reads data from the shared memory area on the field programmable gate array device side, which can be used by the host device to determine whether the field programmable gate array device is ready to receive data to be transmitted.
Step 220: when the data represents that the field programmable gate array device determines to receive the data to be transmitted, the data to be transmitted is stored in a shared memory area of the field programmable gate array device.
When the data represents that the field programmable gate array device determines to receive the data to be transmitted, the host device prepares the data to be transmitted and places the data to be transmitted into the shared memory area of the field programmable gate array device side.
Step 230: and informing the field programmable gate array device to read the data to be transmitted from the shared memory area.
The host device notifies the field programmable gate array device to begin reading data to be transferred from the shared memory area.
Step 240: and acquiring a signal which is sent by the field programmable gate array equipment and is transmitted by the data to be transmitted at the time.
After the data transmission to be transmitted is completed, the host device acquires a signal that the data transmission to be transmitted is completed, which is sent by the field programmable gate array device, and the host device continues to read data from the shared memory area at the side of the field programmable gate array device.
Fig. 6 is a schematic diagram of a data transmission method based on FPGA according to another exemplary embodiment of the present application, as shown in fig. 6, in which a host device side is used as a client (an RDMA client is implemented in a field programmable gate array device and is used together with a client interface in the field programmable gate array device), and after a program is started, data is read from a shared memory area in the field programmable gate array device side (S21); judging whether the field programmable gate array device side is ready to receive data (S22); if the data is ready to be received, go to S23; otherwise, go to S21; preparing data to be transmitted (S23); placing the data into a shared memory area at the field programmable gate array device side (S24); notifying the field programmable gate array device side that data can be read from the shared memory area (S25); returning to S21.
Exemplary method 3
Fig. 7 is a flowchart of an FPGA-based data transmission method according to an exemplary embodiment of the present application, as shown in fig. 7, where the FPGA-based data transmission method is applicable to the FPGA-based data transmission system according to any one of the embodiments of the present application, the FPGA-based data transmission method uses a field programmable gate array device as a server, a remote memory direct access device as a client, the field programmable gate array device is communicatively connected to a host device, and the exemplary method 3 uses the field programmable gate array device as the server as an execution subject, and the FPGA-based data transmission method includes:
step 310: the field programmable gate array device listens for connection requests from clients.
The field programmable gate array device continuously monitors the connection request of the client and checks whether the client has the connection request.
Step 320: when a client initiates a connection request, the field programmable gate array device receives client data for the client.
If a client initiates a connection request, the field programmable gate array device determines to begin receiving client data of a remote memory direct access client at a remote end.
Step 330: the field programmable gate array device stores the client data to a shared memory area of the field programmable gate array device.
And storing the client data into a shared memory area of the field programmable gate array device so that the host device receives the client data sent by the client.
Step 340: the field programmable gate array device notifies the host device to begin receiving client data.
After the field programmable gate array device completes the storage of the client data in the shared memory area, the host device is informed to start copying the client data from the shared memory area, and the client data is copied into the host device.
Step 350: and when the field programmable gate array device receives a signal of receiving completion sent by the host device, continuing to monitor the connection request of the client.
And judging whether the host equipment receives the data or not by reading the designated area in the shared memory area of the field programmable gate array equipment side, and if the host equipment receives the data, continuously monitoring the connection request of the client so as to finish the data transmission of the next client.
FIG. 8 is a schematic diagram of a data transmission method based on an FPGA according to another exemplary embodiment of the present application, as shown in FIG. 8, when a program is started, an RDMA server module is initialized (S31); continuously listening for client connection requests (S32); judging whether a connection request exists (S33), and if so, going to S34; otherwise, go to S32; receiving client data (S34); placing the received client data into a shared memory area of the field programmable gate array device side (S35); notifying the host device side that client data can be started to be accepted (S36); reading a designated area of a shared memory area at the field programmable gate array device side (S37); judging whether the host device side has received the data (S38); if the data has been received, go to S32; otherwise, go to S37.
In an embodiment, after the step 320, the data transmission method based on FPGA may further include: the field programmable gate array device analyzes the client data to obtain an operation command; the field programmable gate array device transmits the operation command to the host device through the shared memory area; the field programmable gate array device acquires data to be transmitted stored by the host device based on the operation command from the shared memory area; and sending the data to be transmitted to the client.
In addition, after the RDMA application program module of the field programmable gate array device end analyzes the command request, a corresponding operation command and data are transmitted to the Host device end through the shared memory area, the Host device judges whether the data need to be returned to the client end according to the analyzed command, if so, the data to be transmitted are put into the shared memory area, after the field programmable gate array device end obtains the data, the data are transmitted to the remote memory direct access device of the remote end through the RDMA application program module, and therefore data interaction from the remote memory direct access client end of the remote end to a service end formed by the field programmable gate array device and the Host end is completed once.
Exemplary method 4
Fig. 9 is a flow chart of an FPGA-based data transmission method according to an exemplary embodiment of the present application, as shown in fig. 9, where the FPGA-based data transmission method is applicable to the FPGA-based data transmission system according to any one of the embodiments of the present application, the FPGA-based data transmission method uses a host device as a server, and a field programmable gate array device is communicatively connected to the host device, and a remote memory direct access device is used as a client, and the FPGA-based data transmission method includes:
step 410: the host device reads data from the shared memory area of the field programmable gate array device.
The host device reads the data of the designated area in the shared memory area, thereby judging whether the data can be received from the field programmable gate array device side.
Step 420: when the host device determines that there is data to be transmitted in the data, the host device copies the data to be transmitted from the shared memory area.
When the host device determines that the data can be started to be received from the field programmable gate array device and the data to be transmitted exists in the data, the host device copies the data to be transmitted from the shared memory area and copies the data to be transmitted into the host device.
Step 430: when the copying is completed, the host device transmits a reception completion signal to the field programmable gate array device.
When the copying is completed, the host device notifies the field programmable gate array device that the data to be transferred has been copied from the shared memory area. After the copying is completed, the host device continues to read data from the shared memory area of the field programmable gate array device, thereby judging whether the next data transmission exists.
Fig. 10 is a schematic diagram of a data transmission method based on FPGA according to another exemplary embodiment of the present application, and as shown in fig. 10, a host device is used as a server side and a field programmable gate array device is used as an interaction flow chart of the server side. Starting a program; reading data of a designated area of a shared memory area at the field programmable gate array equipment side (S41); judging whether or not the data can be received from the field programmable gate array device side (S42); if the data is ready to be received, go to S43; otherwise, go to S41; copying data from the shared memory area on the field programmable gate array device side to the host device (S43); notifying the field programmable gate array device side that the copying of the data from the shared memory area is completed (S44); go to S41.
Exemplary electronic device
An electronic device, the electronic device comprising: a processor; a memory for storing processor-executable instructions; and the processor is used for executing the data transmission method based on the FPGA according to the embodiment provided by the application.
Next, an electronic device according to an embodiment of the present application is described with reference to fig. 11. The electronic device may be either or both of the first device and the second device, or a stand-alone device independent thereof, which may communicate with the first device and the second device to receive the acquired input signals therefrom.
Fig. 11 illustrates a block diagram of an electronic device according to an embodiment of the application.
As shown in fig. 11, the electronic device 10 includes one or more processors 11 and a memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM) and/or cache memory (cache), and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer readable storage medium that can be executed by the processor 11 to implement the FPGA-based data transmission method and/or other desired functions of the various embodiments of the present application described above. Various contents such as an input signal, a signal component, a noise component, and the like may also be stored in the computer-readable storage medium.
In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown).
When the electronic device is a stand-alone device, the input means 13 may be a communication network connector for receiving the acquired input signals from the first device and the second device.
In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.
The output device 14 may output various information to the outside, including the determined distance information, direction information, and the like. The output means 14 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, etc.
Of course, only some of the components of the electronic device 10 that are relevant to the present application are shown in fig. 11 for simplicity, components such as buses, input/output interfaces, etc. being omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.
The computer program product may write program code for performing operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.
A computer readable storage medium stores a computer program for executing the FPGA-based data transmission method according to the embodiment of the present application.
The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the application to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.

Claims (10)

1. An FPGA-based data transmission system, comprising:
a host device;
the field programmable gate array device is loaded in the host device, is in communication connection with the host device and is connected with remote memory direct access equipment at a remote end;
the field programmable gate array device is provided with a shared memory area, and the field programmable gate array device and the host device both read or write data from the shared memory area;
the field programmable gate array apparatus further includes:
an XDMA module, the XDMA module being coupled to the host device, the XDMA module being configured to complete control interaction and data transfer with the host device;
an mb_xdma_handleshake module configured to enable control command communication; the mb_xdma_handleshake module comprises a block RAM, wherein the block RAM is used for forming the shared memory area;
and the double-rate synchronous dynamic random access memory module forms the shared memory area.
2. The FPGA-based data transmission system of claim 1, wherein the field programmable gate array device comprises:
the system comprises a ciphertext block link message authentication code module, a data packet and a data packet, wherein the ciphertext block link message authentication code module is connected with the Ethernet and is constructed as an outgoing Ethernet data packet;
the data packet filter module is connected with the ciphertext block link message authentication code module and is configured to divide the Ethernet data packet into a RoCE v2 protocol packet and a non-RoCE v2 protocol packet;
an ERNIC module, coupled to the packet filter module, the ERNIC module configured to parse the RoCE v2 protocol packet;
an AXI DMA module, where the AXI DMA module is connected to the ERNIC module, and the AXI DMA module is configured to write the data parsed by the ERNIC module into the shared memory area;
and the soft core processor is connected with the ciphertext block link message authentication code module.
3. The FPGA-based data transmission system of claim 2, wherein the mb_xdma_handleshake module is coupled to the soft-core processor.
4. The FPGA-based data transmission system of claim 1, wherein the field programmable gate array device is connected to the host device via a PCIe bus, and the field programmable gate array device is connected to the remote memory direct access device at a remote end via a QSFP interface.
5. An FPGA-based data transmission method, applicable to the FPGA-based data transmission system of any one of claims 1 to 4, characterized in that the FPGA-based data transmission method uses the field programmable gate array device as a client, a remote memory direct access device at a remote end as a server, and the field programmable gate array device is communicatively connected with a host device, and the FPGA-based data transmission method includes:
the field programmable gate array device reads data in a current shared memory area;
when the data in the shared memory area indicates that a transmission task exists, the field programmable gate array device analyzes the data in the current shared memory area and acquires an operation command and the data to be transmitted; wherein the operation command and the data to be transmitted are from the host device;
according to the operation command, the field programmable gate array device sends data to be transmitted to the client;
After the data transmission is completed, the field programmable gate array device writes communication end information into the shared memory area, and sends a signal of the completion of the data transmission to be transmitted to the host device.
6. The FPGA-based data transmission method of claim 5, wherein before the field programmable gate array device reads the data in the current shared memory area, the FPGA-based data transmission method further comprises:
the field programmable gate array device clears the data in the shared memory area.
7. An FPGA-based data transmission method, applicable to the FPGA-based data transmission system of any one of claims 1 to 4, characterized in that the FPGA-based data transmission method uses a host device as a client, and a field programmable gate array device is communicatively connected with the host device, and a remote memory direct access device at a remote end is used as a server, the FPGA-based data transmission method includes:
the host device reads data from a shared memory area of the field programmable gate array device;
when the data represents that the field programmable gate array device determines to receive data to be transmitted, storing the data to be transmitted into the shared memory area of the field programmable gate array device;
Notifying the field programmable gate array device to read the data to be transmitted from the shared memory area;
and acquiring a signal of completion of data transmission to be transmitted at the time sent by the field programmable gate array device.
8. The data transmission method based on FPGA, which is applicable to the data transmission system based on FPGA of any one of claims 1 to 4, is characterized in that the data transmission method based on FPGA uses the field programmable gate array device as a server side, remote memory direct access device at a far end is used as a client side, the field programmable gate array device is in communication connection with a host device, and the data transmission method based on FPGA includes:
the field programmable gate array device monitors a connection request of the client;
when the client initiates a connection request, the field programmable gate array device receives client data of the client;
the field programmable gate array device stores the client data into a shared memory area of the field programmable gate array device;
the field programmable gate array device notifying the host device to begin receiving the client data;
And when the field programmable gate array equipment receives a signal of receiving completion sent by the host equipment, continuing to monitor the connection request of the client.
9. The FPGA-based data transmission method according to claim 8, wherein after the field programmable gate array device receives the client data of the client when the client initiates the connection request, the FPGA-based data transmission method further comprises:
the field programmable gate array device analyzes the client data to obtain an operation command;
the field programmable gate array device transmits the operation command to the host device through the shared memory area;
the field programmable gate array device acquires data to be transmitted stored by the host device based on the operation command from the shared memory area;
and sending the data to be transmitted to the client.
10. An FPGA-based data transmission method, applicable to the FPGA-based data transmission system of any one of claims 1 to 4, characterized in that the FPGA-based data transmission method uses a host device as a server, and a field programmable gate array device is communicatively connected with the host device, and a remote memory direct access device at a remote end is used as a client, the FPGA-based data transmission method includes:
The host device reads data from a shared memory area of the field programmable gate array device;
when the host equipment determines that the data to be transmitted exist in the data, the host equipment copies the data to be transmitted from the shared memory area;
when copying is completed, the host device transmits a signal of completion of reception to the field programmable gate array device.
CN202310854194.0A 2023-07-13 2023-07-13 FPGA-based data transmission system and method Active CN116594951B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310854194.0A CN116594951B (en) 2023-07-13 2023-07-13 FPGA-based data transmission system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310854194.0A CN116594951B (en) 2023-07-13 2023-07-13 FPGA-based data transmission system and method

Publications (2)

Publication Number Publication Date
CN116594951A CN116594951A (en) 2023-08-15
CN116594951B true CN116594951B (en) 2023-09-26

Family

ID=87599356

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310854194.0A Active CN116594951B (en) 2023-07-13 2023-07-13 FPGA-based data transmission system and method

Country Status (1)

Country Link
CN (1) CN116594951B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108989317A (en) * 2018-07-26 2018-12-11 浪潮(北京)电子信息产业有限公司 A kind of RoCE network card data communication method and network interface card based on FPGA
CN110619595A (en) * 2019-09-17 2019-12-27 华中科技大学 Graph calculation optimization method based on interconnection of multiple FPGA accelerators
CN112099940A (en) * 2016-08-26 2020-12-18 华为技术有限公司 Method, equipment and system for realizing hardware acceleration processing
CN113326154A (en) * 2021-06-28 2021-08-31 深信服科技股份有限公司 Connection management method, device, electronic equipment and storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11704059B2 (en) * 2020-02-07 2023-07-18 Samsung Electronics Co., Ltd. Remote direct attached multiple storage function storage device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112099940A (en) * 2016-08-26 2020-12-18 华为技术有限公司 Method, equipment and system for realizing hardware acceleration processing
CN108989317A (en) * 2018-07-26 2018-12-11 浪潮(北京)电子信息产业有限公司 A kind of RoCE network card data communication method and network interface card based on FPGA
CN110619595A (en) * 2019-09-17 2019-12-27 华中科技大学 Graph calculation optimization method based on interconnection of multiple FPGA accelerators
CN113326154A (en) * 2021-06-28 2021-08-31 深信服科技股份有限公司 Connection management method, device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN116594951A (en) 2023-08-15

Similar Documents

Publication Publication Date Title
CN107203484B (en) PCIe and SRIO bus bridging system based on FPGA
CN106951388B (en) PCIe-based DMA data transmission method and system
CN106775434B (en) A kind of implementation method, terminal, server and the system of NVMe networking storage
CN103902486B (en) System, device and method for implementation of remote direct memory access
CN108132897B (en) SRIO controller based on ZYNQ platform soft core
KR101720134B1 (en) Bus bridge apparatus
CN105183680B (en) Realize that PCIe interface turns the fpga chip and method of CF card interfaces
WO2018120780A1 (en) Method and system for pcie interrupt
CN114003392B (en) Data accelerated computing method and related device
CN108255776B (en) I3C master device compatible with APB bus, master-slave system and communication method
CN114253740A (en) Protocol stack data transmission method and device based on Linux kernel
US20230080588A1 (en) Mqtt protocol simulation method and simulation device
JP2008502976A (en) Bus controller for processing split transactions
CN106662895B (en) The method of computer equipment and computer equipment reading and writing data
CN112947857B (en) Data moving method, device, equipment and computer readable storage medium
CN102207920B (en) Conversion bridge for conversion from BVCI (basic virtual component interface) bus to AHB (advanced high performance bus)
CN109634901A (en) A kind of data transmission system and its control method based on UART
US7383372B2 (en) Bus system, station for use in a bus system, and bus interface
US20160077986A1 (en) Electronic apparatus providing real-time switching and sharing of usb electronic devices among hosts
CN116225992A (en) NVMe verification platform and method supporting virtualized simulation equipment
JP2007501472A (en) USB host controller with transfer descriptor memory
US10176133B2 (en) Smart device with no AP
CN116594951B (en) FPGA-based data transmission system and method
KR20120134918A (en) Electronic apparatus including a plurality of processors
JP2008502977A (en) Interrupt method for bus controller

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant