WO2016074619A1 - Pcie bus based data transmission method and device - Google Patents

Pcie bus based data transmission method and device Download PDF

Info

Publication number
WO2016074619A1
WO2016074619A1 PCT/CN2015/094300 CN2015094300W WO2016074619A1 WO 2016074619 A1 WO2016074619 A1 WO 2016074619A1 CN 2015094300 W CN2015094300 W CN 2015094300W WO 2016074619 A1 WO2016074619 A1 WO 2016074619A1
Authority
WO
WIPO (PCT)
Prior art keywords
tlp
bus
pcie
field
function
Prior art date
Application number
PCT/CN2015/094300
Other languages
French (fr)
Chinese (zh)
Inventor
侯新宇
俞洲
林沐晖
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2016074619A1 publication Critical patent/WO2016074619A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14Handling requests for interconnection or transfer

Definitions

  • the requesting device may also include the bus number, device number, and function number of the requesting device in the first PCIe bus domain being written into the requester ID Requester ID field of the read request TLP.
  • the value of the identification Tag field of the read request TLP in the second PCIe bus domain is allocated by the NTB.
  • the value of the identifier Tag field of each read request TLP may be sequentially generated by sequential numbering.
  • the value of the identifier Tag field of the read request TLP in the second PCIe bus domain is the value of the bus number, the device number and the function number of the requesting device, and the value of the identifier Tag field carried in the read request TLP (ie, the read request TLP is in the
  • the mapping relationship of the value of the tag Tag field in a PCIe bus domain can be saved in the form of a table (for example, LUT, etc.).
  • the first indexing unit 521 is configured to save the value of the identifier Tag field of the replaced read request TLP in the second PCIe bus domain as an index in the display lookup table LUT;

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Bus Control (AREA)

Abstract

The present invention relates to the technical field of communication, and disclosed are a PCIe bus based data transmission method and device. The method comprises: a non-transparent bridge (NTB) receives a request TLP transmitted by a requesting device, the request TLP comprising a Requester ID field, a Tag field and an address field; the Requester ID field carries the bus number, device number and function number of the requesting device; the address field is used to indicate the address of a target device, the Requester ID field carrying the bus number, device number and function number of the requesting device; the requesting device and the target device are respectively in the first, and second PCIe bus domains connected via the NTB; substituting the bus number, device number and function number of the requesting device in the requesting TLP with the bus number, device number and function number of the NTB in the second PCIe bus domain, and transmitting the substituted requesting TLP to the target device, such that no error will occur when IOMMU performs legitimacy check on the requesting TLP.

Description

基于PCIe总线的数据传输方法和装置Data transmission method and device based on PCIe bus 技术领域Technical field
本发明涉及通信技术领域,特别涉及一种基于PCIe总线的数据传输方法和装置。The present invention relates to the field of communications technologies, and in particular, to a data transmission method and apparatus based on a PCIe bus.
背景技术Background technique
***设备高速互连(Peripheral Component Interconnect Express,简称“PCIe”)为总线和接口标准。在处理器***中,PCIe总线属于局部总线,主要用于连接外部设备。在一个处理器***中,PCIe设备使用独立的地址空间,即PCIe总线的地址空间,也称为PCIe总线域。处理器可以通过根联合体(Root Complex,简称“RC”)访问RC所属的PCIe总线域中的PCIe设备,该RC所属的PCIe总线域中的PCIe设备也可以通过RC访问主存储器。在包括多个处理器***的计算机***中,可以采用PCIe非透明桥(Non-Transparent Bridging,简称“NTB”)将不同的PCIe总线域的地址空间进行隔离,以连接不同的处理器***,此时该NTB同时作为与其连接的两个PCIe总线域的PCIe设备。一个处理器***中的处理器可以通过NTB访问另一个处理器***中的处理器或者另一个处理器***的PCIe总线域中的PCIe设备。Peripheral Component Interconnect Express ("PCIe") is a bus and interface standard. In the processor system, the PCIe bus belongs to the local bus and is mainly used to connect external devices. In a processor system, the PCIe device uses a separate address space, the address space of the PCIe bus, also known as the PCIe bus domain. The processor can access the PCIe device in the PCIe bus domain to which the RC belongs through a Root Complex ("RC"). The PCIe device in the PCIe bus domain to which the RC belongs can also access the main memory through the RC. In a computer system including multiple processor systems, a PCIe non-transparent bridge (Non-Transparent Bridging (NTB)) may be used to isolate address spaces of different PCIe bus domains to connect different processor systems. The NTB also acts as a PCIe device for the two PCIe bus domains connected to it. A processor in one processor system can access a processor in another processor system or a PCIe device in a PCIe bus domain of another processor system through the NTB.
当一个处理器***中的处理器(简称为第一处理器)通过NTB访问另一个处理器***中的处理器(简称为第二处理器)时,例如,第一处理器想读取第二处理器管理的存储器中的数据时,NTB会对第一处理器发送的存储器读请求事务层包(Transaction Layer Packet,简称“TLP”)中的请求者身份标识Requester ID(即第一处理器的RC在第一PCIe总线域中的ID)进行转换,以满足第二PCIe总线域中的地址空间的要求。When a processor in one processor system (referred to as a first processor for short) accesses a processor in another processor system (referred to as a second processor) through NTB, for example, the first processor wants to read the second When the data in the memory managed by the processor, the NTB sends a request for the memory identifier of the first processor to the requester ID in the Transaction Layer Packet ("TLP") (ie, the first processor The RC is converted in the ID of the first PCIe bus domain to meet the address space requirements in the second PCIe bus domain.
现有技术中,提供了一种NTB对Requester ID进行转换的方法,以存储器读请求TLP为例,当NTB接收到第一处理器发送的存储器读请求TLP时,先在显示查找表(Look-Up Table,简称“LUT”)中建立索引,并将该存储器读请求TLP中的Requester ID1的中的Bus Number1和Device Number1以该索引对应的表项内容保存在LUT中,再将该存储器读请求TLP中的Bus Number1修改为NTB在第二处理器对应的PCIe总线域中的Bus Number2,将Device Number1修改为LTU的索引值,Function Number1保持不变,然后将修改后的存储器读请求TLP发送给第二处理器。 In the prior art, a method for converting the Requester ID by the NTB is provided. Taking the memory read request TLP as an example, when the NTB receives the memory read request TLP sent by the first processor, the lookup table is first displayed (Look- The Up Table, referred to as "LUT" for example, establishes an index, and stores the Bus Number1 and Device Number1 in the Requester ID1 in the memory read request TLP in the LUT with the contents of the entry corresponding to the index, and then the memory read request. The Bus Number1 in the TLP is modified to be the Bus Number2 of the NTB in the PCIe bus domain corresponding to the second processor, the Device Number1 is changed to the index value of the LTU, the Function Number1 remains unchanged, and then the modified memory read request TLP is sent to Second processor.
当第二处理器中的输入输出存储管理单元(input/output memory management unit,简称“IOMMU”)开启时,会对存储器读请求TLP进行合法性检测,由于现有的NTB对存储器读请求TLP进行转换时,其Requester ID中只有Bus Number采用了NTB在第二PCIe总线域中的Bus Number,而Device Number为LUT的索引值,Function Number为NTB在第一PCIe总线域中的Function Number,在IOMMU查询的页表结构中并不存在这样的Requester ID对应的PCIe设备(说明该读请求TLP不合法),因此,会引起IOMMU错误,例如不支持的请求等,导致IOMMU无法正常工作。When the input/output memory management unit ("IOMMU") in the second processor is turned on, the memory read request TLP is checked for validity, because the existing NTB performs the memory read request TLP. In the conversion, only the Bus Number in the Requester ID uses the Bus Number of the NTB in the second PCIe bus domain, and the Device Number is the index value of the LUT, and the Function Number is the Function Number of the NTB in the first PCIe bus domain, in the IOMMU. The PCIe device corresponding to the Requester ID does not exist in the page table structure of the query (indicating that the read request TLP is invalid), and therefore, an IOMMU error, such as an unsupported request, may be caused, and the IOMMU may not work properly.
发明内容Summary of the invention
本发明实施例提供了一种基于PCIe总线的数据传输方法和装置,可以使IOMMU对请求TLP进行合法性检测时,不会再引起错误,所述技术方案如下:The embodiment of the invention provides a data transmission method and device based on the PCIe bus, which can cause the IOMMU to detect the legality of the requesting TLP without causing any error. The technical solution is as follows:
第一方面,本发明实施例提供了一种基于PCIe总线的数据传输方法,所述方法包括:In a first aspect, an embodiment of the present invention provides a data transmission method based on a PCIe bus, where the method includes:
非透明桥接收请求设备发送的请求事务层包TLP,所述请求TLP包括请求者身份标识Requester ID字段、标识Tag字段和地址字段,所述地址字段用于指示目标设备的地址,所述请求者身份标识Requester ID字段携带所述请求设备的总线号、设备号和功能号,所述请求设备属于第一PCIe总线域,所述目标设备属于第二PCIe总线域,所述第一PCIe总线域和所述第二PCIe总线域通过所述非透明桥连接;The non-transparent bridge receives a request transaction layer packet TLP sent by the requesting device, where the request TLP includes a requester identity identifier Requester ID field, an identifier Tag field, and an address field, where the address field is used to indicate an address of the target device, the requester The identifier Requester ID field carries a bus number, a device number, and a function number of the requesting device, the requesting device belongs to a first PCIe bus domain, the target device belongs to a second PCIe bus domain, and the first PCIe bus domain and The second PCIe bus domain is connected by the non-transparent bridge;
分别采用所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述请求TLP中的所述请求设备的总线号、设备号和功能号,得到替换后的请求TLP;Replacing the bus number, device number, and function number of the requesting device in the request TLP with the bus number, device number, and function number of the non-transparent bridge in the second PCIe bus domain, respectively, and replacing Request TLP;
将所述替换后的请求TLP发送给所述目标设备。Sending the replaced request TLP to the target device.
进一步地,当所述请求TLP为读请求TLP时,在所述分别采用所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述TLP中的所述请求设备的总线号、设备号和功能号之前,所述方法还包括:Further, when the request TLP is a read request TLP, replace the location in the TLP with the bus number, device number, and function number of the non-transparent bridge in the second PCIe bus domain, respectively. Before the bus number, device number, and function number of the device are requested, the method further includes:
保存所述读请求TLP中携带的标识Tag字段的值、所述请求设备的总线号、设备号和功能号与所述替换后的读请求TLP在所述第二PCIe总线域中的标识Tag字段的值的映射关系。And saving a value of the identifier Tag field carried in the read request TLP, a bus number of the requesting device, a device number and a function number, and an identifier Tag field of the replaced read request TLP in the second PCIe bus domain The mapping of the values.
可选地,当所述TLP为读请求TLP时,所述方法还包括:Optionally, when the TLP is a read request TLP, the method further includes:
接收所述目标设备发送的读完成TLP,所述读完成TLP包括请求者身份标 识Requester ID字段、完成者身份标识Completer ID字段和标识Tag字段,所述完成者身份标识Completer ID字段携带所述目标设备的总线号、设备号和功能号;Receiving a read completion TLP sent by the target device, where the read completion TLP includes a requester identity identifier a Requester ID field, a Completer ID field, and an Identity Tag field, where the Completer ID field carries a bus number, a device number, and a function number of the target device;
采用所述读完成TLP中的标识Tag字段的值,根据所述映射关系,得到所述请求设备的总线号、设备号和功能号以及所述读请求TLP中携带的标识Tag字段的值;And using the value of the identifier Tag field in the read completion TLP, according to the mapping relationship, obtaining a bus number, a device number, a function number of the requesting device, and a value of an identifier Tag field carried in the read request TLP;
采用得到的所述请求设备的总线号、设备号和功能号,替换所述读完成TLP中的请求者身份标识Requester ID字段中的总线号、设备号和功能号;Replacing the bus number, device number, and function number in the Requester ID field of the requester identity in the read completion TLP by using the obtained bus number, device number, and function number of the requesting device;
采用得到的所述读请求TLP中携带的标识Tag字段的值,替换所述读完成TLP中的标识Tag字段的值;And replacing, by using the value of the identifier Tag field carried in the read request TLP, a value of the identifier Tag field in the read completion TLP;
采用所述非透明桥在所述第一PCIe总线域中的总线号、设备号和功能号,替换所述完成者身份标识Completer ID字段中的总线号、设备号和功能号,得到替换后的读完成TLP;Replacing the bus number, device number, and function number in the Completer Identity Completer ID field with the bus number, device number, and function number of the non-transparent bridge in the first PCIe bus domain, and obtaining the replaced Read completed TLP;
将所述替换后的读完成TLP发送给所述请求设备。Transmitting the replaced read completion TLP to the requesting device.
进一步地,所述保存所述读请求TLP中携带的标识Tag字段的值、所述请求设备的总线号、设备号和功能号与所述替换后的读请求TLP在所述第二PCIe总线域中的标识Tag字段的值的映射关系,包括:Further, the saving the value of the identifier Tag field carried in the read request TLP, the bus number of the requesting device, the device number and the function number, and the replaced read request TLP in the second PCIe bus domain The mapping relationship between the values of the tagged Tag fields, including:
将所述替换后的读请求TLP在所述第二PCIe总线域中的标识Tag字段的值保存为显示查找表LUT中的索引;Saving the value of the identifier Tag field of the replaced read request TLP in the second PCIe bus domain as an index in the display lookup table LUT;
将所述读请求TLP中携带的标识Tag字段的值、所述请求设备的总线号、设备号和功能号保存为所述索引对应的表项。The value of the identifier Tag field carried in the read request TLP, the bus number of the requesting device, the device number, and the function number are saved as an entry corresponding to the index.
进一步地,所述分别采用所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述请求TLP中的所述请求设备的总线号、设备号和功能号,包括:Further, the bus number, device number, and function number of the requesting device in the request TLP are replaced by a bus number, a device number, and a function number of the non-transparent bridge in the second PCIe bus domain, respectively. Number, including:
从PCIe配置空间寄存器中读取所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号;Reading a bus number, a device number, and a function number of the non-transparent bridge in the second PCIe bus domain from a PCIe configuration space register;
采用读取到的所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述请求设备发送的所述请求TLP中的所述请求设备的总线号、设备号和功能号。Replacing the bus number, the device number, and the function number of the non-transparent bridge in the second PCIe bus domain, replacing the bus number of the requesting device in the request TLP sent by the requesting device, Device number and function number.
第二方面,本发明实施例提供了一种基于PCIe总线的数据传输装置,所述装置包括: In a second aspect, an embodiment of the present invention provides a data transmission apparatus based on a PCIe bus, where the apparatus includes:
第一接收模块,用于非透明桥接收请求设备发送的请求事务层包TLP,所述请求TLP包括请求者身份标识Requester ID字段、标识Tag字段和地址字段,所述地址字段用于指示目标设备的地址,所述请求者身份标识Requester ID字段携带所述请求设备的总线号、设备号和功能号,所述请求设备属于第一PCIe总线域,所述目标设备属于第二PCIe总线域,所述第一PCIe总线域和所述第二PCIe总线域通过所述非透明桥连接;a first receiving module, configured to receive, by the non-transparent bridge, a request transaction layer packet TLP sent by the requesting device, where the request TLP includes a requester identity identifier Requester ID field, an identifier Tag field, and an address field, where the address field is used to indicate the target device Address, the requester ID field carries a bus number, a device number, and a function number of the requesting device, the requesting device belongs to a first PCIe bus domain, and the target device belongs to a second PCIe bus domain. The first PCIe bus domain and the second PCIe bus domain are connected by the non-transparent bridge;
第一替换模块,用于分别采用所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述请求TLP中的所述请求设备的总线号、设备号和功能号,得到替换后的请求TLP;a first replacement module, configured to replace, by using a bus number, a device number, and a function number of the non-transparent bridge in the second PCIe bus domain, a bus number and a device number of the requesting device in the request TLP And the function number to get the replaced request TLP;
发送模块,用于将所述替换后的请求TLP发送给所述目标设备。And a sending module, configured to send the replaced request TLP to the target device.
当所述请求TLP为读请求TLP时,所述装置还包括:When the requesting TLP is a read request TLP, the device further includes:
保存模块,用于保存所述读请求TLP中携带的标识Tag字段的值、所述请求设备的总线号、设备号和功能号与所述替换后的读请求TLP在所述第二PCIe总线域中的标识Tag字段的值的映射关系。a saving module, configured to save a value of an identifier Tag field carried in the read request TLP, a bus number, a device number, and a function number of the requesting device, and the replaced read request TLP in the second PCIe bus domain The mapping relationship between the values of the tagged Tag field.
可选地,当所述TLP为读请求TLP时,所述接收模块还用于接收所述目标设备发送的读完成TLP,所述读完成TLP包括请求者身份标识Requester ID字段、完成者身份标识Completer ID字段和标识Tag字段,所述完成者身份标识Completer ID字段携带所述目标设备的总线号、设备号和功能号;Optionally, when the TLP is a read request TLP, the receiving module is further configured to receive a read completion TLP sent by the target device, where the read completion TLP includes a requester identity identifier, a Requester ID field, and a completion identity. a Completer ID field and an identifier Tag field, where the Completer ID field carries a bus number, a device number, and a function number of the target device;
所述装置还包括:获取模块,用于采用所述读完成TLP中的标识Tag字段的值,根据所述映射关系,得到所述请求设备的总线号、设备号和功能号以及所述读请求TLP中携带的标识Tag字段的值;The device further includes: an obtaining module, configured to adopt a value of the identifier Tag field in the read completion TLP, and obtain a bus number, a device number, a function number, and the read request of the requesting device according to the mapping relationship The value of the tagged Tag field carried in the TLP;
第二替换模块,用于采用得到的所述请求设备的总线号、设备号和功能号,替换所述读完成TLP中的请求者身份标识Requester ID字段中的总线号、设备号和功能号;a second replacement module, configured to replace, by using the obtained bus number, device number, and function number of the requesting device, a bus number, a device number, and a function number in a Requester ID field of the requester identity in the read completion TLP;
所述第二替换模块还用于采用得到的所述读请求TLP中携带的标识Tag字段的值,替换所述读完成TLP中的标识Tag字段的值;The second replacement module is further configured to replace, by using the value of the identifier Tag field carried in the read request TLP, a value of the identifier Tag field in the read completion TLP;
第三替换模块,用于采用所述非透明桥在所述第一PCIe总线域中的总线号、设备号和功能号,替换所述完成者身份标识Completer ID字段中的总线号、设备号和功能号,得到替换后的读完成TLP;a third replacement module, configured to replace, by using a bus number, a device number, and a function number of the non-transparent bridge in the first PCIe bus domain, a bus number, a device number, and a number in the Completer ID field of the Completer Identity Function number, get the read completion TLP after replacement;
所述发送模块还用于将所述替换后的读完成TLP发送给所述请求设备。The sending module is further configured to send the replaced read completion TLP to the requesting device.
进一步地,所述保存模块,包括: Further, the saving module includes:
第一索引单元,用于将所述替换后的读请求TLP在所述第二PCIe总线域中的标识Tag字段的值保存为显示查找表LUT中的索引;a first indexing unit, configured to save, by using the value of the identifier Tag field of the replaced read request TLP in the second PCIe bus domain, an index in the display lookup table LUT;
第二索引单元,用于将所述读请求TLP中携带的标识Tag字段的值、所述请求设备的总线号、设备号和功能号保存为所述索引对应的表项。The second indexing unit is configured to save the value of the identifier Tag field carried in the read request TLP, the bus number of the requesting device, the device number, and the function number as an entry corresponding to the index.
进一步地,所述第一替换模块,包括:Further, the first replacement module includes:
读取单元,用于从PCIe配置空间寄存器中读取所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号;a reading unit, configured to read, from a PCIe configuration space register, a bus number, a device number, and a function number of the non-transparent bridge in the second PCIe bus domain;
替换单元,用于采用读取到的所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述请求设备发送的所述请求TLP中的所述请求设备的总线号、设备号和功能号。a replacement unit, configured to replace, by using the read bus number, device number, and function number of the non-transparent bridge in the second PCIe bus domain, the request in the request TLP sent by the requesting device The bus number, device number, and function number of the device.
本发明实施例提供的技术方案带来的有益效果是:The beneficial effects brought by the technical solutions provided by the embodiments of the present invention are:
通过非透明桥将请求TLP中的请求者身份标识Requester ID字段的总线号、设备号和功能号分别替换为非透明桥在第二PCIe总线域中的总线号、设备号和功能号,使非透明桥发送给目标设备的请求TLP中的请求者身份标识Requester ID对应的PCIe设备为PCIe总线初始化中已经存在于页表中的设备,当IOMMU开启,对请求TLP进行合法性检测时,不会再引起错误,IOMMU可以正常工作。The bus number, device number, and function number of the Requester ID field of the requester ID in the request TLP are replaced by the non-transparent bridge to the bus number, device number, and function number of the non-transparent bridge in the second PCIe bus domain, respectively. The requester ID in the request TLP sent by the transparent bridge to the target device is the device that is already in the page table in the PCIe bus initialization. When the IOMMU is enabled, the validity of the request TLP is not detected. Then cause an error, IOMMU can work normally.
附图说明DRAWINGS
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below.
图1是基于PCIe总线的数据传输架构的示意图;1 is a schematic diagram of a data transmission architecture based on a PCIe bus;
图2是本发明实施例一提供的一种基于PCIe总线的数据传输方法的流程图;2 is a flowchart of a data transmission method based on a PCIe bus according to Embodiment 1 of the present invention;
图3是本发明实施例二提供的一种基于PCIe总线的数据传输方法的流程图;3 is a flowchart of a PCIe bus-based data transmission method according to Embodiment 2 of the present invention;
图4是本发明实施例三提供的一种基于PCIe总线的数据传输方法的流程图;4 is a flowchart of a PCIe bus-based data transmission method according to Embodiment 3 of the present invention;
图5是本发明实施例四提供的一种基于PCIe总线的数据传输装置的结构示意图;FIG. 5 is a schematic structural diagram of a data transmission apparatus based on a PCIe bus according to Embodiment 4 of the present invention; FIG.
图6是本发明实施例五提供的一种基于PCIe总线的数据传输装置的结构示意图。 FIG. 6 is a schematic structural diagram of a data transmission apparatus based on a PCIe bus according to Embodiment 5 of the present invention.
具体实施方式detailed description
为使本发明的目的、技术方案和优点更加清楚,下面将结合附图对本发明实施方式作进一步地详细描述。The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
为了更好地理解本发明的实施例,下面结合图1,以包括两个采用NTB连接的处理器***的计算机***为例,对本发明中涉及的基于PCIe总线的数据传输架构做简要地介绍。For a better understanding of the embodiments of the present invention, the PCIe bus-based data transmission architecture involved in the present invention will be briefly described below with reference to FIG. 1 as an example of a computer system including two NTB-connected processor systems.
参见图1,该计算机***包括中央处理器1(Center Prossecing Unit,简称“CPU”)、CPU2、PCIe总线3、PCIe总线4、主存储器6、主存储器7、PCIe交换机8和PCIe交换机9,其中,CPU1中集成有RC12,CPU2中集成有RC13,RC12与主存储器6连接,PCIe总线3的两端分别与RC12和PCIe交换机8连接,RC13与主存储器7连接,PCIe总线4的两端分别与RC13和PCIe交换机9连接,PCIe交换机8上连接有PCIe设备82和PCIe设备83,PCIe交换机9上连接有PCIe设备91,且PCIe交换机8和PCIe交换机9中分别集成有NTB,PCIe交换机8和PCIe交换机9通过各自的NTB相互连接,RC12、PCIe交换机8、PCIe设备82和PCIe设备83属于PCIe总线域10,RC13、PCIe交换机9和PCIe设备91属于PCIe总线域11。Referring to FIG. 1, the computer system includes a central processing unit 1 (CPU), a CPU 2, a PCIe bus 3, a PCIe bus 4, a main memory 6, a main memory 7, a PCIe switch 8, and a PCIe switch 9, wherein RC12 is integrated in CPU1, RC13 is integrated in CPU2, and RC12 is connected to main memory 6. Both ends of PCIe bus 3 are connected with RC12 and PCIe switch 8, respectively, and RC13 is connected with main memory 7, and both ends of PCIe bus 4 are respectively The RC13 is connected to the PCIe switch 9. The PCIe switch 8 is connected to the PCIe device 82 and the PCIe device 83. The PCIe switch 9 is connected to the PCIe device 91, and the PCIe switch 8 and the PCIe switch 9 are respectively integrated with the NTB, the PCIe switch 8 and the PCIe. The switches 9 are connected to each other through respective NTBs. The RC12, the PCIe switch 8, the PCIe device 82, and the PCIe device 83 belong to the PCIe bus domain 10, and the RC 13, PCIe switch 9, and PCIe device 91 belong to the PCIe bus domain 11.
RC可以包括连接PCIe设备的接口以及IOMMU等。在本发明实施例中,PCIe设备可以为PCIe端点、PCIe桥和PCIe交换机等。The RC may include an interface to connect to the PCIe device, an IOMMU, and the like. In the embodiment of the present invention, the PCIe device may be a PCIe endpoint, a PCIe bridge, a PCIe switch, or the like.
其中,IOMMU的重要组成部分为直接存储器访问(Direct Memory Access,简称“DMA”)重映射。在***初始化的过程中,RC会给每个PCIe设备分别分配总线号(Bus Number)、设备号(Device Number)和功能号(Function Number),且每个PCIe设备的总线号(Bus Number)、设备号(Device Number)和功能号(Function Number)在该PCIe设备所在的PCIe总线域内唯一。因此,在***初始化时,RC可以将每个PCIe设备的总线号(Bus Number)、设备号(Device Number)和功能号(Function Number)作为索引,在DMA重映射的输入输出(In\Out,简称“IO”)页表中保存每个PCIe设备能够访问的主存空间的地址。Among them, an important part of IOMMU is Direct Memory Access ("DMA") remapping. During the system initialization process, the RC will assign a Bus Number, a Device Number, and a Function Number to each PCIe device, and the bus number (Bus Number) of each PCIe device. The Device Number and Function Number are unique within the PCIe bus domain in which the PCIe device resides. Therefore, during system initialization, the RC can use the bus number (Bus Number), device number (Device Number), and function number (Function Number) of each PCIe device as an index, in the DMA remapping input and output (In\Out, The address of the main memory space that each PCIe device can access is stored in the page table referred to as "IO" for short.
当PCIe设备访问主存储器的空间时,IOMMU会根据访问请求中的Bus Number、Device Number和Function Number查找DMA重映射的IO页表,以确定是否存在与该PCIe设备对应的IO页表的页表项,如果不存在,则上报给相应的中央处理器,由中央处理器向该PCIe设备发送IOMMU错误,如果存 在时,IOMMU将查找结果转换为PCIe设备所要访问的主存储器的物理地址。When the PCIe device accesses the space of the main memory, the IOMMU searches the IO remapped IO page table according to the Bus Number, Device Number, and Function Number in the access request to determine whether there is a page table of the IO page table corresponding to the PCIe device. If not present, it is reported to the corresponding central processor, and the central processor sends an IOMMU error to the PCIe device, if At the time, the IOMMU translates the lookup result to the physical address of the main memory that the PCIe device is to access.
以上计算机***的结构仅为举例,在其他的实现方式中,RC也可以不集成在CPU中,而是作为一个单独的芯片或者集成在北桥芯片中。The structure of the above computer system is only an example. In other implementations, the RC may not be integrated in the CPU, but may be implemented as a separate chip or integrated in the Northbridge chip.
实施例一Embodiment 1
本发明实施例提供了一种基于PCIe总线的数据传输方法,适用于前述计算机***,该方法的执行主体为NTB。参见图2,该方法包括:The embodiment of the invention provides a data transmission method based on the PCIe bus, which is applicable to the foregoing computer system, and the execution body of the method is NTB. Referring to Figure 2, the method includes:
步骤201:非透明桥接收请求设备发送的请求TLP,该请求TLP包括请求者身份标识Requester ID字段、标识Tag字段和地址字段,地址字段用于指示目标设备的地址,请求者身份标识Requester ID字段携带请求设备的总线号、设备号和功能号,请求设备属于第一PCIe总线域,目标设备属于第二PCIe总线域,第一PCIe总线域和第二PCIe总线域通过所述非透明桥连接。Step 201: The non-transparent bridge receives the request TLP sent by the requesting device, where the request TLP includes a requester identity identifier Requester ID field, an identifier Tag field, and an address field, where the address field is used to indicate the address of the target device, and the requester identity identifier Requester ID field Carrying the bus number, device number and function number of the requesting device, the requesting device belongs to the first PCIe bus domain, the target device belongs to the second PCIe bus domain, and the first PCIe bus domain and the second PCIe bus domain are connected by the non-transparent bridge.
该请求设备可以为RC或PCIe设备,例如,请求设备可以为图1中的RC12,也可以是PCIe设备81、PCIe设备82和PCIe交换机8,第一PCIe总线域为PCIe总线域10。该目标设备可以为RC或PCIe设备。例如,目标设备可以是图1中的RC13,也可以是PCIe设备91,第二PCIe总线域为PCIe总线域11。The requesting device may be an RC or PCIe device. For example, the requesting device may be the RC 12 in FIG. 1 or the PCIe device 81, the PCIe device 82, and the PCIe switch 8. The first PCIe bus domain is the PCIe bus domain 10. The target device can be an RC or PCIe device. For example, the target device may be the RC 13 in FIG. 1 or the PCIe device 91, and the second PCIe bus domain is the PCIe bus domain 11.
步骤202:分别采用非透明桥在第二PCIe总线域中的总线号、设备号和功能号,替换请求TLP中的请求设备的总线号、设备号和功能号,得到替换后的请求TLP。Step 202: Replace the bus number, device number, and function number of the requesting device in the request TLP with the bus number, the device number, and the function number of the non-transparent bridge in the second PCIe bus domain, respectively, to obtain the replaced request TLP.
步骤203:将替换后的请求TLP发送给目标设备。Step 203: Send the replaced request TLP to the target device.
其中,请求TLP可以是读请求TLP或写请求TLP。实现时,该NTB可以集成在PCIe交换机中,也可以不集成在PCIe交换机中,而是作为一个单独的芯片,或者集成在RC中。The request TLP may be a read request TLP or a write request TLP. When implemented, the NTB can be integrated into the PCIe switch or not integrated into the PCIe switch, but as a separate chip or integrated into the RC.
本发明实施例通过非透明桥将请求TLP中的请求者身份标识Requester ID字段的总线号、设备号和功能号分别替换为非透明桥在第二PCIe总线域中的总线号、设备号和功能号,使非透明桥发送给目标设备的请求TLP中的请求者身份标识Requester ID对应的***设备为PCIe总线初始化中已经存在于页表中的设备,当IOMMU开启,对请求TLP进行合法性检测时,不会再引起错误,IOMMU可以正常工作。In the embodiment of the present invention, the bus number, the device number, and the function number of the Requester ID field in the Requester ID field in the request TLP are replaced by the non-transparent bridge to the bus number, device number, and function of the non-transparent bridge in the second PCIe bus domain. No. The requester ID corresponding to the requester ID in the request TLP that sends the non-transparent bridge to the target device is the device that is already in the page table in the PCIe bus initialization. When the IOMMU is enabled, the legality of the request TLP is detected. When it does not cause an error, the IOMMU can work normally.
实施例二 Embodiment 2
本发明实施例提供了一种基于PCIe总线的数据传输方法,适用于前述计算机***,该方法的执行主体为NTB。本实施例中,以请求TLP为读请求TLP 为例,对本发明进行说明,参见图3,方法包括:The embodiment of the invention provides a data transmission method based on the PCIe bus, which is applicable to the foregoing computer system, and the execution body of the method is NTB. In this embodiment, the TLP is requested as a read request TLP. For example, the present invention will be described. Referring to FIG. 3, the method includes:
步骤301:接收请求设备发送的读请求TLP。Step 301: Receive a read request TLP sent by the requesting device.
其中,读请求TLP可以包括请求者身份标识Requester ID字段、标识Tag字段和地址字段,请求者身份标识Requester ID字段携带请求设备在第一PCIe总线域中的总线号(Bus Number)、设备号(Device Number)和功能号(Function Number),第一PCIe总线域为请求设备所在的PCIe总线域。地址字段用于指示目标设备的地址。The read request TLP may include a requester identity requester ID field, an identifier tag field, and an address field, and the requester identity identifier Requester ID field carries a bus number (Bus Number) and a device number of the requesting device in the first PCIe bus domain ( Device Number) and Function Number. The first PCIe bus domain is the PCIe bus domain where the requesting device is located. The address field is used to indicate the address of the target device.
实现时,读请求TLP可以是存储器读请求TLP和输入输出I/O读请求TLP。In implementation, the read request TLP may be a memory read request TLP and an input/output I/O read request TLP.
在PCIe总线标准中,请求者身份标识Requester ID和标识Tag字段合称为Transaction ID。同一时间段内,Transaction ID字段必须是唯一的,即该时间段内一个PCIe总线域中不存在两个或两个以上的读请求TLP中的Transaction ID字段的值完全相同。在这同一时间段内,同一个PCIe设备发送的多个TLP(例如读请求TLP),其Requester ID字段是相同的,因此,可以将Tag字段取为不同的值,以区分同一时间段内,同一个PCIe设备发送的各个TLP。也就是说,同一PCIe设备发送的请求TLP中的Tag字段的值唯一,所以可以采用NTB对接收的某个请求TLP进行转换后得到的请求TLP的标识Tag字段的值作为索引,例如图1中的PCIe交换机8中的NTB将从RC12接收的读请求TLP中的Tag字段替换为第二PCIe总线域中的Tag字段,并将替换后的读请求TLP在第二PCIe总线域中的Tag字段的值作为索引。In the PCIe bus standard, the requester identity requester ID and the identity tag field are collectively referred to as a Transaction ID. During the same period of time, the Transaction ID field must be unique, that is, the value of the Transaction ID field in the TLP that does not exist in a PCIe bus domain does not exist in the same period. During the same time period, multiple TLPs (for example, read request TLPs) sent by the same PCIe device have the same Requester ID field. Therefore, the Tag field can be set to a different value to distinguish between the same time period. Each TLP sent by the same PCIe device. That is, the value of the Tag field in the request TLP sent by the same PCIe device is unique. Therefore, the value of the Tag field of the request TLP obtained by converting the received request TLP by the NTB may be used as an index, for example, in FIG. The NTB in the PCIe switch 8 replaces the Tag field in the read request TLP received from the RC 12 with the Tag field in the second PCIe bus domain, and replaces the read read request TLP in the Tag field of the second PCIe bus domain. The value is used as an index.
实现时,请求设备可以为RC或PCIe设备。例如,参见图1,请求设备为CPU1中的RC12,第一PCIe总线域为PCIe总线域10。其中,PCIe设备可以是PCIe端点(Endpoint)(例如网卡、显卡等)、PCIe交换机和PCI桥片等。PCI桥片可以将PCIe总线转换为PCI总线或者PCI-X总线之后,连接PCI设备或者PCI-X设备。When implemented, the requesting device can be an RC or PCIe device. For example, referring to FIG. 1, the requesting device is the RC 12 in the CPU 1, and the first PCIe bus domain is the PCIe bus domain 10. The PCIe device may be a PCIe endpoint (such as a network card, a graphics card, etc.), a PCIe switch, and a PCI bridge. The PCI bridge can be connected to a PCI device or a PCI-X device after converting the PCIe bus to a PCI bus or a PCI-X bus.
在步骤301之前,还可以包括请求设备将自己在第一PCIe总线域中的总线号、设备号和功能号写入读请求TLP的请求者身份标识Requester ID字段中。Prior to step 301, the requesting device may also include the bus number, device number, and function number of the requesting device in the first PCIe bus domain being written into the requester ID Requester ID field of the read request TLP.
具体地,PCIe总线规定,在一个PCIe总线域空间中,最多只能有256条PCIe总线,因此,在一个TLP中,总线号(Bus Number)字段的长度为8比特(bit),在每一条PCIe总线中最多包含32个设备,因此,一个TLP中的设备号(Device Number)字段的长度为5比特,每一个PCIe设备中最多包含8个功能,因此,一个TLP中的功能号(Function Number)字段的长度为5比 特。由于PCIe总线使用“端对端”的连接方式,在每条PCIe总线上只能连接一个下游PCIe设备,当该下游PCIe设备为PCIe端点时,其设备号(Device Number)的值通常为0,当下游PCIe设备为PCIe交换机(Switch)时,可以通过PCIe交换机在一条PCIe总线上连接多个PCIe设备,则多个PCIe设备的设备号(Device Number)可以取不同的值。Specifically, the PCIe bus stipulates that there can be at most 256 PCIe buses in a PCIe bus domain space. Therefore, in a TLP, the bus number (Bus Number) field has a length of 8 bits (bit), in each The PCIe bus contains up to 32 devices. Therefore, the Device Number field in a TLP is 5 bits long, and each PCIe device contains up to 8 functions. Therefore, the function number in one TLP (Function Number) ) the length of the field is 5 special. Since the PCIe bus uses an "end-to-end" connection, only one downstream PCIe device can be connected to each PCIe bus. When the downstream PCIe device is a PCIe endpoint, the device number is usually 0. When the downstream PCIe device is a PCIe switch (Switch), multiple PCIe devices can be connected to one PCIe bus through the PCIe switch. The device numbers of multiple PCIe devices can take different values.
在本实施例中,当***通电后,CPU可以通过枚举的方式(例如采用深度优先搜索算法)对PCIe总线域中的总线号进行初始化,并将初始化结果保存在PCIe设备的配置空间寄存器中,CPU再对各个总线号对应的PCIe设备的设备号和功能号进行初始化,然后CPU再将各组总线号、设备号和功能号作为DMA重映射的IO页表的索引进行保存。In this embodiment, after the system is powered on, the CPU can initialize the bus number in the PCIe bus domain by enumerating (for example, using a depth-first search algorithm), and save the initialization result in the configuration space register of the PCIe device. The CPU then initializes the device number and function number of the PCIe device corresponding to each bus number, and then the CPU saves each group bus number, device number, and function number as an index of the DMA remapped IO page table.
需要说明的是,一个完整的TLP一般可以包括一个或者多个TLP Prefix(前缀)、TLP头和Data Payload(数据有效负载)等,而请求者身份标识Requester ID字段、标识Tag字段和地址字段均携带于TLP头中,此为现有技术,在此省略详细描述。It should be noted that a complete TLP may generally include one or more TLP Prefix, TLP header, and Data Payload, and the requester ID, Request Tag ID field, ID Tag field, and address field are all specified. It is carried in the TLP header, which is a prior art, and a detailed description is omitted here.
步骤302:保存读请求TLP中携带的标识Tag字段的值、请求设备的总线号、设备号和功能号与替换后的读请求TLP在第二PCIe总线域中的标识Tag字段的值的映射关系。Step 302: The mapping between the value of the identifier Tag field carried in the read request TLP, the bus number of the requesting device, the device number and the function number, and the value of the identifier Tag of the replaced read request TLP in the second PCIe bus domain is saved. .
该读请求TLP在第二PCIe总线域中的标识Tag字段的值由NTB分配。实现时,各个读请求TLP的标识Tag字段的值可以采用顺序编号的方式依次产生。替换后的读请求TLP在第二PCIe总线域中的标识Tag字段的值与请求设备的总线号、设备号和功能号以及读请求TLP中携带的标识Tag字段的值(即读请求TLP在第一PCIe总线域中的标识Tag字段的值)的映射关系可以采用表格的形式(例如,LUT等)进行保存。The value of the identification Tag field of the read request TLP in the second PCIe bus domain is allocated by the NTB. In implementation, the value of the identifier Tag field of each read request TLP may be sequentially generated by sequential numbering. The value of the identifier Tag field of the read request TLP in the second PCIe bus domain is the value of the bus number, the device number and the function number of the requesting device, and the value of the identifier Tag field carried in the read request TLP (ie, the read request TLP is in the The mapping relationship of the value of the tag Tag field in a PCIe bus domain can be saved in the form of a table (for example, LUT, etc.).
其中,保存读请求TLP中携带的Tag字段的值、请求设备的总线号、设备号和功能号与替换后的读请求TLP在第二PCIe总线域中的标识Tag字段的值的映射关系,可以包括:The mapping between the value of the Tag field carried in the read request TLP, the bus number of the requesting device, the device number and the function number, and the value of the tag T of the replaced read request TLP in the second PCIe bus domain may be include:
将替换后的读请求TLP在第二PCIe总线域中的Tag字段的值保存为显示查找表LUT中的索引;Saving the value of the Tag field of the replaced read request TLP in the second PCIe bus domain as an index in the display lookup table LUT;
将读请求TLP中携带的标识Tag字段的值、请求设备的总线号、设备号和功能号保存为索引对应的表项。例如,一读请求TLP替换前后的标识Tag字段的值为分别为6和5、在第一PCIe总线域中的总线号为2、设备号为1、功能 号为4,则可以建立索引为Tag值5,显示查找表LUT中的对应的表项是Tag值为6、总线号为2、设备号为1、功能号为4的记录。The value of the identifier Tag field carried in the read request TLP, the bus number of the requesting device, the device number, and the function number are saved as entries corresponding to the index. For example, the value of the tag Tag field before and after the read request TLP replacement is 6 and 5 respectively, and the bus number 2 in the first PCIe bus domain is 2, and the device number is 1. If the number is 4, the index can be set to Tag value 5. The corresponding entry in the lookup table LUT is a record with a tag value of 6, a bus number of 2, a device number of 1, and a function number of 4.
其中,该显示查找表LUT可以由NTB建立和维护。容易理解地,标识Tag字段的长度决定了NTB能够暂存多少个同类型的TLP,如果标识Tag字段长度为5,NTB能够暂存32个不同类型的TLP;如果PCIe设备使能了扩展标识(Extended Tag)位,则标识Tag字段可以由8位组成,此时NTB能够暂存256个不同类型的TLP。当标识Tag字段的长度为8比特时,该显示查找表LUT可以包括256个表项。The display lookup table LUT can be established and maintained by the NTB. It is easy to understand that the length of the tagged Tag field determines how many TLPs of the same type can be temporarily stored by the NTB. If the Tag field length is 5, the NTB can temporarily store 32 different types of TLPs; if the PCIe device enables the extended tag ( The Extended Tag bit indicates that the Tag field can be composed of 8 bits. At this time, the NTB can temporarily store 256 different types of TLPs. When the length of the identification Tag field is 8 bits, the display lookup table LUT may include 256 entries.
步骤303:分别采用非透明桥在第二PCIe总线域中的总线号、设备号和功能号,替换读请求TLP中的请求设备的总线号、设备号和功能号,得到替换后的读请求TLP。Step 303: Replace the bus number, device number, and function number of the requesting device in the read request TLP with the bus number, device number, and function number of the non-transparent bridge in the second PCIe bus domain, respectively, to obtain the read request TLP after replacement. .
实现时,分别采用非透明桥在第二PCIe总线域中的总线号、设备号和功能号,替换读请求TLP中的请求设备的总线号、设备号和功能号,可以包括:In the implementation, the bus number, the device number, and the function number of the non-transparent bridge in the second PCIe bus domain are replaced by the bus number, the device number, and the function number of the requesting device in the read request TLP, which may include:
从PCIe配置空间寄存器中读取非透明桥在第二PCIe总线域中的总线号、设备号和功能号;其中,PCIe配置空间寄存器从数字电路来讲通常就是一组D触发器构成,能存储01比特,不同01比特组成的字段对应不同的总线号、设备号和功能号。PCIe配置空间寄存器的不同的地址空间分配给不同的PCIe总线域。The bus number, device number, and function number of the non-transparent bridge in the second PCIe bus domain are read from the PCIe configuration space register; wherein the PCIe configuration space register is usually a set of D flip-flops and can be stored from the digital circuit. 01 bit, the field consisting of different 01 bits corresponds to different bus number, device number and function number. The different address spaces of the PCIe configuration space registers are allocated to different PCIe bus domains.
采用读取到的非透明桥在第二PCIe总线域中的总线号、设备号和功能号,替换读请求TLP中的请求设备的总线号、设备号和功能号。The bus number, device number, and function number of the requesting device in the read request TLP are replaced by the bus number, device number, and function number of the read non-transparent bridge in the second PCIe bus domain.
步骤304:将替换后的读请求TLP发送给目标设备。Step 304: Send the replaced read request TLP to the target device.
实现时,NTB是采用基于地址(Address)的路由的将读请求TLP发送给目标设备。该目标设备的路由信息携带于读请求TLP的TLP头的地址Address字段中。In implementation, the NTB sends the read request TLP to the target device using an address based route. The routing information of the target device is carried in the address Address field of the TLP header of the read request TLP.
当目标设备接收到读请求TLP时,IOMMU会根据读请求TLP中的Bus Number、Device Number和Function Number(即NTB在第二PCIe总线域中的总线号、设备号和功能号)查询DMA重映射的IO页表,以检测该读请求TLP的合法性。由于NTB在第二PCIe总线域中的总线号、设备号和功能号在***初始化时,已经分配,所以IOMMU对该读请求TLP检测结果为合法。When the target device receives the read request TLP, the IOMMU queries the DMA remapping according to the Bus Number, Device Number, and Function Number in the read request TLP (ie, the bus number, device number, and function number of the NTB in the second PCIe bus domain). The IO page table to detect the legitimacy of the read request TLP. Since the bus number, device number, and function number of the NTB in the second PCIe bus domain have been allocated at the time of system initialization, the IOMMU detects that the TLP detection result is legal.
步骤305:接收目标设备发送的读完成TLP。Step 305: Receive a read completion TLP sent by the target device.
其中,读完成TLP可以包括请求者身份标识Requester ID字段、完成者身 份标识Completer ID字段和标识Tag字段。该完成者身份标识Completer ID字段携带目标设备在第二PCIe总线域中的总线号、设备号和功能号。读完成TLP中的Tag字段的值仍为替换后的读请求TLP在第二PCIe总线域中的Tag字段的值。The read completion TLP may include a requester identity identifier, a Requester ID field, and a completion body. The identity identifies the Completer ID field and the identity Tag field. The Completer ID Completeer ID field carries the bus number, device number, and function number of the target device in the second PCIe bus domain. The value of the Tag field in the Read Complete TLP is still the value of the Tag field of the Replaced Read Request TLP in the second PCIe bus domain.
实现时,该读完成TLP还携带有读请求TLP所请求访问的数据,该数据可以承载于读请求TLP的数据有效负载(Data Payload)字段中。In implementation, the read completion TLP also carries data requested by the read request TLP, which may be carried in the Data Payload field of the read request TLP.
需要说明的是,完成者身份标识Completer ID字段也携带于TLP头中,此为现有技术,在此省略详细描述。It should be noted that the Completer ID field is also carried in the TLP header. This is a prior art, and a detailed description is omitted here.
步骤306:采用读完成TLP中的Tag字段的值,查找LUT,得到请求设备在第一PCIe总线域中的的总线号、设备号和功能号以及读请求TLP中携带的标识Tag字段的值。Step 306: The value of the Tag field in the TLP is read to find the LUT, and the bus number, the device number, and the function number of the requesting device in the first PCIe bus domain and the value of the identifier Tag field carried in the read request TLP are obtained.
读完成TLP中的Tag字段的值仍为替换后的读请求TLP在第二PCIe总线域中的Tag字段的值,所以可以直接用读完成TLP中的Tag字段的值查找LUT表。The value of the Tag field in the read completion TLP is still the value of the Tag field of the replacement read request TLP in the second PCIe bus domain, so the LUT table can be directly searched for by reading the value of the Tag field in the TLP.
步骤307:采用得到的请求设备在第一PCIe总线域中的总线号、设备号和功能号,替换读完成TLP中的请求者身份标识Requester ID字段中的总线号、设备号和功能号。Step 307: Replace the bus number, device number, and function number in the Requester ID field of the requester identity in the read completion TLP by using the bus number, device number, and function number of the obtained requesting device in the first PCIe bus domain.
步骤308:采用得到的读请求TLP中携带的标识Tag字段的值,替换读完成TLP中的标识Tag字段的值。Step 308: Replace the value of the identifier Tag field in the read completion TLP with the value of the identifier Tag field carried in the obtained read request TLP.
需要说明的是,步骤307和308的执行没有先后顺序,也可以先执行步骤308,再执行步骤307。It should be noted that the execution of steps 307 and 308 is not sequential, and step 308 may be performed first, and then step 307 is performed.
步骤309:采用非透明桥在第一PCIe总线域中的总线号、设备号和功能号,替换完成者身份标识Completer ID字段中的总线号、设备号和功能号,得到替换后的读完成TLP。Step 309: Replace the bus number, device number, and function number in the Complete ID of the Completer ID field with the bus number, device number, and function number of the non-transparent bridge in the first PCIe bus domain, and obtain the read completion TLP after replacement. .
其中,非透明桥可以直接从PCIe配置空间寄存器中读取非透明桥在第一PCIe总线域中的总线号、设备号和功能号,并采用读取到的非透明桥在第一PCIe总线域中的总线号、设备号和功能号,替换完成者身份标识Completer ID字段中的总线号、设备号和功能号,得到替换后的读完成TLP。The non-transparent bridge can directly read the bus number, device number and function number of the non-transparent bridge in the first PCIe bus domain from the PCIe configuration space register, and adopt the read non-transparent bridge in the first PCIe bus domain. The bus number, device number, and function number in the replacement, replace the bus number, device number, and function number in the Complete ID field of the Completer ID to obtain the read completion TLP.
步骤310:将转换后的读完成TLP发送给请求设备。Step 310: Send the converted read completion TLP to the requesting device.
在步骤305~310中,目标设备采用基于ID(即读完成TLP中的Bus Number、Device Number和Function Number)的路由方式向NTB发送读完成TLP,NTB 也采用基于ID的路由方式向请求设备发送读完成TLP。In steps 305-310, the target device sends a read completion TLP to the NTB based on the ID (ie, the Bus Number, Device Number, and Function Number in the read completion TLP), NTB. The ID-based routing method is also used to send a read completion TLP to the requesting device.
本发明实施例通过非透明桥将读请求TLP中的请求者身份标识Requester ID字段的总线号、设备号和功能号分别替换为非透明桥在第二PCIe总线域中的总线号、设备号和功能号,使非透明桥发送给目标设备的读请求TLP中的请求者身份标识Requester ID对应的PCIe设备为PCIe总线初始化中已经存在于页表中的***设备,当IOMMU开启,对读请求TLP进行合法性检测时,不会再引起错误,IOMMU可以正常工作。In the embodiment of the present invention, the bus number, the device number, and the function number of the Requester ID field of the requester ID in the read request TLP are replaced by the non-transparent bridge to the bus number, device number, and the non-transparent bridge in the second PCIe bus domain, respectively. The function number is such that the requester ID in the read request TLP sent to the target device by the non-transparent bridge is the peripheral device that is already in the page table in the PCIe bus initialization. When the IOMMU is enabled, the read request TLP is When the legality test is performed, no more errors will be caused and the IOMMU can work normally.
实施例三 Embodiment 3
本发明实施例提供了一种基于PCIe总线的数据传输方法,适用于前述计算机***,该方法的执行主体为NTB。本实施例中的请求TLP为写请求TLP,参见图4,该方法包括:The embodiment of the invention provides a data transmission method based on the PCIe bus, which is applicable to the foregoing computer system, and the execution body of the method is NTB. The request TLP in this embodiment is a write request TLP. Referring to FIG. 4, the method includes:
步骤401:非透明桥接收请求设备发送的写请求TLP,写请求TLP包括请求者身份标识Requester ID字段、标识Tag字段和地址字段,地址字段用于指示目标设备的地址,请求者身份标识Requester ID字段携带请求设备的总线号、设备号和功能号,请求设备属于第一PCIe总线域,目标设备属于第二PCIe总线域,第一PCIe总线域和第二PCIe总线域通过非透明桥连接。Step 401: The non-transparent bridge receives the write request TLP sent by the requesting device, where the write request TLP includes a requester identity identifier Requester ID field, an identifier Tag field, and an address field, where the address field is used to indicate the address of the target device, and the requester identity identifier Requester ID The field carries the bus number, device number and function number of the requesting device, the requesting device belongs to the first PCIe bus domain, the target device belongs to the second PCIe bus domain, and the first PCIe bus domain and the second PCIe bus domain are connected by a non-transparent bridge.
其中,写请求TLP可以是存储器写请求TLP和I/O写请求TLP。The write request TLP may be a memory write request TLP and an I/O write request TLP.
步骤402:分别采用非透明桥在第二PCIe总线域中的总线号、设备号和功能号,替换写请求TLP中的请求设备的总线号、设备号和功能号,得到替换后的写请求TLP。Step 402: Replace the bus number, device number, and function number of the requesting device in the write request TLP with the bus number, device number, and function number of the non-transparent bridge in the second PCIe bus domain, respectively, to obtain the replaced write request TLP. .
步骤403:将替换后的写请求TLP发送给目标设备。Step 403: Send the replaced write request TLP to the target device.
本发明实施例通过非透明桥将写请求TLP中的请求者身份标识Requester ID字段的总线号、设备号和功能号分别替换为非透明桥在第二PCIe总线域中的总线号、设备号和功能号,使非透明桥发送给目标设备的写请求TLP中的请求者身份标识Requester ID对应的PCIe设备为PCIe总线初始化中已经存在于页表中的***设备,当IOMMU开启,对写请求TLP进行合法性检测时,不会再引起错误,IOMMU可以正常工作。In the embodiment of the present invention, the bus number, the device number, and the function number of the requester ID field in the request request ID field in the write request TLP are replaced by the bus number, the device number, and the non-transparent bridge in the second PCIe bus domain, respectively, by the non-transparent bridge. The function number, the requester ID in the write request TLP sent to the target device by the non-transparent bridge, the PCIe device corresponding to the Requester ID is the peripheral device already in the page table in the PCIe bus initialization, when the IOMMU is enabled, the write request TLP When the legality test is performed, no more errors will be caused and the IOMMU can work normally.
实施例四 Embodiment 4
本发明实施例提供了一种基于PCIe总线的数据传输装置,可以用于实现实施例一提供的方法,参见图5,该装置包括:第一接收模块41、第一替换模块42和发送模块43。 The embodiment of the present invention provides a PCIe bus-based data transmission device, which can be used to implement the method provided in Embodiment 1. Referring to FIG. 5, the device includes: a first receiving module 41, a first replacement module 42, and a sending module 43. .
其中,第一接收模块41用于非透明桥接收请求设备发送的请求事务层包TLP,请求TLP包括请求者身份标识Requester ID字段、标识Tag字段和地址字段,地址字段用于指示目标设备的地址,请求者身份标识Requester ID字段携带请求设备的总线号、设备号和功能号,请求设备属于第一PCIe总线域,目标设备属于第二PCIe总线域,第一PCIe总线域和第二PCIe总线域通过非透明桥连接。The first receiving module 41 is configured to receive, by the non-transparent bridge, the request transaction layer packet TLP sent by the requesting device, where the request TLP includes a requester identity identifier Requester ID field, an identifier Tag field, and an address field, where the address field is used to indicate the address of the target device. The requester ID field carries the bus number, device number and function number of the requesting device, the requesting device belongs to the first PCIe bus domain, the target device belongs to the second PCIe bus domain, the first PCIe bus domain and the second PCIe bus domain Connected via a non-transparent bridge.
第一替换模块42用于分别采用非透明桥在第二PCIe总线域中的总线号、设备号和功能号,替换请求TLP中的请求设备的总线号、设备号和功能号,得到替换后的请求TLP。The first replacement module 42 is configured to replace the bus number, the device number, and the function number of the requesting device in the request TLP with the bus number, the device number, and the function number of the non-transparent bridge in the second PCIe bus domain, respectively, and obtain the replaced bus number, device number, and function number. Request a TLP.
发送模块43用于将替换后的请求TLP发送给目标设备。The sending module 43 is configured to send the replaced request TLP to the target device.
本发明实施例通过非透明桥将请求TLP中的请求者身份标识Requester ID字段的总线号、设备号和功能号分别替换为非透明桥在第二PCIe总线域中的总线号、设备号和功能号,使非透明桥发送给目标设备的请求TLP中的请求者身份标识Requester ID对应的PCIe设备为PCIe总线初始化中已经存在于页表中的***设备,当IOMMU开启,对请求TLP进行合法性检测时,不会再引起错误,IOMMU可以正常工作。In the embodiment of the present invention, the bus number, the device number, and the function number of the Requester ID field in the Requester ID field in the request TLP are replaced by the non-transparent bridge to the bus number, device number, and function of the non-transparent bridge in the second PCIe bus domain. Number, the requester ID in the request TLP that sends the non-transparent bridge to the target device. The PCIe device corresponding to the Requester ID is the peripheral device already in the page table in the PCIe bus initialization. When the IOMMU is enabled, the legality of the request TLP is performed. When detected, no more errors will be caused and the IOMMU will work properly.
实施例五Embodiment 5
本发明实施例提供了一种基于PCIe总线的数据传输装置,可以用于实现实施例二或实施例三提供的方法。参见图6,该装置包括:接收模块51、保存模块52、第一替换模块53、发送模块54、获取模块55、第二替换模块56和第三替换模块57。The embodiment of the invention provides a data transmission device based on the PCIe bus, which can be used to implement the method provided in the second embodiment or the third embodiment. Referring to FIG. 6, the apparatus includes: a receiving module 51, a saving module 52, a first replacement module 53, a transmitting module 54, an obtaining module 55, a second replacing module 56, and a third replacing module 57.
其中,第一接收模块51用于非透明桥接收请求设备发送的请求事务层包TLP,请求TLP包括请求者身份标识Requester ID字段、标识Tag字段和地址字段,地址字段用于指示目标设备的地址,请求者身份标识Requester ID字段携带请求设备的总线号、设备号和功能号,请求设备属于第一PCIe总线域,目标设备属于第二PCIe总线域,第一PCIe总线域和第二PCIe总线域通过非透明桥连接。The first receiving module 51 is configured to receive the request transaction layer packet TLP sent by the requesting device by the non-transparent bridge, where the request TLP includes a requester identity identifier Requester ID field, an identifier Tag field, and an address field, where the address field is used to indicate the address of the target device. The requester ID field carries the bus number, device number and function number of the requesting device, the requesting device belongs to the first PCIe bus domain, the target device belongs to the second PCIe bus domain, the first PCIe bus domain and the second PCIe bus domain Connected via a non-transparent bridge.
在本实施例中,请求TLP可以是读请求TLP或写请求TLP。In this embodiment, the request TLP may be a read request TLP or a write request TLP.
当请求TLP为读请求TLP时,保存模块52用于保存读请求TLP中携带的标识Tag字段的值、请求设备的总线号、设备号和功能号与替换后的读请求TLP在第二PCIe总线域中的标识Tag字段的值的映射关系。 When the request TLP is a read request TLP, the saving module 52 is configured to save the value of the identifier Tag field carried in the read request TLP, the bus number of the requesting device, the device number and the function number, and the replaced read request TLP on the second PCIe bus. The mapping of the value of the tagged Tag field in the domain.
第一替换模块53用于分别采用非透明桥在第二PCIe总线域中的总线号、设备号和功能号,替换请求TLP中的请求设备的总线号、设备号和功能号,得到替换后的请求TLP。The first replacement module 53 is configured to replace the bus number, the device number, and the function number of the requesting device in the request TLP with the bus number, the device number, and the function number of the non-transparent bridge in the second PCIe bus domain, respectively, and obtain the replaced Request a TLP.
发送模块54用于将替换后的请求TLP发送给目标设备。The sending module 54 is configured to send the replaced request TLP to the target device.
可选地,当请求TLP为读请求TLP时,接收模块51还用于接收目标设备发送的读完成TLP,读完成TLP包括请求者身份标识Requester ID字段、完成者身份标识Completer ID字段和标识Tag字段,完成者身份标识Completer ID字段携带目标设备的总线号、设备号和功能号;Optionally, when the requesting TLP is a read request TLP, the receiving module 51 is further configured to receive a read completion TLP sent by the target device, where the read completion TLP includes a requester identity identifier Requester ID field, a completer identity identifier Completer ID field, and a tag Tag. Field, Completer ID The Complete ID field carries the bus number, device number, and function number of the target device.
获取模块55用于采用读完成TLP中的标识Tag字段的值,根据映射关系,得到请求设备的总线号、设备号和功能号以及所述读请求TLP中携带的标识Tag字段的值;The obtaining module 55 is configured to use the value of the identifier Tag field in the read completion TLP, and obtain the bus number, the device number, and the function number of the requesting device, and the value of the identifier Tag field carried in the read request TLP according to the mapping relationship;
第二替换模块56用于采用得到的请求设备的总线号、设备号和功能号,替换读完成TLP中的请求者身份标识Requester ID字段中的总线号、设备号和功能号;The second replacement module 56 is configured to replace the bus number, the device number, and the function number in the Requester ID field of the requester identity in the read completion TLP by using the obtained bus number, device number, and function number of the requesting device;
第二替换模块56还用于采用得到的读请求TLP中携带的标识Tag字段的值,替换读完成TLP中的标识Tag字段的值;The second replacement module 56 is further configured to replace the value of the identifier Tag field in the read completion TLP by using the value of the identifier Tag field carried in the obtained read request TLP;
第三替换模块57用于采用非透明桥在第一PCIe总线域中的总线号、设备号和功能号,替换完成者身份标识Completer ID字段中的总线号、设备号和功能号,得到替换后的读完成TLP;The third replacement module 57 is configured to replace the bus number, the device number, and the function number in the Complete ID of the Completer ID field by using the bus number, the device number, and the function number of the non-transparent bridge in the first PCIe bus domain. Reading completion TLP;
发送模块54还用于将替换后的读完成TLP发送给请求设备。The sending module 54 is further configured to send the replaced read completion TLP to the requesting device.
实现时,保存模块52可以包括:When implemented, the saving module 52 can include:
第一索引单元521用于将替换后的读请求TLP在第二PCIe总线域中的标识Tag字段的值保存为显示查找表LUT中的索引;The first indexing unit 521 is configured to save the value of the identifier Tag field of the replaced read request TLP in the second PCIe bus domain as an index in the display lookup table LUT;
第二索引单元522用于将读请求TLP中携带的标识Tag字段的值、请求设备的总线号、设备号和功能号保存为索引对应的表项。The second indexing unit 522 is configured to save the value of the identifier Tag field carried in the read request TLP, the bus number of the requesting device, the device number, and the function number as an entry corresponding to the index.
第一替换模块53可以包括:The first replacement module 53 can include:
读取单元531用于从PCIe配置空间寄存器中读取非透明桥在第二PCIe总线域中的总线号、设备号和功能号;The reading unit 531 is configured to read, from the PCIe configuration space register, a bus number, a device number, and a function number of the non-transparent bridge in the second PCIe bus domain;
替换单元532用于采用读取到的非透明桥在第二PCIe总线域中的总线号、设备号和功能号,替换请求TLP中的请求设备的总线号、设备号和功能号。The replacing unit 532 is configured to replace the bus number, the device number, and the function number of the requesting device in the request TLP by using the bus number, the device number, and the function number of the read non-transparent bridge in the second PCIe bus domain.
本发明实施例通过非透明桥将请求TLP中的请求者身份标识Requester ID 字段的总线号、设备号和功能号分别替换为非透明桥在第二PCIe总线域中的总线号、设备号和功能号,使非透明桥发送给目标设备的请求TLP中的请求者身份标识Requester ID对应的PCIe设备为PCIe总线初始化中已经存在于页表中的***设备,当IOMMU开启,对请求TLP进行合法性检测时,不会再引起错误,IOMMU可以正常工作。The embodiment of the present invention requests the requester ID of the requester ID in the TLP through the non-transparent bridge. The bus number, device number, and function number of the field are replaced with the bus number, device number, and function number of the non-transparent bridge in the second PCIe bus domain, respectively, so that the non-transparent bridge sends the requester identity in the request TLP to the target device. The PCIe device corresponding to the Requester ID is a peripheral device already existing in the page table in the PCIe bus initialization. When the IOMMU is enabled, the legality detection of the requesting TLP does not cause an error, and the IOMMU can work normally.
需要说明的是:上述实施例提供的基于PCIe总线的数据传输装置在进行数据传输时,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将芯片的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。另外,上述实施例提供的基于PCIe总线的数据传输装置与方法实施例属于同一构思,其具体实现过程详见方法实施例,这里不再赘述。It should be noted that the data transmission device based on the PCIe bus provided by the foregoing embodiment is only illustrated by the division of the foregoing functional modules when performing data transmission. In actual applications, the foregoing functions may be assigned differently according to requirements. The function module is completed, that is, the internal structure of the chip is divided into different functional modules to complete all or part of the functions described above. In addition, the data transmission apparatus based on the PCIe bus provided by the foregoing embodiment is the same as the method embodiment, and the specific implementation process is described in detail in the method embodiment, and details are not described herein again.
上述本发明实施例序号仅仅为了描述,不代表实施例的优劣。The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成,也可以通过程序来指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,上述提到的存储介质可以是只读存储器,磁盘或光盘等。A person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium. The storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.
以上所述仅为本发明的较佳实施例,并不用以限制本发明,凡在本发明的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本发明的保护范围之内。 The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalents, improvements, etc., which are within the spirit and scope of the present invention, should be included in the protection of the present invention. Within the scope.

Claims (10)

  1. 一种基于PCIe总线的数据传输方法,其特征在于,所述方法包括:A data transmission method based on PCIe bus, characterized in that the method comprises:
    非透明桥接收请求设备发送的请求事务层包TLP,所述请求TLP包括请求者身份标识Requester ID字段、标识Tag字段和地址字段,所述地址字段用于指示目标设备的地址,所述请求者身份标识Requester ID字段携带所述请求设备的总线号、设备号和功能号,所述请求设备属于第一PCIe总线域,所述目标设备属于第二PCIe总线域,所述第一PCIe总线域和所述第二PCIe总线域通过所述非透明桥连接;The non-transparent bridge receives a request transaction layer packet TLP sent by the requesting device, where the request TLP includes a requester identity identifier Requester ID field, an identifier Tag field, and an address field, where the address field is used to indicate an address of the target device, the requester The identifier Requester ID field carries a bus number, a device number, and a function number of the requesting device, the requesting device belongs to a first PCIe bus domain, the target device belongs to a second PCIe bus domain, and the first PCIe bus domain and The second PCIe bus domain is connected by the non-transparent bridge;
    分别采用所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述请求TLP中的所述请求设备的总线号、设备号和功能号,得到替换后的请求TLP;Replacing the bus number, device number, and function number of the requesting device in the request TLP with the bus number, device number, and function number of the non-transparent bridge in the second PCIe bus domain, respectively, and replacing Request TLP;
    将所述替换后的请求TLP发送给所述目标设备。Sending the replaced request TLP to the target device.
  2. 根据权利要求1所述的方法,其特征在于,当所述请求TLP为读请求TLP时,在所述分别采用所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述请求TLP中的所述请求设备的总线号、设备号和功能号之前,所述方法还包括:The method according to claim 1, wherein when the request TLP is a read request TLP, the bus number, the device number, and the said non-transparent bridge are respectively used in the second PCIe bus domain. The function number, before replacing the bus number, the device number, and the function number of the requesting device in the requesting TLP, the method further includes:
    保存所述读请求TLP中携带的标识Tag字段的值、所述请求设备的总线号、设备号和功能号与所述替换后的读请求TLP在所述第二PCIe总线域中的标识Tag字段的值的映射关系。And saving a value of the identifier Tag field carried in the read request TLP, a bus number of the requesting device, a device number and a function number, and an identifier Tag field of the replaced read request TLP in the second PCIe bus domain The mapping of the values.
  3. 根据权利要求2所述的方法,其特征在于,当所述请求TLP为读请求TLP时,所述方法还包括:The method according to claim 2, wherein when the requesting TLP is a read request TLP, the method further comprises:
    接收所述目标设备发送的读完成TLP,所述读完成TLP包括请求者身份标识Requester ID字段、完成者身份标识Completer ID字段和标识Tag字段,所述完成者身份标识Completer ID字段携带所述目标设备的总线号、设备号和功能号;Receiving a read completion TLP sent by the target device, where the read completion TLP includes a requester identity identifier Requester ID field, a performer identity identifier Completer ID field, and an identity Tag field, where the completer identity identifier Completer ID field carries the target The bus number, device number and function number of the device;
    采用所述读完成TLP中的标识Tag字段的值,根据所述映射关系,得到所述请求设备的总线号、设备号和功能号以及所述读请求TLP中携带的标识Tag字段的值;And using the value of the identifier Tag field in the read completion TLP, according to the mapping relationship, obtaining a bus number, a device number, a function number of the requesting device, and a value of an identifier Tag field carried in the read request TLP;
    采用得到的所述请求设备的总线号、设备号和功能号,替换所述读完成TLP中的请求者身份标识Requester ID字段中的总线号、设备号和功能 号;Replacing the bus number, device number, and function in the Requester ID field of the requester identity in the read completion TLP using the obtained bus number, device number, and function number of the requesting device number;
    采用得到的所述读请求TLP中携带的标识Tag字段的值,替换所述读完成TLP中的标识Tag字段的值;And replacing, by using the value of the identifier Tag field carried in the read request TLP, a value of the identifier Tag field in the read completion TLP;
    采用所述非透明桥在所述第一PCIe总线域中的总线号、设备号和功能号,替换所述完成者身份标识Completer ID字段中的总线号、设备号和功能号,得到替换后的读完成TLP;Replacing the bus number, device number, and function number in the Completer Identity Completer ID field with the bus number, device number, and function number of the non-transparent bridge in the first PCIe bus domain, and obtaining the replaced Read completed TLP;
    将所述替换后的读完成TLP发送给所述请求设备。Transmitting the replaced read completion TLP to the requesting device.
  4. 根据权利要求2所述的方法,其特征在于,所述保存所述读请求TLP中携带的标识Tag字段的值、所述请求设备的总线号、设备号和功能号与所述替换后的读请求TLP在所述第二PCIe总线域中的标识Tag字段的值的映射关系,包括:The method according to claim 2, wherein the value of the identifier Tag field carried in the read request TLP, the bus number of the requesting device, the device number and the function number, and the read after the replacement are saved. And mapping the value of the identifier Tag field of the TLP in the second PCIe bus domain, including:
    将所述替换后的读请求TLP在所述第二PCIe总线域中的标识Tag字段的值保存为显示查找表LUT中的索引;Saving the value of the identifier Tag field of the replaced read request TLP in the second PCIe bus domain as an index in the display lookup table LUT;
    将所述读请求TLP中携带的标识Tag字段的值、所述请求设备的总线号、设备号和功能号保存为所述索引对应的表项。The value of the identifier Tag field carried in the read request TLP, the bus number of the requesting device, the device number, and the function number are saved as an entry corresponding to the index.
  5. 根据权利要求1所述的方法,其特征在于,所述分别采用所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述请求TLP中的所述请求设备的总线号、设备号和功能号,包括:The method according to claim 1, wherein said replacing said number in said request TLP with a bus number, a device number and a function number of said non-transparent bridge in said second PCIe bus domain Request the bus number, device number, and function number of the device, including:
    从PCIe配置空间寄存器中读取所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号;Reading a bus number, a device number, and a function number of the non-transparent bridge in the second PCIe bus domain from a PCIe configuration space register;
    采用读取到的所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述请求设备发送的所述请求TLP中的所述请求设备的总线号、设备号和功能号。Replacing the bus number, the device number, and the function number of the non-transparent bridge in the second PCIe bus domain, replacing the bus number of the requesting device in the request TLP sent by the requesting device, Device number and function number.
  6. 一种基于PCIe总线的数据传输装置,其特征在于,所述装置包括:A data transmission device based on a PCIe bus, characterized in that the device comprises:
    接收模块,用于非透明桥接收请求设备发送的请求事务层包TLP,所述请求TLP包括请求者身份标识Requester ID字段、标识Tag字段和地址字段,所述地址字段用于指示目标设备的地址,所述请求者身份标识Requester ID字段携带所述请求设备的总线号、设备号和功能号,所述请求设备属于第一PCIe总线域,所述目标设备属于第二PCIe总线域,所述第一PCIe总线域和所述第二PCIe总线域通过所述非透明桥连接;a receiving module, configured to receive, by the non-transparent bridge, a request transaction layer packet TLP sent by the requesting device, where the request TLP includes a requester identity identifier Requester ID field, an identifier Tag field, and an address field, where the address field is used to indicate an address of the target device The requester ID field carries a bus number, a device number, and a function number of the requesting device, the requesting device belongs to a first PCIe bus domain, and the target device belongs to a second PCIe bus domain, where the a PCIe bus domain and the second PCIe bus domain are connected by the non-transparent bridge;
    第一替换模块,用于分别采用所述非透明桥在所述第二PCIe总线域中 的总线号、设备号和功能号,替换所述请求TLP中的所述请求设备的总线号、设备号和功能号,得到替换后的请求TLP;a first replacement module, configured to respectively adopt the non-transparent bridge in the second PCIe bus domain a bus number, a device number, and a function number, replacing a bus number, a device number, and a function number of the requesting device in the requesting TLP, to obtain a replaced request TLP;
    发送模块,用于将所述替换后的请求TLP发送给所述目标设备。And a sending module, configured to send the replaced request TLP to the target device.
  7. 根据权利要求6所述的装置,其特征在于,当所述请求TLP为读请求TLP时,所述装置还包括:The apparatus according to claim 6, wherein when the requesting TLP is a read request TLP, the apparatus further comprises:
    保存模块,用于保存所述读请求TLP中携带的标识Tag字段的值、所述请求设备的总线号、设备号和功能号与所述替换后的读请求TLP在所述第二PCIe总线域中的标识Tag字段的值的映射关系。a saving module, configured to save a value of an identifier Tag field carried in the read request TLP, a bus number, a device number, and a function number of the requesting device, and the replaced read request TLP in the second PCIe bus domain The mapping relationship between the values of the tagged Tag field.
  8. 根据权利要求7所述的装置,其特征在于,当所述请求TLP为读请求TLP时,所述接收模块还用于接收所述目标设备发送的读完成TLP,所述读完成TLP包括请求者身份标识Requester ID字段、完成者身份标识Completer ID字段和标识Tag字段,所述完成者身份标识Completer ID字段携带所述目标设备的总线号、设备号和功能号;The apparatus according to claim 7, wherein when the request TLP is a read request TLP, the receiving module is further configured to receive a read completion TLP sent by the target device, where the read completion TLP includes a requester An identity identifier Requester ID field, a Completer ID identifier, and an identifier Tag field, where the Completer ID field carries a bus number, a device number, and a function number of the target device;
    所述装置还包括:获取模块,用于采用所述读完成TLP中的标识Tag字段的值,根据所述映射关系,得到所述请求设备的总线号、设备号和功能号以及所述读请求TLP中携带的标识Tag字段的值;The device further includes: an obtaining module, configured to adopt a value of the identifier Tag field in the read completion TLP, and obtain a bus number, a device number, a function number, and the read request of the requesting device according to the mapping relationship The value of the tagged Tag field carried in the TLP;
    第二替换模块,用于采用得到的所述请求设备的总线号、设备号和功能号,替换所述读完成TLP中的请求者身份标识Requester ID字段中的总线号、设备号和功能号;a second replacement module, configured to replace, by using the obtained bus number, device number, and function number of the requesting device, a bus number, a device number, and a function number in a Requester ID field of the requester identity in the read completion TLP;
    所述第二替换模块还用于采用得到的所述读请求TLP中携带的标识Tag字段的值,替换所述读完成TLP中的标识Tag字段的值;The second replacement module is further configured to replace, by using the value of the identifier Tag field carried in the read request TLP, a value of the identifier Tag field in the read completion TLP;
    第三替换模块,用于采用所述非透明桥在所述第一PCIe总线域中的总线号、设备号和功能号,替换所述完成者身份标识Completer ID字段中的总线号、设备号和功能号,得到替换后的读完成TLP;a third replacement module, configured to replace, by using a bus number, a device number, and a function number of the non-transparent bridge in the first PCIe bus domain, a bus number, a device number, and a number in the Completer ID field of the Completer Identity Function number, get the read completion TLP after replacement;
    所述发送模块还用于将所述替换后的读完成TLP发送给所述请求设备。The sending module is further configured to send the replaced read completion TLP to the requesting device.
  9. 根据权利要求7所述的装置,其特征在于,所述保存模块,包括:The device according to claim 7, wherein the saving module comprises:
    第一索引单元,用于将所述替换后的读请求TLP在所述第二PCIe总线域中的标识Tag字段的值保存为显示查找表LUT中的索引;a first indexing unit, configured to save, by using the value of the identifier Tag field of the replaced read request TLP in the second PCIe bus domain, an index in the display lookup table LUT;
    第二索引单元,用于将所述读请求TLP中携带的标识Tag字段的值、所述请求设备的总线号、设备号和功能号保存为所述索引对应的表项。 The second indexing unit is configured to save the value of the identifier Tag field carried in the read request TLP, the bus number of the requesting device, the device number, and the function number as an entry corresponding to the index.
  10. 根据权利要求6所述的装置,其特征在于,所述第一替换模块,包括:The device according to claim 6, wherein the first replacement module comprises:
    读取单元,用于从PCIe配置空间寄存器中读取所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号;a reading unit, configured to read, from a PCIe configuration space register, a bus number, a device number, and a function number of the non-transparent bridge in the second PCIe bus domain;
    替换单元,用于采用读取到的所述非透明桥在所述第二PCIe总线域中的总线号、设备号和功能号,替换所述请求设备发送的所述请求TLP中的所述请求设备的总线号、设备号和功能号。 a replacement unit, configured to replace, by using the read bus number, device number, and function number of the non-transparent bridge in the second PCIe bus domain, the request in the request TLP sent by the requesting device The bus number, device number, and function number of the device.
PCT/CN2015/094300 2014-11-14 2015-11-11 Pcie bus based data transmission method and device WO2016074619A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410647345.6A CN105653481B (en) 2014-11-14 2014-11-14 Data transmission method and device based on PCIe bus
CN201410647345.6 2014-11-14

Publications (1)

Publication Number Publication Date
WO2016074619A1 true WO2016074619A1 (en) 2016-05-19

Family

ID=55953745

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/094300 WO2016074619A1 (en) 2014-11-14 2015-11-11 Pcie bus based data transmission method and device

Country Status (2)

Country Link
CN (1) CN105653481B (en)
WO (1) WO2016074619A1 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106445573B (en) * 2015-08-11 2020-09-04 华为技术有限公司 Firmware upgrading method and device and high-speed peripheral equipment interconnection cluster system
WO2021147045A1 (en) * 2020-01-22 2021-07-29 华为技术有限公司 Pcie-based data transmission method and apparatus
CN111651397B (en) * 2020-05-09 2022-11-15 山东浪潮科学研究院有限公司 Method and equipment for accessing peripheral module of PXIe
CN116724308A (en) * 2021-05-24 2023-09-08 华为技术有限公司 Data transmission method and device
CN116257479B (en) * 2023-05-16 2023-08-15 北京象帝先计算技术有限公司 Reorder buffer, system, device, equipment and transmission method
CN116303148B (en) * 2023-05-17 2023-08-22 北京中科网威信息技术有限公司 Multi-slot device network card detection method and device, electronic device and storage medium
CN117076374B (en) * 2023-10-17 2024-02-09 苏州元脑智能科技有限公司 PCIe stream bus conversion method, device, equipment and medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101242371B (en) * 2008-03-14 2010-11-10 杭州华三通信技术有限公司 Method, system and device router overlapping based on PCIe exchange architecture
US20120096192A1 (en) * 2010-10-19 2012-04-19 Hitachi, Ltd. Storage apparatus and virtual port migration method for storage apparatus
US20130054867A1 (en) * 2011-08-23 2013-02-28 Fujitsu Limited Communication apparatus and id setting method
WO2014027416A1 (en) * 2012-08-17 2014-02-20 富士通株式会社 Information processing device and method for controlling information processing device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103430161B (en) * 2012-11-15 2016-12-21 华为技术有限公司 The method of a kind of Based PC IE Switch communication, Apparatus and system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101242371B (en) * 2008-03-14 2010-11-10 杭州华三通信技术有限公司 Method, system and device router overlapping based on PCIe exchange architecture
US20120096192A1 (en) * 2010-10-19 2012-04-19 Hitachi, Ltd. Storage apparatus and virtual port migration method for storage apparatus
US20130054867A1 (en) * 2011-08-23 2013-02-28 Fujitsu Limited Communication apparatus and id setting method
WO2014027416A1 (en) * 2012-08-17 2014-02-20 富士通株式会社 Information processing device and method for controlling information processing device

Also Published As

Publication number Publication date
CN105653481A (en) 2016-06-08
CN105653481B (en) 2019-03-26

Similar Documents

Publication Publication Date Title
WO2016074619A1 (en) Pcie bus based data transmission method and device
US10152433B2 (en) Inline PCI-IOV adapter
US9672143B2 (en) Remote memory ring buffers in a cluster of data processing nodes
KR101835851B1 (en) System and method for extended peripheral component interconnect express fabrics
US9189441B2 (en) Dual casting PCIE inbound writes to memory and peer devices
US8806098B1 (en) Multi root shared peripheral component interconnect express (PCIe) end point
US6658521B1 (en) Method and apparatus for address translation on PCI bus over infiniband network
RU2491616C2 (en) Apparatus, method and system for managing matrices
US20060190655A1 (en) Apparatus and method for transaction tag mapping between bus domains
US20040117561A1 (en) Snoop filter bypass
US20130151750A1 (en) Multi-root input output virtualization aware switch
US20140006659A1 (en) Driver-assisted base address register mapping
US8631184B2 (en) Interconnection method and device, for example for systems-on-chip
US10061707B2 (en) Speculative enumeration of bus-device-function address space
US20100250807A1 (en) Switch system, sub-switch and method of controlling switch system
US10754808B2 (en) Bus-device-function address space mapping
WO2021120623A1 (en) Data transmission method and apparatus, and related assembly
US20100306442A1 (en) Detecting lost and out of order posted write packets in a peripheral component interconnect (pci) express network
CN115269457A (en) Method and apparatus for enabling cache to store process specific information within devices supporting address translation services
JP5664187B2 (en) Interconnect apparatus and control method thereof
WO2019140885A1 (en) Directory processing method and device, and storage system
US20120324078A1 (en) Apparatus and method for sharing i/o device
US11836105B2 (en) Communication device, information processing system, and communication method
US10628373B2 (en) Systems and methods for transmitting an access request via a flexible register access bus
WO2024061344A1 (en) Data migration method and apparatus, and chip and computer-readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15859754

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15859754

Country of ref document: EP

Kind code of ref document: A1