CN118034917A

CN118034917A - PCIe resource allocation method and device, electronic equipment and storage medium

Info

Publication number: CN118034917A
Application number: CN202410077430.7A
Authority: CN
Inventors: 贺文彬; 艾山彬; 李道童; 孙秀强
Original assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Current assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Priority date: 2024-01-18
Filing date: 2024-01-18
Publication date: 2024-05-14

Abstract

The invention provides a PCIe resource allocation method, a PCIe resource allocation device, electronic equipment and a storage medium. The method comprises the following steps: under the condition that initial resources allocated by BIOS of a server for PCIe equipment connected to the server do not meet the requirements of the PCIe equipment, calling a first interrupt through Driver of the PCIe equipment, and informing an Operating System (OS) to write the resource requirement information of the PCIe equipment into different registers of the PCIe equipment; after restarting the server, the BIOS reads the resource demand information of the PCIe device, and reallocates PCIe resources for the PCIe device according to the resource demand information of the PCIe device. By utilizing the invention, the self-adaptive resource allocation of PCIe equipment can be realized, manual adaptation in BIOS codes is not needed, and the reserved resource requirements of various PCIe equipment are intelligently and dynamically adapted.

Description

PCIe resource allocation method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the PCIe field, and in particular, to a PCIe resource allocation method, apparatus, electronic device, and storage medium.

Background

The PCIe bus architecture is a high-speed serial interface for connecting external devices in a computer system. The system adopts a brand new serial communication protocol, and has higher bandwidth and lower delay compared with the traditional PCI bus architecture. The PCIe bus architecture adopts a point-to-point connection mode, namely each device is directly connected with the host, so that the bus contention problem is avoided.

On the Intel x86 platform, PCIe device manufacturers often need to consider compatibility of the UEFI mode and the Legacy mode for reasons of history alternation and downward compatibility, and in addition, PCIe device manufacturers need bus resources, MMIO resources, and IO resources for complete function use in the operation stage. In recent years, the iterative use of server product adaptation can find out a plurality of abnormal use problems of various PCIe devices caused by improper adaptation resources. In the related art, a conventional method is that each PCIe device needs to be adapted in the BIOS, that is, how many resources need to be reserved to adapt the BIOS one by one, which results in excessive cost and failure to achieve adaptive resource adaptation. Second, the prior art cannot self-adapt to the additional resources that are required by many devices today. With the development of technology, PCIe devices are increasingly demanded, and the prior art cannot meet the needs of various scenes. In addition, in the prior art, resource allocation is performed only by judging whether slots are idle, and the reserved resource requirements of all PCIe devices cannot be intelligently and dynamically adapted. The prior art only now follows the complement of PCIe protocol resource allocation, so a more intelligent and adaptive method is needed to meet the increasing PCIe device resource requirements and reduce the cost.

Disclosure of Invention

The embodiment of the invention provides a PCIe (peripheral component interconnect express) resource allocation method, a PCIe resource allocation device, electronic equipment and a storage medium, and aims to solve the problems in the background technology.

In order to solve the technical problems, the invention is realized as follows:

in a first aspect, an embodiment of the present invention provides a PCIe resource allocation method, including:

Under the condition that initial resources allocated by BIOS of a server for PCIe equipment connected to the server do not meet the requirements of the PCIe equipment, calling a first interrupt through Driver of the PCIe equipment, and informing an Operating System (OS) to write the resource requirement information of the PCIe equipment into different registers of the PCIe equipment;

After restarting the server, the BIOS reads the resource demand information of the PCIe device, and reallocates PCIe resources for the PCIe device according to the resource demand information of the PCIe device.

Optionally, the method further comprises: the BIOS allocates initial resources for the PCIe device according to the Option ROM of the PCIe device;

The BIOS loads an Operating System (OS) of the server according to an ACPI protocol and transfers control rights to the OS, and the OS loads a Driver of the PCIe device;

The Driver checks whether the resource type and the size of a first type of resource of the PCIe device are enough, where the first type of resource includes: memory address space, I/O address space, interrupt request IRQ;

the Driver checks whether the resource type and the size of a second type of resource of the PCIe device are enough, where the second type of resource includes: memory requirements, custom DMA channels, custom firmware, and configuration options are extended to determine whether initial resources allocated for the PCIe device meet the requirements of the PCIe device.

Optionally, the method further comprises: the Driver informs the BIOS of carrying out resource reservation and updates the Flag state of the BIOS;

After the server is restarted, the BIOS reads the resource requirement information of the PCIe device, including:

After restarting the server, the BIOS judges whether to read the register of the PCIe device according to the status of Flag;

and under the condition that the status of the Flag is updated, the BIOS reads the resource requirement information of the PCIe device from the register of the PCIe device.

Optionally, before the OS loads the Driver of the PCIe device, the method further includes: the OS sets different priorities for a plurality of PCIe devices connected with the server according to the respective importance degrees of the PCIe devices;

Reallocating PCIe resources for the PCIe device, including:

And in the case that the sum of PCIe resources required by all PCIe devices exceeds the maximum capacity of the server, reallocating PCIe resources for PCIe devices connected with the server according to the order of priority from high to low.

Optionally, in the process of reallocating PCIe resources for PCIe devices connected to the server in order of priority from high to low, the method further includes:

the server monitors the whole process and collects monitoring data, wherein the monitoring data comprises performance data of the PCIe equipment and service condition data of server system resources;

The server generates a monitoring log according to the monitoring data and a predefined rule;

and the server adjusts the distribution scheme of PCIe resources according to the monitoring log.

Optionally, the PCIe device connected to the server comprises a multi-tier PCIe device; reallocating PCIe resources for the PCIe device according to the resource demand information of the PCIe device, including:

Determining resource demand information of PCIe devices of a first hierarchy, and then traversing to PCIe devices of a next hierarchy step by step;

In the whole traversal process, corresponding resources are allocated to each PCIe device according to the resource demand information of the PCIe device.

In a second aspect, an embodiment of the present invention provides a PCIe resource allocation apparatus, where the apparatus includes: the interrupt calling module is configured to execute a first interrupt through a Driver of the PCIe device to inform an Operating System (OS) to write resource requirement information of the PCIe device into different registers of the PCIe device when initial resources allocated by a BIOS of a server for the PCIe device connected to the server do not meet the requirement of the PCIe device;

And the redistribution module is configured to execute the steps that after the server is restarted, the BIOS reads the resource demand information of the PCIe equipment, and redistributes PCIe resources to the PCIe equipment according to the resource demand information of the PCIe equipment.

Optionally, the apparatus further comprises:

the initial allocation module is configured to allocate initial resources to the PCIe equipment by the BIOS according to the Option ROM of the PCIe equipment;

the loading module is configured to execute BIOS to load the OS according to the ACPI protocol and transfer control rights to the OS, and the OS loads a Driver of the PCIe device;

An evaluation module configured to execute the Driver to check whether a resource type and size of a first type of resource of the PCIe device are sufficient, the first type of resource including: memory address space, I/O address space, interrupt request IRQ;

a checking module configured to execute the Driver to check whether a resource type and a size of a second class of resources of the PCIe device are sufficient, the second class of resources including: memory requirements, custom DMA channels, custom firmware, and configuration options are extended to determine whether initial resources allocated for the PCIe device meet the requirements of the PCIe device.

In a third aspect, an embodiment of the present invention provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and capable of running on the processor, which when executed by the processor, performs the steps of PCIe resource allocation.

In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium having stored thereon a computer program that, when executed by a processor, performs steps for PCIe resource allocation.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

according to the scheme, the upper Driver informs the OS of the resource requirement required by the PCIe equipment, writes the resource requirement information into the register corresponding to the PCIe equipment, and informs the BIOS to carry out resource adaptation, so that automatic resource allocation is realized, and the complicated process of manually adapting the resources is avoided. When a new PCIe device is accessed into the system, the Driver can identify the type and the size of the required resources, and the BIOS reserves and distributes the resources according to the information in the register in the restarting process, so that the adaptive resource adaptation of the PCIe device is realized. By means of automatic resource adaptation, the modification requirement on BIOS codes is reduced, and development and maintenance costs are reduced. In addition, the automatic resource adaptation can also improve the flexibility and the expandability of the system, so that the system can be better adapted to PCIe devices of different types and specifications. By utilizing the invention, more intelligent and efficient PCIe equipment resource adaptation can be realized.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart illustrating a PCIe resource allocation method according to an embodiment of the present invention;

FIG. 2 is a flowchart of a PCIe initial resource allocation method in an embodiment of the invention;

FIG. 3 is a flowchart of a PCIe resource reallocation method in an embodiment of the invention;

FIG. 4 is a block diagram of a PCIe resource allocation device in an embodiment of the invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

To facilitate the following description, definitions of some terms are first given:

PCIe: PCIe refers to PERIPHERAL COMPONENT INTERCONNECT EXPRESS, a high-speed serial computer expansion bus. Which is an interface standard for connecting external devices in a computer system. Compared with the traditional PCI bus architecture, PCIe adopts a brand new serial communication protocol and has higher bandwidth and lower delay. PCIe buses are widely used in modern computer systems, such as consumer notebook computers, desktop computers, enterprise data servers, and the like. In a server system, PCIe bus architecture is used to connect various peripherals, such as a network card, a storage device, an accelerator card, etc., to implement data input and output, and in the embodiment of the present invention, these PCIe peripherals connected to a server are referred to as PCIe devices. The PCIe bus architecture meets the requirements of large-scale data centers and cloud computing environments on network performance by providing a high-speed and stable data transmission channel, and simultaneously meets the high-speed data interaction requirements of a server on storage equipment and an acceleration card.

BIOS refers to Basic Input/Output System, i.e., basic Input/Output System. The system is firmware in the server system, is responsible for hardware initialization and self-checking when the server is started, and provides basic input and output functions. The BIOS is located on the motherboard of the server and is software stored in the flash memory chip. It acts as a bridge with the operating system OS and PCIe devices. When the server is started, the BIOS firstly performs self-checking to detect whether PCIe devices connected with the server work normally. The BIOS then loads the boot program of the operating system OS, giving control to the OS. In addition, BIOS provides basic input and output functions such as keyboard, display and hard disk control.

Option ROM-Option ROM refers to an Option ROM (Read-Only Memory), also known as expansion ROM or plug-in ROM. It is firmware stored on a server hardware device that provides additional functionality and drivers for the device. In an embodiment of the invention, the Option ROM is integrated on PCIe devices such as a graphics card, a network card, a RAID card, and the like. It contains the drivers and other associated firmware code of the PCIe device for communicating with the OS and controlling the functions of the device.

Driver: driver refers to a device Driver, which is software used to communicate and control with hardware devices. Drivers expose the functionality and features of the device to operating system and application use by interacting with the operating system. In the embodiment of the invention, the Driver is used for transmitting the resources required by the PCIe equipment through the upper layer Driver and informing the OS to inform the BIOS of realizing the purpose of self-adapting the resources by writing the resource demand information into the register corresponding to the PCIe equipment.

The embodiment of the invention provides a PCIe resource allocation method, and the application object of the embodiment of the invention can be a server of a large data center. This ensures that critical devices within the data center, such as high performance network cards or storage devices, always get enough resources to improve the operational efficiency and stability of the overall data center. The technical scheme of the invention can ensure real-time critical PCIe equipment such as a real-time data acquisition card to self-adaptively allocate enough resources, thereby ensuring the response speed and reliability of the system. Fig. 1 is a complete flowchart of a PCIe resource allocation method according to an embodiment of the present invention, where, as shown in fig. 1, the PCIe resource allocation method provided by the present invention is divided into two processes of initial resource allocation and reallocation, and the two processes are coupled through a loop structure, and the loop process is repeated to implement adaptive resource allocation for all PCIe devices connected to a server, and each process will be described in detail below.

Referring to fig. 2, fig. 2 is a flowchart of a PCIe initial resource allocation method according to an embodiment of the present invention, where the method includes:

In step S101, the BIOS allocates initial resources to the PCIe device according to the Option ROM of the PCIe device.

In this embodiment, this step begins after the BIOS has completed initializing all PCIe devices connected to the server. The initialization of the PCIe device by the BIOS mainly includes three steps of identifying, training, and enumerating.

And (3) identification: during the server boot process, the BIOS scans the PCIe bus to identify all PCIe devices connected to the bus. This is accomplished by reading the configuration space of the PCIe device, which contains information such as vendor ID, device ID, subsystem ID, etc. of the device. The BIOS determines the presence and type of the device by reading this information.

Training: once PCIe devices are identified, the BIOS trains the devices. The training process is accomplished primarily through LTSSM (LINK TRAINING AND Status STATE MACHINE), which is a state machine used to negotiate and establish a communication link between the PCIe device and the host. In the training process, the BIOS can communicate with the PCIe device to negotiate the bandwidth, speed and other parameters of the device, so as to ensure that the device can work normally.

Enumerating: after training is completed, the BIOS enumerates the PCIe devices. The enumeration process is mainly used for allocating unique resource addresses to each PCIe device, including I/O addresses, memory addresses, interrupt vectors, and the like. The BIOS allocates resources for each PCIe device and records this information in the configuration space of the server system for use by the operating system and device drivers.

Option ROM is firmware embedded in the memory of a PCIe device to provide initialization and boot functions for the device, and is described above in a term that is not repeated here.

Specifically, when the BIOS is identifying a PCIe device, it is detected whether the PCIe device has an Option ROM. If the device has an Option ROM, the BIOS will load the Option ROM into system memory and execute the code therein. The code in the Option ROM tells the BIOS the nature and needs of the device, as well as the resources required. Based on the information provided by the Option ROM, the BIOS allocates initial resources for the PCIe device. These resources include I/O addresses, memory addresses, interrupt vectors, etc., and can be understood as the most basic resources required for the device to operate. The BIOS will allocate the appropriate resources to the device based on the needs of the device and the available resources of the server system. Through the Option ROM, the device can provide detailed initialization and resource requirement information for the BIOS, so that the BIOS can perform resource allocation according to the requirements of the device. Thus, the equipment can be ensured to obtain the required initial resources when the server system is started, and can work normally. Note that the Option ROM is firmware provided by the device manufacturer, so the Option ROM of different devices may be different, and the BIOS parses and executes the Option ROM of different devices to ensure accuracy and compatibility of resource allocation.

In step S102, the BIOS loads the OS of the server according to the ACPI protocol and hands over control to the OS, and the OS loads the Driver of the PCIe device.

The BIOS loads the server's Operating System (OS) according to ACPI (Advanced Configuration and Power Interface) protocol, ACPI is an open standard for managing power and configuration information, which defines a set of interfaces and data structures that enable the operating system to interact and manage with the hardware devices. Through the ACPI protocol, the BIOS can communicate with an OS (operating system) to transfer control of the system to the OS, thereby enabling the OS to manage and control PCIe devices. Loading Driver of PCIe devices is an important task of the OS, which is responsible for identifying and managing PCIe devices, providing them with the necessary drivers and resources to ensure that the devices are working properly. By loading drivers, the OS can communicate with PCIe devices and control and manage the devices. Thus, the server system can fully utilize the functions of the PCIe device and provide better performance and functions.

Step S103, the Driver checks whether the resource type and size of the first type of resource of the PCIe device are sufficient, where the first type of resource includes: memory address space, I/O address space, interrupt request IRQ.

For better management and allocation of these resources, we divide the resources required for PCIe into first class resources and second class resources. The first type of resource is a basic resource indicated by an Option ROM of the PCIe device, and the following details are developed for the first type of resource:

PCIe devices may request two types of address spaces: memory address space and I/O address space. For memory address space, PCIe devices may request 32-bit or 64-bit address ranges, depending on the capabilities and requirements of the device. PCIe devices use Base ADDRESS REGISTERS (BARs) to specify the memory address range that they need, and each BAR contains an address and a size field that specifies the starting address and size of the memory range that the device needs. Each PCIe device has an addressability field to indicate the size of the address space supported by the device.

The I/O address space is an address range for input and output communications with PCIe devices, which for PCIe devices typically includes an address range for configuration, control, and data transfer of the device. Similar to the memory address space, a PCIe device uses BARs to specify the I/O address range that it needs. Each I/O BAR contains an address and a size field specifying the starting address and size of the I/O address range required by the device. Similarly, each PCIe device has an addressability field to indicate the I/O address space size supported by the device.

PCIe devices may obtain interrupt service by requesting an allocation of one or more interrupt lines (IRQ lines). The resource requirements of PCIe device Interrupt Requests (IRQs) require consideration of both hardware and software aspects, including interrupt line allocation and MSI support for PCIe devices, while software aspects involve the co-operation of the OS and PCIe device drivers to ensure that interrupt services can be efficiently managed and executed.

Step S104, the Driver checks whether the resource type and size of the second class resource of the PCIe device are sufficient, where the second class resource includes: memory requirements, custom DMA channels, custom firmware, and configuration options are extended to determine whether initial resources allocated for the PCIe device meet the requirements of the PCIe device.

The second type of resource is an additional specific resource for the PCIe device, and the following details are given for the second type of resource:

the extended memory requirements of a PCIe device are the memory resources that the PCIe device accesses over the PCIe bus, rather than the memory requirements of the PCIe device itself. These extended memory resources are typically used for data buffering, shared data areas, device registers, and the like. PCIe devices may access system memory through memory mapped I/O (MMIO), which is a mechanism that allows PCIe devices to directly access system memory, with device registers and buffers mapped into the physical address space of the system, thereby enabling the exchange, sharing, and storage of data.

Custom DMA channels for PCIe devices are device-supported channels or paths for Direct Memory Access (DMA) operations, and these channels are custom or configured to meet device specific requirements. DMA is a data transmission method by which external devices can directly read and write to the system memory without intervention of a CPU, thereby improving the efficiency of data transmission. Typically, PCIe devices support multiple DMA channels, each of which can be independently configured and used. Custom DMA channels mean that a device may support a particular configuration of channels to meet the needs of the device manufacturer or a particular application.

Custom firmware and configuration options for PCIe devices are hardware-specific firmware and related configuration options provided by the device manufacturer. These firmware and options allow the device manufacturer or system integrator to personalize and optimize the PCIe device according to particular needs, use cases, or environments. Where custom firmware is hardware-specific software provided by a device manufacturer for its PCIe devices, including the firmware, drivers, and related software components of the device, and configuration options refer to adjustable parameters and settings provided by the PCIe device, allowing a user or system integrator to customize the behavior of the device. Custom firmware and configuration options provide flexibility and configurability for PCIe devices so that the devices can better accommodate a variety of application scenarios and requirements.

The method for specifically executing and checking the first type of resource and the second type of resource by the Driver is not limited in this embodiment of the present invention, for example, the Driver may obtain the requirement information of the resource by reading the related register of the configuration space of the PCIe device.

The PCIe resources are divided into a first class of resources and a second class of resources in the sense of realizing more accurate and customized resource allocation to meet specific requirements of different devices and to improve performance and reliability of the system. The first type of resources is necessary for the PCIe device to transmit and communicate data. The second type of resource is an additional specific resource of the PCIe device, which is provided to meet the functional and performance requirements of the specific device. Distinguishing the two types of resources facilitates our management and definition of the initially allocated and reallocated resource objects.

FIG. 3 is a flowchart of a PCIe resource reallocation method in an embodiment of the invention, as shown in FIG. 3, comprising:

In step S201, when the initial resource allocated by the BIOS of the server for the PCIe device connected to the server does not meet the requirement of the PCIe device, the Driver of the PCIe device invokes a first interrupt, and notifies the operating system OS to write the resource requirement information of the PCIe device into a different register of the PCIe device.

And when the BIOS of the server does not meet the requirement of the PCIe device for the initial resources allocated for the PCIe device connected to the server, namely, when the Driver evaluates that the type and the size of the resources of the PCIe device are not enough, starting the step. The Driver of the PCIe device will invoke a first interrupt (SMI) to trigger an interrupt event, and the operating system OS writes the resource requirement information of the device into a different register of the device according to the requirements of the Driver. These registers may store additional resource information required by the PCIe device, such as more I/O address space, memory Mapped IO (MMIO) resources, etc. SMI is a system management Interrupt (SYSTEM MANAGEMENT Interrupt), which is a special Interrupt type. Which is an interrupt triggered by a system management Controller (SYSTEM MANAGEMENT Controller, SMC) for handling system-level management and control tasks. SMIs are typically triggered by system management controller hardware or firmware to perform some specific system management tasks, such as power management, temperature monitoring, system status monitoring, and the like. When the system management controller detects that the system management operation needs to be performed, the system management controller triggers the SMI interrupt and transfers control to the SMI handler. In the embodiment of the invention, the Driver of the PCIe device can communicate with the OS by triggering the SMI and communicate the resource requirement of the PCIe device, and the OS correspondingly writes the resource requirement information of the PCIe device into different registers of the device according to the resource requirement communicated by the Driver. In summary, after triggering the SMI interrupt, the server system pauses the current execution flow and transfers control to the SMI handler, where the system may perform some special operations, such as reading and writing registers, modifying system configuration, etc., and Driver may inform the OS of the specific needs of the device, and the OS may write the device's resource needs information into the device's registers by writing them into the device's registers.

Step S202, after restarting the server, the BIOS reads the resource requirement information of the PCIe device, and reallocates PCIe resources for the PCIe device according to the resource requirement information of the PCIe device.

After Driver triggers SMI, the server will perform an automatic restart. The restart process is to implement adaptive resource reservation of the PCIe device, and during the restart process, the BIOS reads resource information in the register according to the Flag state, and performs resource reservation and reallocation on the PCIe device to meet resource requirements of different devices, which will be described in detail later. And the BIOS reserves and reallocates PCIe resources of different PCIe devices under different ports according to the read resource demand information. This means that the BIOS allocates additional, appropriate PCIe resources, such as bus resources, MMIO resources, for each device according to the actual requirements of the PCIe device, so as to meet the actual resource requirements of the PCIe device. After the redistribution of the PCIe resources is completed, the server system reenters the OS of the operating system, and if the PCIe devices still do not meet the resource requirements, the OS reloads the Driver of the PCIe devices to start a new cycle until all types and all types of resources required by all PCIe devices meet respective requirements, so as to implement successful adaptive PCIe resource distribution.

When Driver of PCIe device triggers SMI, the method further includes:

in step S301, the Driver informs the BIOS of reserving resources, and updates the Flag state of the BIOS.

In this step, the Flag of the BIOS is updated, that is, the Flag is set, which may be considered separately from the software and hardware levels, which is not limited, and for example, updating the value of Flag to 1 may indicate that the Flag bit is enabled. The Flag may be one or more bits to indicate whether the BIOS is to read a register of the PCIe device.

Step S302, after the server is restarted, the BIOS reads the resource requirement information of the PCIe device, including:

After restarting the server, the BIOS judges whether to read the register of the PCIe device according to the status of Flag; and under the condition that the status of the Flag is updated, the BIOS reads the resource requirement information of the PCIe device from the register of the PCIe device.

If the BIOS detects that Flag has been set, it indicates that the PCIe device has additional resource requirements, i.e., that the operating system OS has written the resource requirements into registers of the PCIe device. The BIOS reads the resource information in the register and performs resource reservation and adaptation according to the information. By reading the resource information in the register, the BIOS can learn the various resource types and amounts required by the PCIe device, thereby being able to reserve appropriate resources for the device in advance. Through the adaptive resource reservation scheme, the BIOS can flexibly allocate resources according to the requirements of different PCIe devices, and the traditional manual adaptation process is avoided. Therefore, the cost can be reduced, the efficiency can be improved, and the method can adapt to the change of the demands of PCIe devices of different types.

Considering that a server may face resource exception handling when PCIe resources are allocated, it is more common that the sum of PCIe resource requirements required by a device is greater than the maximum capacity of the server. There is a need to propose a policy to cope with such anomalies, before the OS loads the Driver of the PCIe device, further comprising:

in step S401, the OS sets different priorities for the PCIe devices according to the importance degrees of the PCIe devices connected to the server.

The OS needs to identify the type of all PCIe devices connected to the server, with different types of devices (e.g., graphics cards, network interface cards, memory controllers, etc.) having different importance in the running environment of the server. Application dependencies should be considered at this point, i.e., identifying which applications depend on a particular PCIe device. Devices that are essential to critical applications should be given higher priority. While analyzing the performance requirements of each device, some devices will require higher bandwidth and faster response times. A set of priority setting criteria is formulated based on the type of device, performance requirements and application dependencies. These criteria should be configurable to accommodate different usage scenarios and requirements, and embodiments of the present invention are not limited in this regard. Of course, the priority setting should have a certain flexibility, allowing the system administrator to adjust according to the actual needs. During the operation of the server system, the OS dynamically adjusts the resource allocation according to the current workload and performance metrics. On the premise that the priority setting mechanisms are effectively integrated with the existing hardware abstraction layer and driver framework of the OS, each priority setting mechanism needs to be thoroughly tested in the system so as to ensure that the priority policy does not introduce performance bottlenecks or system instability. In order for the user to manage the priority of the PCIe devices, a user interface tool may be provided so that a system administrator may easily configure and adjust the priority of the PCIe devices, and may also set up monitoring and logging to enable the administrator to track resource usage and device performance and adjust the priority settings as necessary, as will be described in greater detail below.

In step S402, if the sum of PCIe resources required by all PCIe devices exceeds the maximum capacity of the server, PCIe resources are reallocated to PCIe devices connected to the server in order of priority from high to low.

As already mentioned above, in a server, PCIe resources (e.g., bandwidth and interrupt lines) are limited, and when the sum of the resources required by all PCIe devices exceeds the maximum capacity of the server, the operating system or management software needs to reallocate PCIe resources from high to low according to a set priority. First, it is necessary to ensure that the highest priority device gets its smallest necessary resources. For the remaining resources, the other devices are allocated proportionally, or continue to be allocated in priority order. In the priority resource allocation process under the condition of insufficient resources, the Driver communicates with PCIe devices to inform new resource limitations, and some PCIe devices need to adjust the working modes of the PCIe devices to adapt to the priority resource allocation policy. In the whole process, attention is required to be paid to a performance feedback loop, namely, the performance of a server system and the states of all PCIe devices are monitored in real time, and if performance problems or resource waste are detected, the resource allocation is dynamically adjusted.

By providing the priority resource allocation strategy for handling the resource abnormality, the server can more intelligently manage the resource, and the overall performance, stability and maintainability of the system are improved. This is critical to servers supporting various workloads and applications.

In the process of reallocating PCIe resources for PCIe devices connected to the server in the order from high priority to low priority, the method further includes:

Step S501, the server monitors the whole process and collects monitoring data, wherein the monitoring data comprises performance data of the PCIe equipment and service condition data of the server system resources;

Step S502, the server generates a monitoring log according to the monitoring data and a predefined rule;

In step S503, the server adjusts the PCIe resource allocation scheme according to the monitor log.

Referring to step S501, the server monitors the entire process and collects monitoring data, where the monitoring data includes performance data of the PCIe device and usage data of server system resources.

The performance data of PCIe devices includes bandwidth usage of the device, data transfer rate, error rate. Broadband usage represents the extent to which a device uses the PCIe bus, i.e., the ratio of the data transfer rate to the maximum bandwidth of the bus; the data transfer rate is the rate at which the device actually transfers data over the PCIe bus, typically measured in terms of the amount of data transferred per second; and the error rate is the number of errors that occur during data transmission, which may be due to hardware failure, incompatibility, or signal interference, etc. By monitoring this data, the server may evaluate whether the performance of each PCIe device reaches an expected level. These data may help to quickly locate problems when PCIe devices fail or degrade. Knowing which devices are resource intensive can help to reasonably allocate PCIe resources throughout the server system.

The server system resource usage data comprises CPU usage and system load, wherein the CPU usage characterizes the usage of processor resources, and is usually expressed in percentage; and network bandwidth usage characterizes the data transfer rate of the server network interface.

The server can be helped to acquire the current load condition and whether a resource bottleneck exists or not through monitoring the service condition data of the server system resources. Additionally, how future PCIe devices are upgraded or expanded may also be planned based on historical data of server resource usage.

Referring to step S502, the server generates a monitoring log according to the monitoring data and predefined rules.

Predefined rules are used to instruct the server to automatically record events, alarms, and other related activities into the log file. In the embodiment of the invention, the predefined rule needs to make the following claims: first, event levels, events that set different levels need to be recorded. Such as information, warnings, errors, and major errors. More severe events are typically always recorded, while lower level information may only be recorded in detailed mode. Next is the event type, the predefined rules specify which types of events need to be logged. Such as system login, system error, network connection, security alert, application state change, etc.

The resource usage of the server also needs to be declared, such as disk space, memory usage, CPU usage, etc., information may be recorded when certain thresholds are reached. According to the method, the monitoring log can be used for facilitating problem tracking and historical data analysis.

Step S503 is involved, and the server adjusts the distribution scheme of PCIe resources according to the monitoring log.

To understand this step, an example is described herein in connection with step S601 and step S602, assuming that there is one server to which a plurality of PCIe devices including a high-performance Graphics Processing Unit (GPU), a memory controller, and a Network Interface Card (NIC) are connected. This server may be used to run a variety of applications including data analysis, video processing, and web services. The server will monitor the performance of each PCIe device, including bandwidth usage, data transfer rate, and error rate. The monitored data is recorded in a monitoring log. In performing the priority resource allocation of step S502, we find that the log shows that at high load, the bandwidth of the GPU with higher priority often reaches a limit, while the bandwidth usage of the memory controller is relatively low. At this time, a corresponding adjustment is needed, and this adjustment is performed automatically, when the system detects a high load of the GPU, the bandwidth quota of the GPU is automatically increased, and at the same time, the bandwidth quota of the storage controller is correspondingly reduced. After dynamic adjustment, the system monitors the GPU for performance improvement and ensures that the memory controller's performance is not negatively impacted, and all adjustment actions and results are recorded in a new monitor log.

The execution sequence and the allocation sequence of the resource allocation to the plurality of PCIe devices follow a priority traversal algorithm, and the method comprises the following steps:

PCIe devices connected to the server include multi-tier PCIe devices; reallocating PCIe resources for the PCIe device according to the resource demand information of the PCIe device, including: determining resource demand information of PCIe devices of a first hierarchy, and then traversing to PCIe devices of a next hierarchy step by step; in the whole traversal process, corresponding resources are allocated to each PCIe device according to the resource demand information of the PCIe device.

Here, it should be mentioned that the entire PCIe system according to the embodiment of the present invention has a tree structure, where the tree structure is formed by a root multiplexer, a switch, an endpoint device, and a bridge. The root multiplexer is a core part of the PCIe architecture, typically integrated in the chipset or CPU of the motherboard. It is responsible for managing and configuring all devices connected to the PCIe bus. The root multiplexer is connected to the CPU and memory, processes transactions on the PCIe bus, and is a bridge between the PCIe bus and the host processor and other system resources. PCIe switches are used to extend the number of PCIe ports in a system, allowing multiple devices to be connected to a host at the same time. They are similar to network switches that can efficiently manage and route packets on PCIe buses, and in addition, switches can provide bandwidth allocation and load balancing between ports. An endpoint device is any device that is directly connected to the PCIe bus, i.e., the PCIe peripheral device to which the server described above is connected, such as a graphics card, network card, memory controller, etc. They are the final target devices in the PCIe architecture for performing specific functions. Each endpoint device has its own configuration space for system identification and configuration. PCIe bridges are used to connect two different types of buses, such as connecting PCIe to a PCI or other type of bus, which translate data and regulate signals between different bus standards, the bridges enabling legacy devices to be compatible with modern PCIe-based systems.

Based on the tree structure, the execution sequence and the allocation sequence of the resource allocation for the PCIe devices in the embodiment of the present invention follow a priority traversal algorithm, and the purpose of the algorithm is to ensure that each device, whether a switch, a bridge or an endpoint device, obtains appropriate resource allocation according to the resource requirements of the device. The following describes the concept of this algorithm:

First, the resource requirements of the root multiplexer, i.e., the resource requirement information of the PCIe devices of the first hierarchy, are determined, and traversal and resource allocation of the PCIe system begins with the root multiplexer. The root multiplexer is the starting point of the entire PCIe tree, and first determines the resource requirements of the root multiplexer itself, including the necessary address space, interrupt lines, etc. Traversing from the root port, the root multiplexer has multiple root ports, each of which may be connected to a switch or directly to an endpoint device. From these root ports, the traversal algorithm steps deep into each connected device, i.e., traverses the PCIe devices of the next hierarchy. The switch and downstream ports are then processed, and when a root port is connected to the switch, the algorithm first allocates resources for the switch itself. The algorithm then traverses each downstream port of the switch and allocates resources for the devices to which each port is connected. This includes assigning address space, configuration space, interrupts, etc. to each device. If the downstream port of a switch is connected to another switch again, the algorithm will recursively repeat the above process until the endpoint device is reached. In the whole process, the system allocates resources according to the requirements of each PCIe device. Once the algorithm reaches the end of the tree (i.e., the endpoint device), it will trace back to the last node (which may be a port of a switch or root multiplexer) and continue to explore other non-traversed branches. This process continues until all devices are accessed and resources are allocated according to their needs. In this way, the entire PCIe system can efficiently manage a large number of PCIe devices in the resource allocation process while maintaining high performance and low latency communications.

It should also be mentioned that the identification and allocation sequence of PCIe devices follows a certain code logic, which includes 8 main functions, and the following description is given for these 8 main functions:

QueryPciDevice: this step involves identifying all PCIe devices connected to the server, which will obtain basic information for all PCIe devices, including device type (e.g., graphics card, network card, etc.) and manufacturer ID. This information is critical for subsequent resource allocation.

ConvertResources: the step of converting the resource requirements (such as memory, I/O ports, interrupts, etc.) of the PCIe device into a data structure that the BIOS can recognize and process ensures the compatibility of the hardware resource requirements and the underlying management mode of the system.

CalculateBrgResources: the amount of resources required by PCIe bridge devices is calculated, and the bridge devices play an important role in PCIe systems, connecting different devices or groups of devices, and requiring sufficient resources to ensure the efficiency of data transmission.

CreateRootResDsc: the method comprises the steps of creating a resource data structure for describing a PCIe root multiplexer, wherein the root multiplexer is a core part of a PCIe system, the resource management of the root multiplexer is critical to the performance of the whole system, the detailed description of the root multiplexer in a tree structure part of the PCIe system is described above, and the resource management of the root multiplexer is critical to the performance of the whole system.

SubmitResources: the resource requirement of the PCIe device is submitted to the BIOS for processing, which is a key link for ensuring that the hardware requirement is identified by the bottom layer of the system and corresponding resources are prepared.

EfiPciHostBridgeAllocateResources: at the BIOS level, resources are allocated for the main bridge of PCIe devices, which is the main channel connecting root complexity and other PCIe devices, and the correct allocation of resources is critical to the stability and efficiency of the whole system.

ApplyAcpiResources: ACPI resource configuration is applied, including power management and device configuration.

AssignBusResources: allocating resources on the PCIe bus to connected devices involves the allocation of resources for the particular PCIe device (e.g., graphics card, network card, etc.).

Through the code logic formed by the 8 main functions, the whole process from the identification of the PCIe equipment by the system to the acquisition of the required resources is ensured to be smoothly carried out according to the preset logic, which is significant for maintaining the stability of the system, ensuring the effective communication among the equipment and optimizing the overall performance.

FIG. 4 is a block diagram of a PCIe resource allocation device according to an embodiment of the present invention, as shown in FIG. 4: the device comprises:

An interrupt calling module 601, configured to execute, when an initial resource allocated by a BIOS of a server for a PCIe device connected to the server does not meet a requirement of the PCIe device, call a first interrupt through a Driver of the PCIe device, and notify an operating system OS to write resource requirement information of the PCIe device into a different register of the PCIe device;

And the reassignment module 602 is configured to execute the steps of reading the resource requirement information of the PCIe device by the BIOS after the server is restarted, and reassigning PCIe resources to the PCIe device according to the resource requirement information of the PCIe device.

In a possible embodiment, the apparatus further comprises:

The embodiment of the invention also provides electronic equipment, which comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program realizes the processes of the embodiment of the PCIe resource allocation method when being executed by the processor, and can achieve the same technical effects, and the repetition is avoided, so that the description is omitted.

The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the above processes in the PCIe resource allocation method embodiment, and can achieve the same technical effects, and in order to avoid repetition, a detailed description is omitted here. In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.

It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article, or terminal device that includes the element. The PCIe resource allocation method, apparatus, electronic device and storage medium provided by the present invention are described in detail above, and specific examples are applied to illustrate the principles and embodiments of the present invention, where the descriptions of the above examples are only used to help understand the method and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims

1. A PCIe resource allocation method, said method comprising:

2. The method as recited in claim 1, further comprising:

the BIOS allocates initial resources for the PCIe device according to the Option ROM of the PCIe device;

3. The method as recited in claim 1, further comprising:

The Driver informs the BIOS of carrying out resource reservation and updates the Flag state of the BIOS;

4. The method of claim 2, further comprising, prior to the OS loading the Driver of the PCIe device:

The OS sets different priorities for a plurality of PCIe devices connected with the server according to the respective importance degrees of the PCIe devices;

Reallocating PCIe resources for the PCIe device, including:

5. The method of claim 4, wherein during the reallocating PCIe resources to PCIe devices connected to the server in a priority order from high to low, further comprising:

6. The method of claim 1, wherein the PCIe devices connected to the server comprise a multi-tier PCIe device; reallocating PCIe resources for the PCIe device according to the resource demand information of the PCIe device, including:

7. A PCIe resource allocation apparatus, said apparatus comprising:

The interrupt calling module is configured to execute a first interrupt through a Driver of the PCIe device to inform an Operating System (OS) to write resource requirement information of the PCIe device into different registers of the PCIe device when initial resources allocated by a BIOS of a server for the PCIe device connected to the server do not meet the requirement of the PCIe device;

8. The apparatus of claim 7, wherein the apparatus further comprises:

9. An electronic device, comprising: a processor, a memory and a computer program stored on the memory and capable of running on the processor, which when executed by the processor performs the steps of the method according to any of claims 1-6.

10. A computer readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the method according to any of claims 1-6.