CN115586867B

CN115586867B - NVMe controller

Info

Publication number: CN115586867B
Application number: CN202211182563.8A
Authority: CN
Inventors: 张宇军; 段宗胜; 孟繁毅
Original assignee: Yusur Technology Co ltd
Current assignee: Yusur Technology Co ltd
Priority date: 2022-09-27
Filing date: 2022-09-27
Publication date: 2023-08-22
Anticipated expiration: 2042-09-27
Also published as: CN115586867A

Abstract

The application provides an NVMe controller, comprising: one NVMe command subsystem and a plurality of NVM subsystems respectively connected to the control registers; the NVMe command subsystem realizes an admin queue based on hardware and is used for executing NVMe admin commands and configuring a plurality of NVME IO queues; the NVM subsystems are arranged in parallel, the NVM subsystems are multiplexing systems, each NVM subsystem is based on hardware to realize an NVMe IO queue, and the NVM subsystems are used for executing NVMe IO commands in parallel. When the NVMe controller executes the command, the application can realize that interaction with the ARM core is not needed, effectively reduce communication overhead and resource consumption, improve parallelism, ensure stable delay, avoid being influenced by an operating system and an algorithm, and improve the application reliability and the service life of the NVMe controller.

Description

NVMe controller

Technical Field

The application relates to the technical field of computers, in particular to an NVMe controller.

Background

The NVMe protocol has been widely applied to consumer-level Solid State Disk (SSD) or enterprise-level SSD, including full flash array (AFA) composed of NVMe SSD used by data center, wherein, the NVMe controller is used as a management interface for communicating host with the back end of SSD, which has an important effect on NVMe SSD performance. One command standard flow of NVMe is shown in fig. 1, and the specific steps are as follows:

I. the host creates a command to be executed in a particular command commit queue SQ (submission queue).

And II, updating a queue tail doorbell register of the command submitting queue SQ by the host computer, and storing a new pointer pointing to the queue tail entry in the register. By which a new command to be executed is instructed to be submitted by the NVMe controller.

The nvme controller fetches the command from the command commit queue SQ in the host for subsequent execution.

The NVMe controller arbitrates the commands, and selects the command to be executed next from the acquired commands according to an arbitration mechanism.

And V, after the command execution is completed, the NVMe controller writes a command completion entry into the related command completion queue. The completion entry contains identification information of the relevant command submission queue and command.

The nvme controller sends an interrupt request to the host indicating that the host has a command complete entry to process.

And VII, processing a command completion entry in the completion queue command. The processing includes error processing required according to the error prompt.

And VIII, updating a queue head doorbell register of the completion queue command by the host to indicate that the command completion entry is processed.

At present, the existing NVMe controller is composed of one or more ARM cores, the performance of the NVMe SSD has a great relationship with the number of ARM cores and the performance of the ARM cores, the parallelism of the queues is increased, the overhead of the ARM cores is gradually increased, and particularly, the execution mode is simple for the NVM command, namely the IO data command, so that it is needed to design an NVMe controller capable of reducing the resource consumption and improving the parallelism of the queue execution.

Disclosure of Invention

In view of this, embodiments of the present application provide an NVMe controller to obviate or ameliorate one or more of the disadvantages of the prior art.

The application provides an NVMe controller, which comprises: one NVMe command subsystem and a plurality of NVM subsystems respectively connected to the control registers;

the NVMe command subsystem is used for realizing an admin queue based on hardware, executing an NVMe admin command and configuring a plurality of NVME IO queues;

the NVM subsystems are arranged in parallel, the NVM subsystems are multiplexing systems, each NVM subsystem is based on hardware to realize an NVMe IO queue, and the NVM subsystems are used for executing NVMe IO commands in parallel.

In some embodiments of the application, the NVMe command subsystem includes: the first handling analysis module group and the first processing interruption module group are connected with each other;

the first carrying and analyzing module group is in communication connection with the control register and is used for carrying and analyzing an NVMe admin command issued by the host to the interior of the NVMe controller through the control register;

the first processing interrupt module group is in communication connection with the host memory, and is used for processing the NVMe admin command, sending a corresponding processing result to the host memory and then performing queue interrupt processing.

In some embodiments of the application, the first transport resolution module group includes: the first doorbell arbitration module, the first command taking module and the first decoding module are connected in sequence;

the first doorbell arbitration module is in communication connection with the control register and is used for checking whether the current queue contains an NVMe admin command issued by the host through the control register;

the first command taking module is used for sending a corresponding DMA request to obtain the NVMe admin command when the first doorbell arbitration module detects that the current queue contains the NVMe admin command, and sending the NVMe admin command to the first decoding module;

the first decoding module is configured to parse the NVMe admin command, and determine an operation type of the NVMe admin command, so that the NVMe admin command is sent to a corresponding processing module in the first processing interrupt module group according to the operation type.

In some embodiments of the application, the first set of processing interrupt modules includes: the first processing module group and the first sending CQ and interrupt module are mutually connected;

the first processing module group is connected with the first decoding module and is used for processing NVMe admin commands of different operation types;

the first sending CQ and the interrupt module are in communication connection with the host memory, and are used for assembling the corresponding NVMe CQ according to the completion state of the NVMe admin command after the NVMe admin command is completed, sending the NVMe CQ to the host memory, and then sending the interrupt instruction of the corresponding queue.

In some embodiments of the application, the first set of processing modules includes: the device comprises an information processing module, a characteristic processing module and a queue processing module which are connected in sequence;

the information processing module, the characteristic processing module and the queue processing module are arranged in parallel, and the input ends of the information processing module, the characteristic processing module and the queue processing module are connected to the first decoding module; the output ends of the information processing module, the characteristic processing module and the queue processing module are connected to the first CQ and interrupt sending module;

the information processing module is used for executing an NVMe admin command with an operation type of information processing;

the feature processing module is used for executing an NVMe admin command with the operation type being feature processing;

the queue processing module is used for executing an NVMe admin command with an operation type of queue processing so as to create a corresponding NVME IO queue request and dynamically configuring an NVM subsystem to which the NVME IO queue request belongs.

In some embodiments of the application, each of the NVM subsystems shares a second doorbell arbitration module;

the second doorbell arbitration module is respectively connected with the control register and the NVMe command subsystem and is used for checking whether the host computer issues an NVME IO queue request or not through the control register and/or receiving the NVME IO queue request sent by the NVMe command subsystem;

the second doorbell arbitration module is further used for arbitrating the NVM subsystem to which the NVME IO queue request belongs after receiving or detecting the NVME IO queue request.

In some embodiments of the application, each of the NVM subsystems further comprises: the second transport analysis module group and the second processing interruption module group are connected with each other;

the second carrying analysis module group is connected with the second doorbell arbitration module and is used for carrying the IO command corresponding to the NVME IO queue request into the NVMe controller and analyzing the IO command;

the second processing interrupt module group is in communication connection with the host memory, and is used for processing the IO command, sending a corresponding processing result to the host memory and then performing queue interrupt processing.

In some embodiments of the application, the second transport resolution module group includes: the second command fetching module and the second decoding module are connected with each other;

the second command fetching module is configured to send a corresponding DMA request to obtain the IO command, and send the IO command to the second decoding module;

the second decoding module is configured to parse the IO command and send the IO command to the second processing interrupt module group.

In some embodiments of the application, the second set of processing interrupt modules includes: a second processing module and a second CQ and interrupt issuing module connected with each other;

the second processing module is connected with the second decoding module and is used for processing the IO command;

and the second CQ sending and interrupting module is in communication connection with the host memory and is used for assembling the corresponding NVMe CQ according to the completion state of the IO command after the IO command is executed, sending the NVMe CQ to the host memory and then sending the interrupting instruction of the corresponding queue.

In some embodiments of the present application, the sending the NVMe CQ to the host memory includes:

and generating and outputting a DMA request corresponding to the NVMe CQ to send the NVMe CQ to a host memory.

The NVMe controller provided by the application is provided with one NVMe command subsystem and a plurality of NVM subsystems which are respectively connected to the control register; the NVMe command subsystem is used for realizing an admin queue based on hardware, executing an NVMe admin command and configuring a plurality of NVME IO queues; the NVM subsystems are arranged in parallel, the NVM subsystems are multiplexing systems, each NVM subsystem is based on hardware to realize an NVMe IO queue, and each NVM subsystem is used for executing NVMe IO commands in parallel; compared with the design of the NVMe controller adopting the ARM core, the NVMe controller provided by the application has the advantages that firstly, the basic admin command is realized through hardware, a plurality of NVME IO queues can be configured through hardware, interaction with the ARM core is not needed, and the communication expense is reduced; secondly, an NVME IO queue is realized through hardware, so that NVMe IO commands can be executed in parallel, and the parallelism is improved; each IO subsystem can be multiplexed, can be dynamically expanded according to performance requirements, and meets corresponding bandwidths; and finally, the data IO access is realized by adopting hardware, compared with an ARM core, the delay is stable, the influence of an operating system and an algorithm is avoided, and the delay has higher stability.

Additional advantages, objects, and features of the application will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.

It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present application are not limited to the above-described specific ones, and that the above and other objects that can be achieved with the present application will be more clearly understood from the following detailed description.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate and together with the description serve to explain the application. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the application. Corresponding parts in the drawings may be exaggerated, i.e. made larger relative to other parts in an exemplary device actually manufactured according to the present application, for convenience in showing and describing some parts of the present application. In the drawings:

fig. 1 is a schematic flow chart of a command standard of NVMe in the prior art.

Fig. 2 is a schematic structural diagram of an NVMe controller according to an embodiment of the application.

Fig. 3 is a schematic structural diagram of an NVMe command subsystem 1 in an NVMe controller according to an embodiment of the application.

Fig. 4 is a schematic structural diagram of the NVM subsystem 2 in the NVMe controller according to an embodiment of the present application.

Fig. 5 is a schematic structural diagram of an NVMe controller with multiple paths configured by hardware according to an embodiment of the present application.

Reference numerals:

1. NVMe command subsystem;

2. an NVM subsystem;

3. a control register;

4. a first transport analysis module group;

41. a first doorbell arbitration module;

42. a first command fetching module;

43. a first decoding module;

5. a first set of processing interrupt modules;

51. a first set of processing modules;

511. an information processing module;

512. a feature processing module;

513. a queue processing module;

52. the first sending CQ and interrupt module;

6. a second doorbell arbitration module;

7. the second carrying analysis module group;

71. a second command fetching module;

72. a second decoding module;

8. a second processing interrupt module group;

81. a second processing module;

82. the second issues CQ and interrupt module.

Detailed Description

The present application will be described in further detail with reference to the following embodiments and the accompanying drawings, in order to make the objects, technical solutions and advantages of the present application more apparent. The exemplary embodiments of the present application and the descriptions thereof are used herein to explain the present application, but are not intended to limit the application.

It should be noted here that, in order to avoid obscuring the present application due to unnecessary details, only structures and/or processing steps closely related to the solution according to the present application are shown in the drawings, while other details not greatly related to the present application are omitted.

It should be emphasized that the term "comprises/comprising" when used herein is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.

It is also noted herein that the term "coupled" may refer to not only a direct connection, but also an indirect connection in which an intermediate is present, unless otherwise specified.

Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

Considering that the existing NVMe controller is composed of one or more ARM cores, the performance of the NVMe SSD has a great relationship with the number of ARM cores and the performance of the ARM cores, the parallelism of the queues is increased, and the overhead of the ARM cores is gradually increased, but according to the execution process of the NVMe command, especially the NVM command, namely the IO data command, the execution mode is simple, the advantage of hardware parallelism (hardware multi-channel) can be utilized, and the resource consumption is low (compared with the ARM cores).

When the application is designed for the NVMe controller, researchers find that although the NVMe admin commands are numerous, the hardware can dynamically open the multi-queue channel by realizing the commands (commands marked as manager in the NVMe1.3 protocol) which are necessary to be realized, so that the parallelism of queue execution is improved, the interaction cost of the admin commands and ARM cores is reduced, the performance of the NVMe controller is improved, and the application has lower delay and higher throughput.

Based on the above, the application aims to solve the problem that the parallelism improvement of the NVMe controller realized by the ARM verification is too high, and realize the design of the NVMe controller realized by hardware in a basic command set.

In one or more embodiments of the application, NVM represents an acronym for non-volatile memory (non-volatile memory), which is a common form of flash memory for solid-state disks (SSDs). The standard is mainly used for providing a low-delay and internal concurrency native interface standard for the flash memory-based storage device, and also providing a native storage concurrency support for a modern CPU, a computer platform and related applications, so that the parallelization energy storage capacity of the solid-state storage device can be fully utilized by host hardware and software. NVMe/NVMHCI reduces I/O operation latency, improves operands at the same time, larger capacity operation queues, etc., as compared to the AHCI of the previous mechanical Hard Disk Drive (HDD) era. NVM Express (NVMe), or nonvolatile memory host controller interface specification (english: non-Volatile Memory Host Controller Interface Specification, abbreviated: NVMHCIS), is a logical device interface specification. It is a bus transport protocol specification (equivalent to an application layer in a communication protocol) based on a device logic interface, similar to AHCI, for accessing non-volatile memory media (e.g., solid state drives employing flash memory) attached via a PCI Express (PCIe) bus.

Embodiments of the NVMe controller are described in detail below with reference to fig. 2 to 4.

The embodiment of the application provides an NVMe controller, which specifically comprises the following components:

one NVMe command subsystem 1 and a plurality of NVM subsystems 2 respectively connected to the control register 3;

the NVMe command subsystem 1 realizes an admin queue based on hardware, and is used for executing NVMe admin commands and configuring a plurality of NVME IO queues;

the NVM sub-systems 2 are arranged in parallel, the NVM sub-systems 2 are multiplexing systems, each NVM sub-system 2 is based on hardware to realize an NVMe IO queue, and each NVM sub-system 2 is used for executing NVMe IO commands in parallel.

Wherein, the admin queue is a management queue, the NVMe admin command is a basic instruction, and the NVMe IO queue is an input/output queue; NVMe IO commands refer to input/output commands.

As can be seen from the above description, the NVMe controller provided by the embodiment of the present application firstly implements a basic admin command through hardware, and can configure multiple NVMe IO queues through hardware, without interaction with an ARM core, so as to reduce communication overhead; secondly, an NVME IO queue is realized through hardware, so that NVMe IO commands can be executed in parallel, and the parallelism is improved; each IO subsystem can be multiplexed, can be dynamically expanded according to performance requirements, and meets corresponding bandwidths; and finally, the data IO access is realized by adopting hardware, compared with an ARM core, the delay is stable, the influence of an operating system and an algorithm is avoided, and the delay has higher stability.

In order to further improve the application reliability of the NVMe command subsystem 1, in the NVMe controller provided by the embodiment of the present application, the NVMe command subsystem 1 specifically includes the following contents:

a first transport analysis module group 4 and a first processing interrupt module group 5 connected to each other;

the first carrying and analyzing module group 4 is in communication connection with the control register 3, and is used for carrying and analyzing an NVMe admin command issued by the host to the interior of the NVMe controller through the control register 3;

the first processing interrupt module group 5 is in communication connection with the host memory, and is configured to process the NVMe admin command, send a corresponding processing result to the host memory, and then perform queue interrupt processing.

In order to further improve the application reliability of the first transport analysis module group 4, in the NVMe controller provided in the embodiment of the present application, the first transport analysis module group 4 specifically includes the following contents:

the first doorbell arbitration module 41, the first command fetching module 42 and the first decoding module 43 are connected in sequence;

the first doorbell arbitration module 41 is in communication connection with the control register 3, and is configured to check, via the control register 3, whether the current queue contains an NVMe admin command issued by the host;

the first command fetching module 42 is configured to issue a corresponding DMA request to obtain an NVMe admin command when the first doorbell arbitration module 41 detects that the current queue contains the NVMe admin command, and send the NVMe admin command to the first decoding module 43;

the first decoding module 43 is configured to parse the NVMe admin command and determine an operation type of the NVMe admin command, so as to send the NVMe admin command to a corresponding processing module in the first processing interrupt module group 5 according to the operation type.

In order to further improve the application reliability of the first processing interrupt module group 5, in the NVMe controller provided in the embodiment of the present application, the first processing interrupt module group 5 specifically includes the following contents:

a first processing module group 51 and a first issuing CQ and interrupt module 52 connected to each other;

the first processing module group 51 is connected to the first decoding module 43, and is configured to process NVMe admin commands with different operation types;

the first CQ sending and interrupting module 52 is communicatively connected to the host memory, and is configured to assemble a corresponding NVMe CQ according to a completion status of the NVMe admin command after the NVMe admin command is executed, send the NVMe CQ to the host memory, and send an interrupt instruction of the corresponding queue.

In order to further improve the application reliability of the first processing module group 51, in the NVMe controller provided in the embodiment of the present application, the first processing module group 51 specifically includes the following contents: an information processing module 511, a feature processing module 512, and a queue processing module 513 connected in this order;

the information processing module 511, the feature processing module 512 and the queue processing module 513 are arranged in parallel, and the input ends of the information processing module 511, the feature processing module 512 and the queue processing module 513 are all connected to the first decoding module 43; the output ends of the information processing module 511, the feature processing module 512 and the queue processing module 513 are all connected to the first sending CQ and interrupt module 52;

the information processing module 511 is configured to execute an NVMe admin command with an operation type of information processing;

the feature processing module 512 is configured to execute an NVMe admin command with an operation type being feature processing;

the queue processing module 513 is configured to execute an NVMe admin command with an operation type of queue processing, so as to create a corresponding NVMe IO queue request, and dynamically configure the NVM subsystem 2 to which the NVMe IO queue request belongs.

In order to further improve the application reliability of the NVM subsystem 2, in the NVMe controller provided by the embodiment of the present application, each NVM subsystem 2 shares a second doorbell arbitration module 6;

the second doorbell arbitration module 6 is respectively connected with the control register 3 and the queue processing module 513, and is configured to check whether the host issues an NVME IO queue request via the control register 3, and/or receive the NVME IO queue request sent by the NVME command subsystem 1;

the second ring arbitration module 6 is further configured to obtain, after receiving or detecting the NVME IO queue request, the NVM subsystem 2 to which the NVME IO queue request belongs, and send the NVME IO queue request to the corresponding NVM subsystem 2.

In order to further improve the application reliability of the NVM subsystem 2, in the NVMe controller provided by the embodiment of the present application, the NVM subsystem 2 further specifically includes the following contents:

a second transport analysis module group 7 and a second processing interrupt module group 8 connected to each other;

the second handling analysis module group 7 is connected with the second doorbell arbitration module 6, and is used for handling and analyzing the IO command corresponding to the NVME IO queue request to the interior of the NVMe controller;

the second processing interrupt module group 8 is in communication connection with the host memory, and is configured to process the IO command, send a corresponding processing result to the host memory, and then perform queue interrupt processing.

In order to further improve the application reliability of the second handling analysis module group 7, in the NVMe controller provided in the embodiment of the present application, the second handling analysis module group 7 further specifically includes the following contents:

a second command fetching module 71 and a second decoding module 72 connected to each other;

the second command fetching module 71 is configured to issue a corresponding DMA request to obtain the IO command, and send the IO command to the second decoding module 72;

the second decoding module 72 is configured to parse the IO command and send the IO command to the second processing interrupt module group 8.

In order to further improve the application reliability of the second processing interrupt module group 8, in the NVMe controller provided in the embodiment of the present application, the second processing interrupt module group 8 further specifically includes the following contents:

a second processing module 81 and a second issue CQ and interrupt module 82 connected to each other;

the second processing module 81 is connected to the second decoding module 72, and is configured to process the IO command;

the second CQ and interrupt module 82 is communicatively connected to the host memory, and configured to assemble a corresponding NVMe CQ according to a completion status of the IO command after the IO command is executed, send the NVMe CQ to the host memory, and then send an interrupt instruction of the corresponding queue. Where, NVMe CQ refers to completion queue.

In addition, in one or more embodiments described above, the specific implementation manner of sending the NVMe CQ to the host memory may be:

In order to further explain the scheme, the application also provides an application example of the NVMe controller, in particular to an NVMe controller with a hardware automatic configuration multichannel, wherein the whole NVMe controller is composed of two parts, an admin subsystem and a plurality of NVM subsystems (IO subsystem and NVME IO queue). The overall structure of the Admin subsystem is the same as that of the NVM subsystem, and the Admin subsystem consists of a doorbell arbitration module (admin_db, nvm_db), a command fetching module (admin_sq_fetch, nvm_sq_fetch), a decoding module (admin_decode, nvm_decode), a processing module (admin_info, admin_ feature, admin _queue, nvm_feature), and a sending cq and interrupt module (admin_cq, nvm_cq). When the host end issues a queue request, the doorbell arbitration module can receive a response, arbitrate a queue with a current request, send the request to the command fetching module, the module sends a DMA request, carries an NVMe admin command or an IO command into an NVMe controller, after the command carrying is completed, the decoding module is responsible for analyzing the NVMe command, the processing module carries out different operations according to different operation types, configures an NVME IO queue or read-write data, when the NVMe command is completed, the sending CQ and the interrupt module complete the state according to the NVMe module command, assembles the NVMe CQ, sends the CQ to a host memory through DMA, and then sends an interrupt of the corresponding queue after the sending is completed, and the whole process, hardware completes carrying, analysis and processing of the NVMe command without CPU participation of ARM and the like, thereby realizing improvement of performance, delay reduction and throughput improvement. The mapping relation between the software queues and the hardware queues adopts a hash method of taking a module, so that each software queue can be mapped to the corresponding hardware queue, and the parallelism of command execution is improved.

Specifically, for the NVMe admin subsystem, there is only one group of queues, the command in the queues is detected, namely the command is fetched to an analysis module, in the admin_queue module, an NVMe IO queue creating request is analyzed, an IO hardware queue to which the NVMe IO queue belongs is dynamically configured, and after execution is finished, corresponding CQ and interrupt are sent out.

The NVM subsystem, each individual queue function similarly, accepts command distribution of NVM _db, initiates execution of the command, decodes, executes, and finally assembles the CQ for this command and issues an interrupt.

Referring to fig. 5, the embodiment of the present application provides an NVMe controller with a hardware auto-configuration multi-path, which specifically includes the following contents:

( One) NVMe command subsystem 1 (may also be referred to as: nonvolatile memory host controller interface command Subsystem (Admin Subsystem) )

For the NVMe command subsystem 1, there is only one set of queues including a first doorbell arbitration module 41 (admin_db), a first command fetch module 42 (admin_sq_fetch), a first decode module 43 (admin_decode), a first processing module set 51, and a first issue CQ and interrupt module 52, which are connected in sequence.

Wherein the first processing module group 51 includes: an information processing module 511 (admin_info), a feature processing module 512 (admin_feature), and a queue processing module 513 (admin_queue).

Through the construction of the NVMe command subsystem 1, the basic admin command is realized through hardware, the NVMe admin command type pipeline is realized, a plurality of NVME IO queues can be configured through hardware, interaction with ARM cores is not needed, the interaction process overhead between the basic NVMe command and the ARM cores can be effectively reduced, and the communication overhead is reduced.

The concrete explanation is as follows:

(1) The first doorbell arbitration module 41 is configured to detect whether an NVMe admin command is present in the current queue through the control register 3 (nvme_reg).

(2) The first command fetching module 42 is configured to issue a DMA request, and to carry the NVMe admin command to the inside of the NVMe controller, that is, fetch the NVMe admin command to the first decoding module 43.

(3) The first decoding module 43 is configured to parse the NVMe admin command and determine an operation type of the NVMe admin command, so as to select one of the information processing module 511, the first feature processing module 512 and the queue processing module 513 in the first processing module group 51 as a target processing module according to the operation type, and send the NVMe admin command to the target processing module for processing.

(4) The information processing module 511 is configured to perform corresponding information processing after receiving an NVMe admin command with an operation type of information processing.

(5) The feature processing module 512 is configured to perform corresponding feature processing after receiving an NVMe admin command with an operation type being feature processing.

(6) The queue processing module 513 is configured to parse and create an NVMe IO queue request after receiving an NVMe admin command with an operation type of queue processing, and dynamically configure the IO hardware queue to which the NVMe IO queue request belongs.

(7) The first issue CQ and interrupt module 52 is configured to, after finishing executing the NVMe admin command, assemble the NVMe CQ according to the NVMe command completion status, send the completion queue CQ (completion queue) to the host memory through DMA, and issue an interrupt of the corresponding queue after the completion of the sending, where the whole process, hardware completes the handling, analysis and processing of the NVMe command, without the participation of CPU such as ARM, so as to achieve performance improvement, delay reduction, and throughput improvement. The mapping relation between the software queues and the hardware queues adopts a modulo hash method, so that each software queue can be mapped to the corresponding hardware queue, and the parallelism of command execution is improved.

( Two) NVM subsystem 2 (which may also be referred to as: nonvolatile memory Subsystem (NVM Subsystem) )

For the NVM subsystems 2, by implementing the NVME IO queue through hardware, the NVME IO commands can be executed in parallel, each NVM subsystem 2 shares the same second doorbell arbitration module 6 (NVM _db), and each NVM subsystem 2 further includes: the second fetch module 71 (nvm _sq_fetch), the second decode module 72 (nvm _decode), the second processing module 81 (nvm _feature), the second issue CQ and interrupt module 82 (nvm _cq).

Through the construction of the NVM subsystem 2, the NVME IO queue can be realized through hardware, and the NVMe IO command can be executed in parallel, so that the parallelism is improved; each IO subsystem can be multiplexed, can be dynamically expanded according to performance requirements, and meets corresponding bandwidths; and the data IO access can be realized by adopting hardware, compared with an ARM core, the delay is stable, the influence of an operating system and an algorithm is avoided, and the delay has higher stability.

The concrete explanation is as follows:

(1) And the second doorbell arbitration module 6 is configured to receive the response when detecting that the host side issues an NVMe IO queue request through the control register 3 (nvme_reg), arbitrate a queue currently having an NVMe IO queue request in each NVM subsystem 2, and send the request to the second command fetching module 71 in the queue.

The second ring arbitration module 6 may also be connected to the queue processing module 513, and configured to receive an NVMe IO queue request (which may also be referred to as an IO command) sent by the queue processing module 513 after the queue processing module 513 executes an NVMe admin command with an operation type of queue processing, arbitrate a queue currently having the NVMe IO queue request in each NVM subsystem 2, and send the request to the second command fetching module 71 in the queue.

(2) The second command fetching module 71 is configured to issue a DMA request, and to carry the IO command to the NVMe controller, that is, fetch the IO command to the second decoding module 72.

(3) The second decoding module 72 is configured to parse the IO command and send the IO command to the second information processing module 511 for processing.

(4) The second information processing module 511 is configured to perform corresponding processing after receiving the IO command.

(5) And the second issue CQ and interrupt module 82 is configured to, after finishing executing the IO command, assemble the NVMe CQ according to the IO command completion status, send the CQ to the host memory through DMA, and issue an interrupt of the corresponding queue after the completion of the sending, where the whole process is completed by hardware, and the hardware completes the handling, analysis and processing of the NVMe command, without the participation of CPU such as ARM, so as to achieve performance improvement, delay reduction, and throughput improvement. The mapping relation between the software queues and the hardware queues adopts a modulo hash method, so that each software queue can be mapped to the corresponding hardware queue, and the parallelism of command execution is improved.

Therefore, the NVMe controller provided by the application at least comprises the following improvements:

(1) NVMe admin command class pipeline implementation; the technical effect is that the cost of the interaction process between the basic NVMe command and the ARM core is reduced;

(2) Hardware realizes an NVMe IO queue and an admin queue; the technical effect is that the overhead of ARM core control data path is removed, and the time delay is reduced;

(3) Multiple NVME IO queues are processed in parallel; the technical effect is that IO command processing capability is improved, and parallelism is improved.

In summary, compared with the design of the NVMe controller adopting the ARM core, the hardware-based multi-path NVMe controller provided by the application example of the application has the advantages that firstly, the basic admin command is realized through hardware, a plurality of NVME IO queues can be configured through hardware, interaction with the ARM core is not needed, and the communication overhead is reduced; secondly, an NVME IO queue is realized through hardware, so that NVMe IO commands can be executed in parallel, and the parallelism is improved; each IO subsystem can be multiplexed, can be dynamically expanded according to performance requirements, and meets corresponding bandwidths; and finally, the data IO access is realized by adopting hardware, compared with an ARM core, the delay is stable, the influence of an operating system and an algorithm is avoided, and the delay has higher stability.

It should be understood that the application is not limited to the particular arrangements and instrumentality described above and shown in the drawings. A detailed description of known structures is omitted herein for the sake of brevity. In the above embodiments, several specific structures are described and shown as examples. However, the device structure of the present application is not limited to the specific structure described and shown, and various changes, modifications and additions may be made by those skilled in the art after appreciating the spirit of the present application.

The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, and various modifications and variations can be made to the embodiments of the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. An NVMe controller for implementing a data IO path in hardware without interaction with an ARM core, the NVMe controller comprising: one NVMe command subsystem and a plurality of NVM subsystems respectively connected to the control registers;

the NVM subsystems are arranged in parallel, the NVM subsystems are multiplexing systems, each NVM subsystem is based on hardware to realize an NVMe IO queue, and each NVM subsystem is used for executing NVMe IO commands in parallel;

the NVMe command subsystem includes: the first handling analysis module group and the first processing interruption module group are connected with each other;

the first processing interrupt module group is in communication connection with the host memory, and is used for processing the NVMe admin command, sending a corresponding processing result to the host memory and then performing queue interrupt processing;

the first handling analysis module group includes: the first doorbell arbitration module, the first command taking module and the first decoding module are connected in sequence;

the first command fetching module is configured to send a corresponding DMA request to obtain an NVMe admin command when the first doorbell arbitration module detects that the current queue contains the NVMe admin command, and send the NVMe admin command to the first decoding module;

2. The NVMe controller of claim 1, wherein the first set of processing interrupt modules comprises: the first processing module group and the first sending CQ and interrupt module are mutually connected;

the first processing module group is connected with the first decoding module and is used for processing NVMeadmin commands of different operation types;

3. The NVMe controller of claim 2, wherein the first set of processing modules comprises: the device comprises an information processing module, a characteristic processing module and a queue processing module which are connected in sequence;

4. The NVMe controller of claim 1, wherein each of the NVM subsystems shares a second doorbell arbitration module;

5. The NVMe controller of claim 4, wherein each NVM subsystem further comprises: the second transport analysis module group and the second processing interruption module group are connected with each other;

6. The NVMe controller of claim 5, wherein the second transport resolution module set comprises: the second command fetching module and the second decoding module are connected with each other;

7. The NVMe controller of claim 6, wherein the second set of processing interrupt modules comprises: a second processing module and a second CQ and interrupt issuing module connected with each other;

8. The NVMe controller of claim 2 or 7, wherein the sending the NVMe CQ to host memory comprises: