WO2013152593A1 - 数据处理方法及装置 - Google Patents
数据处理方法及装置 Download PDFInfo
- Publication number
- WO2013152593A1 WO2013152593A1 PCT/CN2012/085126 CN2012085126W WO2013152593A1 WO 2013152593 A1 WO2013152593 A1 WO 2013152593A1 CN 2012085126 W CN2012085126 W CN 2012085126W WO 2013152593 A1 WO2013152593 A1 WO 2013152593A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- address
- vhba
- written
- request instruction
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
Definitions
- the present invention relates to the field of storage, and in particular, to a data processing method and apparatus.
- Fibre Channel Fibre Channel
- iSCSI Internet SCSI
- the prior art generally uses a virtual machine solution to quickly integrate Server and Storage, which is a complete computer system that runs through a software and has a complete hardware system function and runs in a completely isolated environment.
- Server and Storage is a complete computer system that runs through a software and has a complete hardware system function and runs in a completely isolated environment.
- virtual machines can be configured with virtual hardware, operating systems, and so on.
- the virtual machine can be used to implement the unification of the server and the storage in one physical host.
- the server and the storage respectively correspond to one virtual machine
- Also need to assign two virtual machines Memory will consume a lot of system resources.
- the memory copy during data transfer between Server and Storage will also consume a lot of system resources, thus affecting system performance.
- the embodiment of the invention provides a data processing method and device, which can enable the server and the storage program to run under the same operating system, reduce resource consumption, and use the virtual channel to transmit the address of the processing data when the data is transmitted, thereby reducing Memory copy, improve systemic
- the first application module After receiving the write request command, the first application module writes the data to be written into the first data buffer, and sends the write request instruction and the address of the first data buffer to the first virtual host bus.
- the write request instruction includes an address of the to-be-written data to be written to the storage medium;
- the first VHBA sends the write request instruction and the address of the first data buffer to the first storage array module
- the first storage array module After receiving the write request instruction, acquires the data to be written from the first data buffer according to the address of the first data buffer, and according to the The address to be written to be written to the storage medium writes the data to be written into the storage medium;
- the first storage array module returns a response of the write request instruction to the first VHBA, and the first VHBA returns a response of the write request instruction to the first application module.
- Another data processing method provided by the embodiment of the present invention includes:
- the second application module sends the address of the read request instruction and the second data buffer to the second
- the read request instruction includes an address of the data to be read in the storage medium
- the second VHBA sends the read request instruction and the address of the second data buffer Sending to the second storage array module
- the second storage array module acquires data to be read according to an address of the data to be read carried in the storage medium, and then according to an address of the second data buffer, The data to be read is written into the second data buffer;
- the second memory array module returns a response of the read request instruction to the second VHBA, and the second VHBA returns a response of the read request instruction to the application module.
- the data processing apparatus includes: a first application module, a first VHBA, a first data buffer and a first storage array module; and the first application module is to be written after receiving the write request command Writing in data to the first data buffer, and transmitting a write request instruction and an address of the first data buffer to the first VHBA, the write request instruction including the to-be-written data to be The address written to the storage medium;
- the first VHBA sends the write request instruction and the address of the first data buffer to the first storage array module
- the first storage array module After receiving the write request instruction, acquires the data to be written from the first data buffer according to the address of the first data buffer, and according to the The address to be written to be written to the storage medium writes the data to be written into the storage medium;
- the first storage array module returns a response of the write request instruction to the first VHBA, and the first VHBA returns a response of the write request instruction to the first application module.
- Another data processing apparatus includes: a second application module, a second VHBA, a second data buffer, and a second storage array module;
- the second application module is configured to send a read request instruction and an address of the second data buffer to a second VHBA, where the read request instruction includes an address of the data to be read in the storage medium;
- the second VHBA is configured to send the read request instruction and the address of the second data buffer to the second storage array module;
- the second storage array module is configured to acquire data to be read according to an address of the data to be read in the storage medium, and then write the data to be read according to an address of the second data buffer.
- the second data buffer is configured to acquire data to be read according to an address of the data to be read in the storage medium, and then write the data to be read according to an address of the second data buffer.
- the second memory array module is further configured to return a response of the read request instruction to the second VHBA, and the second VHBA returns a response of the read request instruction to the second application module.
- the application module and the storage array module are in the same physical host and connected through the VHBA card, a set of operating systems can be shared, and only the address of the data buffer and the to-be-written are required to be written when writing data. Data, reduced memory copying, improves system performance.
- FIG. 1 is a flow chart of a data processing method according to an embodiment of the present invention.
- FIG. 2 is a flow chart of another data processing method according to an embodiment of the present invention.
- FIG. 3 is a flowchart of still another data processing method according to an embodiment of the present invention.
- FIG. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention.
- FIG. 5 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present invention. detailed description
- the first application module may include various applications and drivers, and exists in a physical host in the form of an application software stack.
- the first storage array module includes a target target and array control software, and the target is used by The command sent by the first VHBA is received, and the command is parsed into a command that the storage array software can recognize and sent to the storage array software.
- the first application module and the first storage array module are integrated in one physical host, and are connected by a first virtual host bus adapter card VHBA (Virtual Host Bus Adapter), and the first VHBA makes the first application.
- VHBA Virtual Host Bus Adapter
- the first application module After receiving the write request command, the first application module writes the data to be written into the first data buffer, and sends the write request instruction and the address of the first data buffer to the first virtual host bus.
- the write request instruction includes an address of the to-be-written data to be written to the storage medium;
- the write request command includes an address to be written and an address to be written to be written to the storage medium, and the write request instruction includes an address to be written to the storage medium, not including the data to be written.
- the first application module may be an application software stack in the physical host, including various application software and drivers; the first storage array module may be an array software stack in the physical host, including Target and various array controls.
- Software, the first application module and the first storage array module are connected by a first VHBA, the first VHBA is a software simulated HBA card, has the function of a physical HBA card, and includes a first VHBA driver in the application software stack. .
- the first data buffer may be a memory in the physical host.
- the first VHBA sends the write request instruction and the address of the first data buffer to the first storage array module.
- the first storage array module returns a response of the write request instruction to the first VHBA, and the first VHBA returns a response of the write request instruction to the first application module.
- the application virtual machine and the storage virtual machine respectively use different operating systems, so the method of transferring addresses cannot be used in data transfer, and only the way of copying the memory can be used.
- the application virtual machine and the storage virtual machine respectively occupy a part of the host memory.
- the application virtual machine needs to write data to the storage medium
- the data to be written is first written into the memory corresponding to the application virtual machine, and the application virtual machine will
- the write data command is sent to the storage virtual machine
- the data to be written in the memory is copied to the memory corresponding to the storage virtual machine, and the storage virtual machine acquires the data to be written from the memory, and then writes the data to the storage virtual machine.
- Memory copying between the application virtual machine and the storage virtual machine consumes a lot of system resources and affects system performance.
- the first VHBA card is connected, and a set of operating systems can be shared, and only the first data needs to be transmitted when writing data.
- the address of the punch area and the data to be written reduce the memory copy and improve system performance.
- the virtual machine solution mainly supports accessing the storage system through the iSCSI, and the FC mode is not supported.
- the VHBA connection between the application module and the storage array module can be supported.
- the FC mode accesses the storage system.
- FIG. 2 it is a data processing method disclosed by another embodiment of the present invention.
- the second application module sends the address of the read request instruction and the second data buffer to the second VHBA, where the read request instruction includes an address of the data to be read in the storage medium;
- the second VHBA sets the read request instruction and the second data buffer The address is sent to the second storage array module
- the second storage array module acquires data to be read according to an address of the data to be read carried in the storage medium, and then according to an address of the second data buffer. Writing the data to be read into the second data buffer;
- the second storage array module returns a response of the read request instruction to the second VHBA, and the second VHBA returns a response of the read request instruction to the application module.
- the read data request is sent to the storage virtual machine, and after the storage virtual machine obtains the data to be read according to the read data request, the data to be read is written into the memory.
- the data to be read in the memory is copied into the memory of the application virtual machine. Memory copying between the application virtual machine and the storage virtual machine consumes a lot of system resources and affects system performance.
- the second application module may include various applications and drivers, and exists in a physical host in the form of an application software stack.
- the second storage array module includes Target and array control software, and Target is used to receive the first
- the command sent by the second VHBA, and the command is parsed into a command that the storage array software can recognize and sent to the storage array software.
- the second application module and the second storage array module share a set of operating systems in the same physical host, and only the address of the data buffer and the address of the data to be read are transmitted when the data is read, so the embodiment of the present invention
- the integration of the application host and the storage system is realized, the memory copy is reduced, and the system performance is improved.
- FIG. 3 in the following, a specific embodiment is used to illustrate that the present invention only needs to pass the address of the data buffer and the address of the data to be read when reading data, which can reduce memory copy and improve system performance.
- the second application module needs to read a data of size 512 kb, and send the address of the storage medium where the data is located and the address of the data buffer to the VHBA;
- the address of the storage medium where the data is located and the address of the data buffer are equal to the offset plus the length Degree, the total size is 16 bytes;
- the second VHBA sends the address of the storage medium where the data is 16 bytes and the address of the data buffer to the second storage array module.
- the second storage array module acquires data to be read according to the address of the data to be read in the storage medium, and then writes the data to be read into the data according to the address of the data buffer. Punching area
- the second storage array module returns a response of the read request instruction to the second VHBA, and the second VHBA returns a response of the read request instruction to the second application module.
- the second application module and the second target module only need to transmit the address of 16 bytes through the VHBA, which greatly reduces the memory copy and saves system resources.
- the unified storage system refers to a network storage architecture, which supports both file-based NAS (Network Attached Storage) storage and support based on The storage area of the block (S AN (Storage Area Network)).
- NAS Network Attached Storage
- S AN Storage Area Network
- the second application module and the second storage array module are integrated into one physical host, and the second application module may include a file system and a file sharing protocol, and the shared file system may be provided externally.
- the second storage array module can directly process the SCSI command and provide the block service to the outside. Therefore, the embodiment of the present invention can be applied to the unified storage scenario.
- FIG. 4 it is a data processing apparatus according to an embodiment of the present invention.
- the first application module and the first storage array module are integrated in one physical host, and are connected through a first virtual host bus adapter card VHBA (Virtual Hose Bus Adapter).
- VHBA Virtual Hose Bus Adapter
- the first VHBA forms a virtual channel between the first application module and the first storage array module, sharing an operating system.
- the data processing device includes: a first application module 201, a first VHBA 202, a first storage array module 203, and a first data buffer 204; After receiving the write request command, the first application module 201 writes the data to be written into the first data buffer 204, and writes the request command and the address of the first data buffer 204.
- the write request instruction includes an address that the data to be written is to be written into the storage medium;
- the first application module 201 may be an application software stack in the physical host, including various application software And a driver;
- the first storage array module may be an array software stack in the physical host, including a Target and various array control software, and the Target is configured to receive a write request command or a read request command sent by the VHBA, and the instruction is After parsing the commands that the array control software can recognize, it is forwarded to the array control software.
- the first application module and the first storage array module are connected by the first VHBA 202.
- the first VHBA 202 is a software-simulated HBA card, and has the function of a physical HBA card.
- the application software stack further includes a driver of the first VHBA 202.
- the first VHBA 202 is configured to send the write request instruction and the address of the first data buffer 204 to the first storage array module 203;
- the first storage array module 203 After receiving the write request instruction, the first storage array module 203 obtains the data to be written from the data buffer 204 according to the address of the first data buffer 204, and then according to the Describe that the address to be written to the storage medium is to be written into the storage medium;
- the first memory array module 203 is further configured to return a response of the write request instruction to the first VHBA 202, and the first VHBA 202 returns a response of the write request instruction to the first application module 201.
- the apparatus since the first application module 201 and the first storage array module 203 are connected in the same physical host through the first VHBA card 202, a set of operating systems can be shared, and only the first data transfer is required when writing data.
- the address of a data buffer 204 and the data to be written reduce the memory copy and improve system performance.
- 5 is a data processing apparatus according to an embodiment of the present invention, the apparatus includes: a second application module 301, a second VHBA 302, a second storage array module 303, and a second data buffer 304;
- the second application module 301 is configured to send the read request instruction and the address of the second data buffer 304 to the second VHBA 302, where the read request instruction includes an address of the data to be read in the storage medium;
- the second VHBA 302 is configured to send the read request instruction and the address of the second data buffer 304 to the second storage array module 303;
- the second storage array module 303 is further configured to acquire data to be read according to an address of the data to be read in the storage medium, and then to read the address according to the address of the second data buffer 304. Data is written into the second data buffer 304;
- the second storage array module 303 is further configured to return a response of the read request instruction to the second VHBA 302, and the second VHBA 302 returns a response of the read request instruction to the second application module 301.
- the second application module 301 and the second storage array module 302 share a set of operating systems in the same physical host, and only pass the address of the data buffer 304 and the to-be-readed when reading data. The address of the data, therefore, the embodiment of the invention not only realizes the fusion of the application host and the storage system, reduces the memory copy, and improves the system performance.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
- Memory System (AREA)
Abstract
本发明实施例提供了一种数据处理方法,包括:应用模块将待写入数据写入数据缓冲区,并将写请求命令和所述数据缓冲区的地址发送给虚拟主机总线适配卡VHBA,所述写请求命令包括所述待写入数据将要写入存储介质的地址;所述VHBA将所述写请求命令和所述数据缓冲区的地址发送给存储阵列模块;所述存储阵列模块从所述数据缓冲区的地址中获取所述待写入数据,再根据所述待写入数据将要写入存储介质的地址将所述待写入数据写入存储介质中。在本发明实施例中,由于应用模块和存储阵列模块在同一个物理主机内,通过VHBA卡连接,可以共用一套操作***,在写数据时仅需传递数据缓冲区的地址和待写入数据,减少了内存拷贝,提升了***性能。
Description
数据处理方法及装置 本申请要求于 2012年 4月 11 日提交中国专利局、 申请号为
201110439201.8、 发明名称为 "数据处理方法及装置" 的中国专利申请的优 先权, 其全部内容通过引用结合在本申请中。 技术领域 本发明涉及存储领域, 特别涉及一种数据处理方法及装置。
背景技术 目前服务器(Server ) 与存储设备 ( Storage )之间一般釆用 FC ( Fiber Channel, 光纤通道)或者 iSCSI ( internet SCSI )连接, 基于 SCSI协议实 现 Server和 Storage的数据交互。 当 Server需要读取 Storage上的数据时, 下发 SCSI读取命令, 由 Server上连接的 HBA ( Host Bus Adapter, 主机总 线适配)卡将命令转化为相应的光、 电信号, 经过光纤或以太网链路传送 到 Storage, Storage再将接收到的信号转换成小型计算机***接口 (Small Computer System Interface, SCSI )命令并进行处理后沿原通道将该命令的响 应结果返回给 Server, 从而完成一次读写操作。
随着 CPU处理能力越来越强大, 核数越来越多, 把 Server和 Storage 放到一个硬件***中的趋势越来越强烈, 它不仅可以带来成本和管理方面 的优势, 同时对功耗的降低也具有积极意义。
现有技术一般釆用虚拟机方案将 Server和 Storage快速地结合起来,虚 拟机是通过软件模拟的具有完整硬件***功能的、 运行在一个完全隔离环 境中的完整计算机***。 通常, 虚拟机可以根据需要配置虚拟硬件、 安装 操作***等。
使用虚拟机可以在一个物理主机内实现 Server和 Storage的统一,但由 于在一个物理主机内存在两个虚拟机 ( Server和 Storage分别对应一个虚拟 机), 因此需要通过软件模拟两个操作***运行, 也需要给两个虚拟机分配
内存, 将消耗大量***资源。 同时, 在 Server和 Storage之间进行数据传输 时的内存拷贝也将消耗大量***资源, 因此对***性能造成影响。 发明内容
本发明实施例提供了一种数据处理方法及装置, 可以使得 Server 和 Storage在同一个操作***下运行程序, 减少资源的消耗, 并且在进行数据 传输时, 利用虚拟通道传递处理数据的地址, 减少内存拷贝, 提高***性
•6匕
匕。
本发明实施例提供的数据处理方法, 包括:
第一应用模块接收到写请求命令后, 将待写入数据写入第一数据緩 冲区, 并将写请求指令和所述第一数据緩冲区的地址发送给第一虚拟主 机总线适配卡 VHBA, 所述写请求指令包括所述待写入数据将要写入存 储介质的地址;
所述第一 VHBA将所述写请求指令和所述第一数据緩冲区的地址发 送给第一存储阵列模块;
所述第一存储阵列模块接收到所述写请求指令后, 根据所述第一数 据緩冲区的地址, 从所述第一数据緩冲区中获取所述待写入数据, 再根 据所述待写入数据将要写入存储介质的地址将所述待写入数据写入存储 介质中;
所述第一存储阵列模块向所述第一 VHBA返回所述写请求指令的响 应, 所述第一 VHBA向所述第一应用模块返回所述写请求指令的响应。 本发明实施例提供的另一个数据处理方法, 包括:
第二应用模块将读请求指令和第二数据緩冲区的地址发送给第二
VHBA, 所述读请求指令包括待读取数据在存储介质中的地址;
所述第二 VHBA将所述读请求指令和所述第二数据緩冲区的地址发
送给第二存储阵列模块;
所述第二存储阵列模块根据所述读请求指令中携带的所述待读取数 据在存储介质中的地址, 获取待读取数据, 再根据所述第二数据緩冲区 的地址将所述待读取数据写入所述第二数据緩冲区;
所述第二存储阵列模块向所述第二 VHBA 返回所述读请求指令的响 应, 所述第二 VHBA向所述应用模块返回所述读请求指令的响应。
本发明实施例提供的数据处理装置, 包括: 第一应用模块、 第一 VHBA,第一数据緩冲区和第一存储阵列模块; 所述第一应用模块接收到 写请求命令后, 将待写入数据写入所述第一数据緩冲区, 并将写请求指 令和所述第一数据緩冲区的地址发送给所述第一 VHBA, 所述写请求指 令包括所述待写入数据将要写入存储介质的地址;
所述第一 VHBA将所述写请求指令和所述第一数据緩冲区的地址发 送给所述第一存储阵列模块;
所述第一存储阵列模块接收到所述写请求指令后, 根据所述第一数 据緩冲区的地址, 从所述第一数据緩冲区中获取所述待写入数据, 再根 据所述待写入数据将要写入存储介质的地址将所述待写入数据写入存储 介质中;
所述第一存储阵列模块向所述第一 VHBA 返回所述写请求指令的响 应, 所述第一 VHBA向所述第一应用模块返回所述写请求指令的响应。
本发明实施例提供的另一个数据处理装置, 包括: 第二应用模块、 第 二 VHBA、 第二数据緩冲区和第二存储阵列模块;
所述第二应用模块用于将读请求指令和所述第二数据緩冲区的地址 发送给第二 VHBA, 所述读请求指令包括待读取数据在存储介质中的地 址;
所述第二 VHBA用于将所述读请求指令和所述第二数据緩冲区的地 址发送给第二存储阵列模块;
所述第二存储阵列模块用于根据所述待读取数据在存储介质中的地 址, 获取待读取数据, 再根据所述第二数据緩冲区的地址将所述待读取 数据写入所述第二数据緩冲区;
所述第二存储阵列模块还用于向所述第二 VHBA返回所述读请求指令 的响应,所述第二 VHBA向所述第二应用模块返回所述读请求指令的响应。 在本发明实施例中, 由于应用模块和存储阵列模块在同一个物理主机内, 通过 VHBA卡连接, 可以共用一套操作***, 在写数据时仅需传递数据緩 冲区的地址和待写入数据, 减少了内存拷贝, 提升了***性能。 附图说明
为了更清楚地说明本发明实施例中的技术方案, 下面将对实施例描述 中所需要使用的附图作简单地介绍, 显而易见地, 下面描述中的附图仅仅 是本发明的一些实施例, 对于本领域普通技术人员来讲, 在不付出创造性 劳动的前提下, 还可以根据这些附图获得其他的附图。
图 1 是本发明实施方式的一个数据处理方法流程图;
图 2 是本发明实施方式的另一个数据处理方法流程图;
图 3 是本发明实施方式的再一个数据处理方法流程图;
图 4 是本发明实施方式中一个数据处理装置结构示意图;
图 5是本发明实施方式中另一个数据处理装置结构示意图。 具体实施方式
为使本发明实施例的目的、 技术方案和优点更加清楚, 下面将结合本 发明实施例中的附图, 对本发明实施例中的技术方案进行清楚、 完整地描 述, 显然, 所描述的实施例是本发明一部分实施例, 而不是全部的实施例。 基于本发明中的实施例, 本领域普通技术人员在没有作出创造性劳动前提 下所获得的所有其他实施例, 都属于本发明保护的范围。
参见图一, 是本发明一个实施方式所公开的数据处理方法;
在本发明实施例中, 第一应用模块可以包括各种应用程序和驱动程 序, 以应用软件栈的形式存在于物理主机中, 第一存储阵列模块包括目标 器 Target和阵列控制软件, Target用于接收第一 VHBA发送的命令, 并将 该命令解析为存储阵列软件可以识别的命令, 并发送给存储阵列软件。 第 一应用模块和第一存储阵列模块被集成在一个物理主机内, 通过第一虚拟 主机总线适配卡 VHBA (虚拟主机总线适配卡, Virtual Host Bus Adapter ) 连接, 第一 VHBA使得第一应用模块和第一存储阵列模块之间形成一个虚 拟通道, 共用一个操作***。
S101 : 第一应用模块接收到写请求命令后, 将待写入数据写入第一 数据緩冲区, 并将写请求指令和所述第一数据緩冲区的地址发送给第一 虚拟主机总线适配卡 VHBA, 所述写请求指令包括所述待写入数据将要 写入存储介质的地址;
其中, 写请求命令包括待写入数据以及待写入数据将要写入存储介 质的地址, 而写请求指令则包括待写入数据将要写入存储介质的地址, 不包括待写入数据本身。
第一应用模块在该物理主机可以是一个应用软件栈, 其中包括各种 应用软件及驱动程序; 第一存储阵列模块在该物理主机内可以是一个阵 列软件栈, 其中包括 Target和各种阵列控制软件, 第一应用模块和第一 存储阵列模块之间通过第一 VHBA连接,第一 VHBA是软件模拟的 HBA 卡, 具有物理 HBA卡的功能, 在应用软件栈中还包括第一 VHBA的驱 动程序。
在本发明实施例中, 第一数据緩冲区可以是该物理主机内的内存。
S102: 所述第一 VHBA将所述写请求指令和所述第一数据緩冲区的 地址发送给第一存储阵列模块;
S103: 所述第一存储阵列模块接收到所述写请求指令后, 根据所述
第一数据緩冲区的地址, 从所述第一数据緩冲区中获取所述待写入数据 , 再根据所述待写入数据将要写入存储介质的地址将所述待写入数据写入 存储介质中;
S104: 所述第一存储阵列模块向所述第一 VHBA返回所述写请求指 令的响应, 所述第一 VHBA向所述第一应用模块返回所述写请求指令的响 应。
在现有技术中, 应用虚拟机和存储虚拟机分别使用不同的操作***, 因此在数据传递时不能釆用传递地址的方式, 而只能釆用内存拷贝的方 式。 具体来说, 应用虚拟机和存储虚拟机分别占用一部分主机内存, 当 应用虚拟机需要将数据写入存储介质时, 先将待写入数据写入该应用虚 拟机对应的内存, 应用虚拟机将写数据命令发送给存储虚拟机时, 将其 内存中的待写入数据拷贝到存储虚拟机对应的内存中, 存储虚拟机从其 内存中获取该待写入数据之后, 再将其写入存储介质中。 而在应用虚拟 机和存储虚拟机之间进行内存拷贝将消耗大量的***资源, 对***性能 造成影响。
在本发明实施例中, 由于第一应用模块和第一存储阵列模块在同一个 物理主机内, 通过第一 VHBA卡连接, 可以共用一套操作***, 在写数据 时仅需传递第一数据緩冲区的地址和待写入数据, 减少了内存拷贝, 提升 了***性能。
此外,在现有技术中,釆用虚拟机方案主要支持通过 iSCSI访问存储系 统,基本不支持 FC方式, 而在本发明实施例中由于应用模块和存储阵列模 块之间通过 VHBA连接, 可以支持以 FC方式访问存储***。
参见图二, 是本发明另一个实施方式所公开的数据处理方法;
S105: 第二应用模块将读请求指令和第二数据緩冲区的地址发送给 第二 VHBA, 所述读请求指令包括待读取数据在存储介质中的地址;
S106: 所述第二 VHBA将所述读请求指令和所述第二数据緩冲区的
地址发送给第二存储阵列模块;
S107: 所述第二存储阵列模块根据所述读请求指令中携带的所述待 读取数据在存储介质中的地址, 获取待读取数据, 再根据所述第二数据 緩冲区的地址将所述待读取数据写入所述第二数据緩冲区;
S108: 所述第二存储阵列模块向所述第二 VHBA返回所述读请求指 令的响应, 所述第二 VHBA向所述应用模块返回所述读请求指令的响应。
在现有技术中, 当应用虚拟机需要读取数据时, 将读数据请求发送 给存储虚拟机, 存储虚拟机根据读数据请求获得待读取数据之后, 将该 待读取数据写入内存中, 在响应应用虚拟机的读数据请求时, 再将其内 存中的待读取数据拷贝到应用虚拟机的内存中。 而在应用虚拟机和存储 虚拟机之间进行内存拷贝将消耗大量的***资源, 对***性能造成影响。
在本发明实施例中, 第二应用模块可以包括各种应用程序和驱动程 序, 以应用软件栈的形式存在于物理主机中, 第二存储阵列模块包括 Target 和阵列控制软件, Target用于接收第二 VHBA发送的命令, 并将该命令解 析为存储阵列软件可以识别的命令, 并发送给存储阵列软件。 第二应用模 块和第二存储阵列模块在同一个物理主机内, 共用一套操作***, 在读取 数据时仅传递数据緩冲区的地址和待读取数据的地址, 因此本发明实施例 既实现了应用主机和存储***的融合, 减少了内存拷贝, 提升了***性能。
请参见图 3 ,下面以一个具体的实施例来说明本发明在读取数据时仅需 传递数据緩冲区的地址和待读取数据的地址, 可以减少内存拷贝, 提升系 统性能。
S401 : 第二应用模块需读取一个大小为 512kb的数据, 将该数据所在 存储介质的地址以及数据緩冲区的地址发送给 VHBA;
该数据所在存储介质的地址以及数据緩冲区的地址均等于偏移加上长
度, 大小总共为 16字节;
S402:第二 VHBA将大小为 16字节的该数据所在存储介质的地址以及 数据緩冲区的地址发送给第二存储阵列模块;
S403: 第二存储阵列模块根据所述待读取数据在存储介质中的地址, 获取待读取数据, 再根据所述数据緩冲区的地址将所述待读取数据写入 所述数据緩冲区;
S404: 所述第二存储阵列模块向所述第二 VHBA返回所述读请求指令 的响应,所述第二 VHBA向所述第二应用模块返回所述读请求指令的响应。
由此可见, 在本发明实施例中, 第二应用模块和第二目标模块只需要 通过 VHBA传递大小为 16字节的地址, 大大减少了内存拷贝,节省了*** 资源。
在本发明另一个实施例中, 可以应用在统一存储***之中, 统一存储 ***是指一种网络存储架构, 它既支持基于文件的 NAS ( Network Attached Storage,网络附加存储)存储,又支持基于块的 S AN ( Storage Area Network, 存储区域网络)存储。
具体的, 在本发明实施例中, 第二应用模块与第二存储阵列模块被集 成在一个物理主机内, 第二应用模块可以包括文件***以及文件共享协议, 可对外提供共享文件***, 而第二存储阵列模块可以直接处理 SCSI命令, 对外提供块服务, 因此本发明实施例可以应用在统一存储的场景之中。
参见图四, 是本发明一个实施方式所公开的数据处理装置;
在本发明实施例中, 第一应用模块和第一存储阵列模块被集成在一个 物理主机内 , 通过第一虚拟主机总线适配卡 VHBA (虚拟主机总线适配卡, Virtual Hose Bus Adapter )连接, 第一 VHBA使得第一应用模块和第一存储 阵列模块之间形成一个虚拟通道, 共用一个操作***。
数据处理装置包括: 第一应用模块 201、 第一 VHBA202、 第一存储阵 列模块 203和第一数据緩冲区 204;
其中, 所述第一应用模块 201 接收到写请求命令后, 将待写入数据 写入所述第一数据緩冲区 204, 并将写请求指令和所述第一数据緩冲区 204的地址发送给所述第一 VHBA202, 所述写请求指令包括所述待写入 数据将要写入存储介质的地址; 第一应用模块 201 在该物理主机可以是一个应用软件栈, 其中包括各 种应用软件及驱动程序; 第一存储阵列模块在该物理主机内可以是一个阵 列软件栈, 其中包括 Target和各种阵列控制软件, Target用于接收 VHBA 发送的写请求命令或读请求指令, 并将该指令解析为阵列控制软件可以识 别的命令之后, 再转发给阵列控制软件。 第一应用模块和第一存储阵列模 块之间通过第一 VHBA202连接, 第一 VHBA202是软件模拟的 HBA卡, 具有物理 HBA卡的功能, 在应用软件栈中还包括第一 VHBA202的驱动程 序。
第一 VHBA202用于将所述写请求指令和所述第一数据緩冲区 204的地 址发送给第一存储阵列模块 203;
第一存储阵列模块 203 用于接收到所述写请求指令后, 根据所述第一 数据緩冲区 204的地址, 从所述数据緩冲区 204中获取所述待写入数据, 再根据所述待写入数据将要写入存储介质的地址将所述待写入数据写入存 储介质中;
该第一存储阵列模块 203还用于向所述第一 VHBA202返回所述写请求 指令的响应,所述第一 VHBA202向所述第一应用模块 201返回所述写请求 指令的响应。
在本发明实施例中, 由于第一应用模块 201 和第一存储阵列模块 203 在同一个物理主机内, 通过第一 VHBA卡 202连接, 可以共用一套操作系 统, 在写数据时仅需传递第一数据緩冲区 204 的地址和待写入数据, 减少 了内存拷贝, 提升了***性能。
参见图五, 是本发明一个实施方式所公开的数据处理装置, 该装置包 括: 第二应用模块 301、 第二 VHBA302、 第二存储阵列模块 303和第二数 据緩冲区 304;
所述第二应用模块 301用于将读请求指令和第二数据緩冲区 304的 地址发送给第二 VHBA302, 所述读请求指令包括待读取数据在存储介质 中的地址;
所述第二 VHBA302 用于将所述读请求指令和所述第二数据緩冲区 304的地址发送给第二存储阵列模块 303 ;
所述第二存储阵列模块 303还用于根据所述待读取数据在存储介质 中的地址, 获取待读取数据, 再根据所述第二数据緩冲区 304 的地址将 所述待读取数据写入所述第二数据緩冲区 304;
所述第二存储阵列模块 303还用于向所述第二 VHBA302返回所述读请 求指令的响应,所述第二 VHBA302向所述第二应用模块 301返回所述读请 求指令的响应。 在本发明实施例中, 第二应用模块 301和第二存储阵列模块 302在同 一个物理主机内, 共用一套操作***, 在读取数据时仅传递数据緩冲区 304 的地址和待读取数据的地址, 因此本发明实施例既实现了应用主机和存储 ***的融合, 减少了内存拷贝, 提升了***性能。
以上所述仅为本发明的优选实施例, 并非因此限制本发明的专利范围, 凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换, 或直 接或间接运用在其他相关的技术领域, 均同理包括在本发明的专利保护范 围内。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分步 骤是可以通过程序来指令相关的硬件完成, 上述的程序可以存储于一种计 算机可读存储介质中, 上述提到的存储介质可以是只读存储器, 磁盘或光
盘等。
以上对本发明所提供的一种能耗确定方法及装置进行了详细介绍, 对 于本领域的一般技术人员, 依据本发明实施例的思想, 在具体实施方式及 应用范围上均会有改变之处, 因此, 本说明书内容不应理解为对本发明的 限制。
Claims
1、 一种数据处理方法, 其特征在于, 包括:
第一应用模块接收到写请求命令后, 将待写入数据写入第一数据緩 冲区, 并将写请求指令和所述第一数据緩冲区的地址发送给第一虚拟主 机总线适配卡 VHBA, 所述写请求指令包括所述待写入数据将要写入存 储介质的地址;
所述第一 VHBA将所述写请求指令和所述第一数据緩冲区的地址发 送给第一存储阵列模块;
所述第一存储阵列模块接收到所述写请求指令后, 根据所述第一数 据緩冲区的地址, 从所述第一数据緩冲区中获取所述待写入数据, 再根 据所述待写入数据将要写入存储介质的地址将所述待写入数据写入存储 介质中;
所述第一存储阵列模块向所述第一 VHBA返回所述写请求指令的响 , 所述第一 VHBA向所述第一应用模块返回所述写请求指令的响应。
2、 根据权利要求 1 所述的方法, 其特征在于: 所述第一应用模块包括 应用软件栈, 所述应用软件栈包括应用程序和驱动程序。
3、 根据权利要求 1 所述的方法, 其特征在于: 所述第一存储阵列模块 包括目标器和阵列控制软件。
4、、 一种数据处理方法, 其特征在于, 包括:
第二应用模块将读请求指令和第二数据緩冲区的地址发送给第二 VHBA, 所述读请求指令包括待读取数据在存储介质中的地址;
所述第二 VHBA将所述读请求指令和所述第二数据緩冲区的地址发 送给第二存储阵列模块;
所述第二存储阵列模块根据所述读请求指令中携带的所述待读取数 据在存储介质中的地址, 获取待读取数据, 再根据所述第二数据緩冲区 的地址将所述待读取数据写入所述第二数据緩冲区;
所述第二存储阵列模块向所述第二 VHBA返回所述读请求指令的响 应, 所述第二 VHBA向所述应用模块返回所述读请求指令的响应。
5. 根据权利要求 4所述的方法, 其特征在于: 所述第二应用模块包括 应用软件栈, 所述应用软件栈包括应用程序和驱动程序。
6. 根据权利要求 4所述的方法, 其特征在于: 所述第二存储阵列模块 包括目标器和阵列控制软件。
7. 一种数据处理装置,其特征在于,包括:第一应用模块、第一 VHBA、 第一数据緩冲区和第一存储阵列模块;
所述第一应用模块接收到写请求命令后, 将待写入数据写入所述第 一数据緩冲区, 并将写请求指令和所述第一数据緩冲区的地址发送给所 述第一 VHBA, 所述写请求指令包括所述待写入数据将要写入存储介质 的地址;
所述第一 VHBA将所述写请求指令和所述第一数据緩冲区的地址发 送给所述第一存储阵列模块;
所述第一存储阵列模块接收到所述写请求指令后, 根据所述第一数 据緩冲区的地址, 从所述第一数据緩冲区中获取所述待写入数据, 再根 据所述待写入数据将要写入存储介质的地址将所述待写入数据写入存储 介质中;
所述第一存储阵列模块向所述第一 VHBA返回所述写请求指令的响 应, 所述第一 VHBA向所述第一应用模块返回所述写请求指令的响应。
8. 根据权利要求 7所述的装置, 其特征在于: 所述第一应用模块包括 应用软件栈, 所述应用软件栈包括应用程序和驱动程序。
9. 根据权利要求 7所述的装置, 其特征在于: 所述第一存储阵列模块 包括目标器和阵列控制软件。
10. 一种数据处理装置, 其特征在于, 包括: 第二应用模块、 第二 VHBA, 第二数据緩冲区和第二存储阵列模块;
所述第二应用模块用于将读请求指令和所述第二数据緩冲区的地址 发送给第二 VHBA, 所述读请求指令包括待读取数据在存储介质中的地 址;
所述第二 VHBA用于将所述读请求指令和所述第二数据緩冲区的地 址发送给第二存储阵列模块;
所述第二存储阵列模块用于根据所述待读取数据在存储介质中的地 址, 获取待读取数据, 再根据所述第二数据緩冲区的地址将所述待读取 数据写入所述第二数据緩冲区;
所述第二存储阵列模块还用于向所述第二 VHBA返回所述读请求指 令的响应, 所述第二 VHBA向所述第二应用模块返回所述读请求指令的 口向应。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP20120844648 EP2667296A4 (en) | 2012-04-11 | 2012-11-23 | METHOD AND APPARATUS FOR PROCESSING DATA |
US13/909,180 US9213500B2 (en) | 2012-04-11 | 2013-06-04 | Data processing method and device |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201110439201.8 | 2012-04-11 | ||
CN201110439201.8A CN102609215B (zh) | 2012-04-11 | 2012-04-11 | 数据处理方法及装置 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/909,180 Continuation-In-Part US9213500B2 (en) | 2012-04-11 | 2013-06-04 | Data processing method and device |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2013152593A1 true WO2013152593A1 (zh) | 2013-10-17 |
Family
ID=46526622
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2012/085126 WO2013152593A1 (zh) | 2012-04-11 | 2012-11-23 | 数据处理方法及装置 |
Country Status (4)
Country | Link |
---|---|
US (1) | US9213500B2 (zh) |
EP (1) | EP2667296A4 (zh) |
CN (1) | CN102609215B (zh) |
WO (1) | WO2013152593A1 (zh) |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102609215B (zh) * | 2012-04-11 | 2015-05-27 | 华为数字技术(成都)有限公司 | 数据处理方法及装置 |
CN104166647A (zh) * | 2013-05-16 | 2014-11-26 | 华为技术有限公司 | 一种数据拷贝的方法、装置和*** |
CN103516800B (zh) * | 2013-09-29 | 2018-04-10 | 华为技术有限公司 | 服务器***及其操作***启动方法和启动管理节点 |
US8898353B1 (en) * | 2013-12-04 | 2014-11-25 | Oracle International Corporation | System and method for supporting virtual host bus adaptor (VHBA) over infiniband (IB) using a single external memory interface |
US9104637B2 (en) | 2013-12-04 | 2015-08-11 | Oracle International Corporation | System and method for managing host bus adaptor (HBA) over infiniband (IB) using a single external memory interface |
US9311044B2 (en) * | 2013-12-04 | 2016-04-12 | Oracle International Corporation | System and method for supporting efficient buffer usage with a single external memory interface |
CN106020722A (zh) * | 2016-05-19 | 2016-10-12 | 浪潮(北京)电子信息产业有限公司 | 一种云存储***的重复数据去重方法、装置及*** |
CN106210041B (zh) * | 2016-07-05 | 2019-09-20 | 杭州华为数字技术有限公司 | 一种数据写入方法及服务器端网卡 |
CN109002361B (zh) * | 2017-06-07 | 2022-06-03 | 阿里巴巴集团控股有限公司 | 数据处理方法、分配方法、电子设备、客户端和存储介质 |
KR102395190B1 (ko) * | 2017-07-31 | 2022-05-06 | 삼성전자주식회사 | 호스트와 인터페이스를 수행하는 스토리지 장치, 호스트 및 스토리지 장치의 동작방법 |
CN109040221A (zh) * | 2018-07-26 | 2018-12-18 | 郑州云海信息技术有限公司 | 一种数据传输方法、***、设备及存储阵列和存储介质 |
CN112445620B (zh) * | 2020-11-30 | 2022-12-27 | 苏州浪潮智能科技有限公司 | 一种smb协议写入性能的优化方法、装置及可读存储介质 |
CN113204573B (zh) * | 2021-05-21 | 2023-07-07 | 珠海金山数字网络科技有限公司 | 一种数据读写访问***及方法 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102073535A (zh) * | 2011-01-11 | 2011-05-25 | 华中科技大学 | 基于硬件计数器虚拟化的多虚拟机性能分析方法 |
CN102124449A (zh) * | 2008-06-30 | 2011-07-13 | 甲骨文美国公司 | 用于低开销数据传输的方法和*** |
US20120079143A1 (en) * | 2010-09-24 | 2012-03-29 | Xsigo Systems, Inc. | Wireless host i/o using virtualized i/o controllers |
CN102609215A (zh) * | 2012-04-11 | 2012-07-25 | 成都市华为赛门铁克科技有限公司 | 数据处理方法及装置 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7693145B2 (en) | 2005-02-28 | 2010-04-06 | Hewlett-Packard Development Company, L.P. | Method and apparatus for direct reception of inbound data |
US7594060B2 (en) * | 2006-08-23 | 2009-09-22 | Sun Microsystems, Inc. | Data buffer allocation in a non-blocking data services platform using input/output switching fabric |
US7821973B2 (en) | 2006-10-24 | 2010-10-26 | Hewlett-Packard Development Company, L.P. | Sharing of host bus adapter context |
US9092254B2 (en) * | 2012-03-23 | 2015-07-28 | International Business Machines Corporation | Enabling multi-tenancy for integrated development environment software in virtual environments |
US20130346532A1 (en) * | 2012-06-21 | 2013-12-26 | Microsoft Corporation | Virtual shared storage in a cluster |
-
2012
- 2012-04-11 CN CN201110439201.8A patent/CN102609215B/zh active Active
- 2012-11-23 EP EP20120844648 patent/EP2667296A4/en not_active Withdrawn
- 2012-11-23 WO PCT/CN2012/085126 patent/WO2013152593A1/zh active Application Filing
-
2013
- 2013-06-04 US US13/909,180 patent/US9213500B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102124449A (zh) * | 2008-06-30 | 2011-07-13 | 甲骨文美国公司 | 用于低开销数据传输的方法和*** |
US20120079143A1 (en) * | 2010-09-24 | 2012-03-29 | Xsigo Systems, Inc. | Wireless host i/o using virtualized i/o controllers |
CN102073535A (zh) * | 2011-01-11 | 2011-05-25 | 华中科技大学 | 基于硬件计数器虚拟化的多虚拟机性能分析方法 |
CN102609215A (zh) * | 2012-04-11 | 2012-07-25 | 成都市华为赛门铁克科技有限公司 | 数据处理方法及装置 |
Non-Patent Citations (1)
Title |
---|
See also references of EP2667296A4 * |
Also Published As
Publication number | Publication date |
---|---|
EP2667296A4 (en) | 2014-01-22 |
US9213500B2 (en) | 2015-12-15 |
US20130275668A1 (en) | 2013-10-17 |
EP2667296A1 (en) | 2013-11-27 |
CN102609215B (zh) | 2015-05-27 |
CN102609215A (zh) | 2012-07-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2013152593A1 (zh) | 数据处理方法及装置 | |
US9696942B2 (en) | Accessing remote storage devices using a local bus protocol | |
US9727503B2 (en) | Storage system and server | |
TWI732110B (zh) | 對非揮發性快閃記憶體進行低延遲直接資料存取的系統及方法 | |
AU2013388031B2 (en) | Data processing system and data processing method | |
CN109582614B (zh) | 针对远程存储器访问的nvm express控制器 | |
US7640378B2 (en) | Method and apparatus for improving the performance of USB mass storage devices in the presence of long transmission delays | |
US11966612B2 (en) | Solid-state disk (SSD) data migration | |
US8949486B1 (en) | Direct memory access to storage devices | |
US20070006020A1 (en) | Inter-host data transfer method, program, and system | |
CN113424147A (zh) | 用于流式传输存储设备内容的***和方法 | |
US20220222016A1 (en) | Method for accessing solid state disk and storage device | |
US9003077B2 (en) | Sharing IOMMU mappings across devices in a DMA group | |
WO2014202003A1 (zh) | 数据存储***的数据传输方法、装置及*** | |
WO2022032990A1 (zh) | 一种命令信息传输方法、***、装置及可读存储介质 | |
US20060259650A1 (en) | Method of transmitting data between storage virtualization controllers and storage virtualization controller designed to implement the method | |
TWI540511B (zh) | 用以存取虛擬機器之電腦系統及方法 | |
CN114546902A (zh) | 基于多协议访问存储器的***、设备和方法 | |
JP6825263B2 (ja) | ストレージ制御装置、およびストレージシステム | |
KR20200143922A (ko) | 메모리 카드 및 이를 이용한 데이터 처리 방법 | |
CN110737396B (zh) | 数据复制的方法、设备和计算机存储介质 | |
US20220124164A1 (en) | System and method to offload point to multipoint transmissions | |
WO2023000784A1 (zh) | 数据访问方法以及相关设备 | |
WO2020258566A1 (zh) | 一种数据传输方法及乒乓dma架构 | |
US20160275038A1 (en) | Method for communication using pci express dedicated communication module and network device including the same |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 2012844648 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |