CN1280735C - Initiator triggered remote memory access virtual-physical address conversion method - Google Patents
Initiator triggered remote memory access virtual-physical address conversion method Download PDFInfo
- Publication number
- CN1280735C CN1280735C CNB2003101170389A CN200310117038A CN1280735C CN 1280735 C CN1280735 C CN 1280735C CN B2003101170389 A CNB2003101170389 A CN B2003101170389A CN 200310117038 A CN200310117038 A CN 200310117038A CN 1280735 C CN1280735 C CN 1280735C
- Authority
- CN
- China
- Prior art keywords
- smart process
- address
- initiator
- rma
- smart
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The present invention relates to a false-true address conversion method of initiator triggered remote memory access operation in the technical field of the memory access of a computer. In the method, the false-true address conversion to a remote memory, which is referred in the realization of an RMA operation, is realized by an initiator of the RMA operation namely the computer which originates the RMA operation. In the method, one system spirit progress is operated on each computer in which memory systems are distributed, and the system spirit progress is used for maintaining the cache of a local memory page table and a remote memory page table. The RMA operation completes the false-true address conversion through the connection of the local machine system spirit processes and spirit processes on other computers. The method of initiator triggered false-true address conversion is good for enhancing the extensibility of the realization of the RMA operation in a communication system.
Description
Technical field
The present invention relates to Computer Storage access technique field, the long-distance inner accessing operation actual situation address conversion method that particularly a kind of initiator triggers.
Background technology
At present in the distributed store of main flow, do not provide in the high-performance computer system in global physical address space, for supporting to share the storage programming mode, support visit to long-distance inner, people have introduced the long-distance inner accessing operation, i.e. RMA operation.RMA operation makes in the system process moved on the computing machine to visit internal memory on other computing machine by the virtual address space of visiting operation process on other computing machine.The RMA operation mainly comprises long-range reading and two kinds of operations of remote write.General long-distance inner accessing operation flow process is as follows:
1. a process on computing machine is open with its part storage space, allows the process visit that moves on other computing machine.This computing machine becomes the purpose side of long-distance inner accessing operation.
2. the process of moving on other computing machine is obtained the sign in open storage space.This computing machine becomes the initiator of long-distance inner accessing operation.
3. the process on initiator's computing machine is carried out long-range reading or the remote write operation, and the storage space open to purpose side's process conducts interviews, and finishes the read or write to the Open distance internal memory.
The realization of RMA operation at present is basically based on the intelligent network adapter of high-performance internet.So-called intelligent network adapter is meant that local CPU is arranged on the network interface card, but the operational communications control program has the protocol processes ability.The general implementation method that adopts is to send a RMA operational order by RMA operation initiator to purpose side, intelligent network adapter on purpose side's computing machine is after receiving above-mentioned instruction, according to the action type (reading or writing) of appointment and address, length in the instruction, directly visit the memory headroom of (reading or writing) appointment, memory access result (reading) return initiator's process by DMA or PIO mode for long-range.Process on purpose side's computing machine can not known the execution of these storage access operations.
A critical step is the actual situation address translation of carrying out the open storage space in the RMA operation realizes.Because intelligent network adapter can only use the memory headroom of physical address access computer, and RMA operation initiator process is used the visit in open storage space is virtual address or abstract sign, therefore must carry out the actual situation address translation in open storage space in RMA realizes, be physical address with virtual address translation.The main at present method of finishing the actual situation address translation by purpose side's intelligent network adapter that adopts.In case after purpose side's intelligent network adapter is received the RMA operational order that other computing machine sends, just from instruction, isolate the virtual address that to visit storage space, and according to the high-speed cache that is based upon the internal memory page table on the network interface card, promptly Cache finishes the actual situation address translation.In case Cache does not hit, intelligent network adapter can be sent out interruption to main frame, requires main frame to carry out page table and upgrades, and then carry out address translation.In case main frame will be revised page table, can make the corresponding page table entry on the intelligent network adapter invalid earlier.
A this major defect by purpose side's intelligent network adapter realization actual situation address conversion method is that extensibility is not high, agreement is complicated.Because the memory headroom of computing machine is increasing at present, especially after 64 systems are extensive use of, and the space that can be used for depositing page table on the intelligent network adapter is very limited, this can cause card to go up the frequent substitution of page table Cache content undoubtedly, and can make communication control program on the card become complicated by intelligent network adapter maintain page tables Cache, communication performance is impacted.
Summary of the invention
For addressing the above problem, the invention provides a RMA operation actual situation address conversion method that triggers by the initiator.The principal character of this method is that the actual situation address translation has been transferred on initiator's the main frame by the intelligent network adapter of purpose side.(smart process Daemon) is safeguarded by a backstage executive process that operates in the system on each computing machine for the page table in open storage space and Cache thereof.Carrying out a RMA when a process like this operates, will in the page table Cache that the smart process of its local system is safeguarded, search corresponding list item, Cache hits and just carries out the actual situation address translation, otherwise obtain required list item from the smart process of purpose method, system, upgrade local Cache, carry out the actual situation address translation then.Because initiator's host memory is very big, so the extensibility of the method is good, agreement realizes also simple relatively.
Description of drawings
Fig. 1 is the RMA operational flowchart that initiator of the present invention triggers;
Fig. 2 is the address mapping process flow diagram of smart process of the present invention;
Fig. 3 is the RMA running program process flow diagram that initiator of the present invention triggers;
Fig. 4 is the program flow diagram of smart process of the present invention.
Embodiment
Among Fig. 1, arrow is represented the step in the RMA flow process.
At initiator's user program to smart process search purposes side buffer zone physical address;
2. smart process is returned this physical address to user program;
3. initiator's user program uses physical address to carry out the RMA visit.
Among Fig. 2, arrow is represented the step in the address mapping flow process of smart process.
1. consumer process sends the address mapping request;
2. smart process inquiry inner memory page Cache if hit, then goes to step 7;
3. when miss, smart process is sent the address mapping request to the smart process in purpose side;
4. the smart process in purpose side is inquired about local internal memory page table;
5. the smart process in purpose side return address transformation result;
6. smart process is upgraded local inner memory page Cache with the page table entry in the return results.
7. smart process return address transformation results;
Among Fig. 3, constituted the process of initiating a RMA behaviour in the user program by step 3.1 and 3.2; Step 3.1 is sent the request of address lookup to the smart process in this locality earlier, and waits for local smart process return results; Step 3.2 is initiated the RMA operation after receiving Query Result;
Step 3.3 and step 3.4 constituted with the local smart process flow process of user program on same computing machine in the mutual part operation flow process of user program; Step 3.3 is waited for the address translation request of user program; Step 3.4 is changed at the laggard row address of the address translation request of receiving user program, and returns physical address; After being finished, smart process will go back to the address translation request that step 3.3 continues to wait for user program.
Among Fig. 4, step 4.1 is to 4.7 program circuits that constituted smart process; Step 4.1 is waited for the request of being sent by local user's program or long-range smart process that receives, and changes different flow processs over to according to the type of request;
The flow process of handling the request of local application address mapping is finished to 4.5 by step 4.2; The inner memory page Cache of the smart process of step 4.2 inquiry if hit, changes step 4.5, otherwise changes step 4.3; Step 4.3 is sent the address mapping request to the smart process in purpose side, and waits for its return results; The result that step 4.4 will be returned upgrades the inner memory page Cache of smart process; Step 4.5 returns to consumer process with the result who is obtained, and goes to the arrival that step 4.1 is waited for next request;
The flow process of the smart process address conversion request of processing remote is finished by step 4.6 and 4.7; The local internal memory page table of the smart process of step 4.6 inquiry; The conversion of step 4.7 process address returns to the smart process of the request of sending with the physical address of changing gained, and goes to the arrival that step 4.1 is waited for next request.
1) during system's initial start, in system, starts the smart process of a system on each computing machine.
2) when open its part storage space of process, when allowing on other computing machine process remote access, the smart process of the system on this process place computing machine (purpose side) in physical memory, and is set up corresponding page table entry with the respective stored locking space.
When 3) process on other computing machine (initiator) is carried out the RMA operation, in the inner memory page that the smart process of local system is safeguarded, search corresponding list item: if hit, just finish the actual situation address translation, the intelligent network adapter that the physical address of remote storage is write the RMA instruction and issue purpose side is carried out; Otherwise, the smart process of local system can with the smart process communication of the system of purpose side, obtain required list item, upgrade local Cache, carry out the actual situation address translation then.
4) in case the page table of local internal memory will be revised by purpose side, its can be at first has the smart process of system on the computing machine of native page list item to give an order to other maintenance, the corresponding list item among their Cache is changed to inefficacy, and then implements the modification to local list item.
Effect of the present invention is embodied in:
1.RMA the actual situation address transition during operation realizes is finished at operation initiator's main frame.
2. the page table that is used for the actual situation address translation is based upon host memory, and the space is bigger.
3. the local page table Cache of the communication control program Maintenance free on purpose side's intelligent network adapter can directly obtain the physical address of wanting access memory from the order of originating side sent, starts DMA or PIO and conducts interviews, and it is simple that protocol processes becomes.
4.RMA the extensibility of operation improves.
Claims (11)
1. the long-distance inner accessing operation that triggers by the initiator in the distributed storage system, i.e. the method for RMA operation actual situation address translation is characterized in that may further comprise the steps:
Operate the switch process of initiator's RMA destination address at RMA;
Use physical address to carry out the data transmission step of data transmission.
2. the method for claim 1 is characterized in that: the actual situation address translation to long-distance inner that relates to during the RMA operation realizes is implemented in the initiator of RMA operation, the computing machine of promptly initiating the RMA operation.
3. the method for claim 1 is characterized in that: what send to that purpose side implements appointment in the order of RMA operation by the initiator is the physical address of long-distance inner.
4. the method for claim 1 is characterized in that: the page table of being responsible for safeguarding that the actual situation address translation is required by the smart process of a system on each computing machine in the system.
5. the method for claim 1, it is characterized in that: the inner memory page high-speed cache that the smart process of each system is responsible for safeguarding, it is the implementation strategy that Cache adopts Distributed C ache in the storage compartment system, can be set up and maintenance by the smart process of system that operates on this computing machine by remote access internal memory page table on each computing machine in the system, the smart process of system on other computing machine is safeguarded be have above-mentioned can be by the high-speed cache of remote access internal memory page table, retouching operation to local internal memory page table itself can only carry out on its corresponding internal memory page table place computing machine, before carrying out the retouching operation of a page table entry, earlier other to be calculated having of on-board maintenance above-mentioned can be changed to by the high-speed cache of remote access internal memory page table invalid.
6. the long-distance inner accessing operation actual situation address conversion method that triggers of an initiator, the principal character of this method is to have been transferred to the actual situation address translation on initiator's the main frame by the intelligent network adapter of purpose side, the page table in open storage space and Cache thereof are safeguarded by the smart process of system that operates on each computing machine, carrying out a RMA when a process like this operates, will in the page table Cache that the smart process of its local system is safeguarded, search corresponding list item, Cache hits and just carries out the actual situation address translation, otherwise obtain required list item from the smart process of purpose method, system, upgrade local Cache, carry out the actual situation address translation then.
7. the long-distance inner accessing operation actual situation address conversion method that triggers according to the initiator of claim 6, its step is as follows:
1) during system's initial start, in system, starts the smart process of a system on each computing machine;
2) when open its part storage space of a process, when allowing on other computing machine process remote access, the smart process of system on other computing machine on purpose side, the process place computing machine in physical memory, and is set up corresponding page table entry with the respective stored locking space;
When 3) process on initiator's computing machine is carried out the RMA operation, in the inner memory page that the smart process of local system is safeguarded, search corresponding list item: if hit, just finish the actual situation address translation, the intelligent network adapter that the physical address of remote storage is write the RMA instruction and issue purpose side is carried out; Otherwise, the smart process of local system can with the smart process communication of the system of purpose side, obtain required list item, upgrade local Cache, carry out the actual situation address translation then;
4) in case the page table of local internal memory will be revised by purpose side, at first there is the smart process of system on the computing machine of native page list item to give an order to other maintenance, the corresponding page table entry among their Cache is changed to inefficacy, and then implements modification local list item.
8. the long-distance inner accessing operation actual situation address conversion method that triggers according to the initiator of claim 6 or 7 is characterized in that, the long-distance inner accessing operation step that the initiator triggers is as follows:
At initiator's user program to smart process search purposes side buffer zone physical address;
2. smart process is returned this physical address to user program;
3. initiator's user program uses physical address to carry out the RMA visit.
9. the long-distance inner accessing operation actual situation address conversion method that triggers according to the initiator of claim 6 or 7 is characterized in that the step of the address mapping flow process of smart process is as follows;
1. consumer process sends the address mapping request;
2. smart process inquiry inner memory page Cache if hit, then goes to step 7;
3. when miss, smart process is sent the address mapping request to the smart process in purpose side;
4. the smart process in purpose side is inquired about local internal memory page table;
5. the smart process in purpose side return address transformation result;
6. smart process is upgraded inner memory page Cache with the page table entry that returns;
7. smart process return address transformation results.
10, the long-distance inner accessing operation actual situation address conversion method that triggers according to the initiator of claim 6, initiator's triggering step is as follows:
Constituted the process of initiating a RMA operation in the user program by step 3.1 and 3.2; Step 3.1 is sent the request of address lookup to the smart process in this locality earlier, and waits for local smart process return results; Step 3.2 is initiated the RMA operation after receiving Query Result;
Step 3.3 and step 3.4 constituted with the local smart process flow process of user program on same computing machine in the mutual part operation flow process of user program; Step 3.3 is waited for the address translation request of user program; Step 3.4 is changed at the laggard row address of the address translation request of receiving user program, and returns physical address; After being finished, smart process will go back to the address translation request that step 3.3 continues to wait for user program.
11, the long-distance inner accessing operation actual situation address conversion method that triggers according to the initiator of claim 6, the step of smart process is as follows:
Step 4.1 is to 4.7 program circuits that constituted smart process; Step 4.1 is waited for the request of being sent by local user's program or long-range smart process that receives, and changes different flow processs over to according to the type of request;
The flow process of handling the request of local application address mapping is finished to 4.5 by step 4.2; The inner memory page Cache of the smart process of step 4.2 inquiry if hit, changes step 4.5, otherwise changes step 4.3; Step 4.3 is sent the address mapping request to the smart process in purpose side, and waits for its return results; The result that step 4.4 will be returned is updated to the inner memory page Cache of smart process; Step 4.5 returns to consumer process with the result who is obtained, and goes to the arrival that step 4.1 is waited for next request;
The flow process of the smart process address conversion request of processing remote is finished by step 4.6 and 4.7; The local internal memory page table of the smart process of step 4.6 inquiry; The conversion of step 4.7 process address returns to the smart process of the request of sending with the physical address of changing gained, and goes to the arrival that step 4.1 is waited for next request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2003101170389A CN1280735C (en) | 2003-12-04 | 2003-12-04 | Initiator triggered remote memory access virtual-physical address conversion method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2003101170389A CN1280735C (en) | 2003-12-04 | 2003-12-04 | Initiator triggered remote memory access virtual-physical address conversion method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1547126A CN1547126A (en) | 2004-11-17 |
| CN1280735C true CN1280735C (en) | 2006-10-18 |
Family
ID=34337680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2003101170389A Expired - Fee Related CN1280735C (en) | 2003-12-04 | 2003-12-04 | Initiator triggered remote memory access virtual-physical address conversion method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1280735C (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1614577B (en) * | 2004-12-02 | 2010-11-24 | 中国科学院计算技术研究所 | Graphic terminal method and system based on long-range direct memory access |
| CN100377117C (en) * | 2005-07-14 | 2008-03-26 | 中国科学院计算技术研究所 | Method and device for converting virtual address, reading and writing high-speed buffer memory |
| CN100437522C (en) * | 2005-09-09 | 2008-11-26 | 中国科学院计算技术研究所 | Long-distance inner server and its implementing method |
| CN100520739C (en) * | 2006-12-30 | 2009-07-29 | 中国科学院计算技术研究所 | Rapid virtual-to-physical address converting device and its method |
| CN107766259B (en) * | 2016-08-23 | 2021-08-20 | 华为技术有限公司 | Page table cache access method, page table cache, processor chip and storage unit |
| WO2023044861A1 (en) * | 2021-09-26 | 2023-03-30 | Intel Corporation | Concept for providing access to remote memory |
-
2003
- 2003-12-04 CN CNB2003101170389A patent/CN1280735C/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN1547126A (en) | 2004-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10963387B2 (en) | Methods of cache preloading on a partition or a context switch | |
| US10204058B2 (en) | GPU shared virtual memory working set management | |
| US6889288B2 (en) | Reducing data copy operations for writing data from a network to storage of a cached data storage system by organizing cache blocks as linked lists of data fragments | |
| JP5603507B2 (en) | Accelerating cache state migration on directory-based multi-core architectures | |
| US9612975B2 (en) | Page cache device and method for efficient mapping | |
| EP1522928A2 (en) | Priority-based flash memory control apparatus for XIP in serial flash memory, memory management method using the same, and flash memory chip thereof | |
| US9563571B2 (en) | Intelligent GPU memory pre-fetching and GPU translation lookaside buffer management | |
| US9336154B2 (en) | Data bus efficiency via cache line usurpation | |
| CN1302393C (en) | Method for controlling cache system comprising direct-mapped cache and fully-associative buffer | |
| US7752386B1 (en) | Application performance acceleration | |
| CN1726477A (en) | Page descriptors for prefetching and memory management | |
| US9411518B2 (en) | Method, computer program product and apparatus for accelerating responses to requests for transactions involving data operations | |
| US11836133B2 (en) | In-memory database (IMDB) acceleration through near data processing | |
| US20150324300A1 (en) | System and Methods for Efficient I/O Processing Using Multi-Level Out-Of-Band Hinting | |
| WO2016015583A1 (en) | Memory management method and device, and memory controller | |
| CN1896972A (en) | Method and device for converting virtual address, reading and writing high-speed buffer memory | |
| US20030005257A1 (en) | Memory table and memory manager for use in managing memory | |
| CN1280735C (en) | Initiator triggered remote memory access virtual-physical address conversion method | |
| CN1286006C (en) | Cache system and method for managing cache system | |
| US20080162818A1 (en) | Cache-memory control apparatus, cache-memory control method and computer product | |
| CN111273860B (en) | Distributed memory management method based on network and page granularity management | |
| US20050216695A1 (en) | Memory extension for a data processor to provide both common and separate physical memory areas for virtual memory spaces | |
| CN116541415A (en) | Apparatus, system and method for acceleration | |
| US20230333983A1 (en) | Systems and methods for a cross-layer key-value store architecture with a computational storage device | |
| US20030149846A1 (en) | Cache updating in multiprocessor systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20170630 Address after: 100190 room 10, floor 6, scientific research complex, No. 1044 South Road, Haidian District Academy of Sciences, Beijing Patentee after: Beijing Kejiyin Technology Co. Ltd. Address before: 100080 No. 6 South Road, Zhongguancun Academy of Sciences, Beijing Patentee before: Institute of Computing Technology, Chinese Academy of Sciences |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20191223 Address after: Room 801, building 6, Qilu Chuangzhi Park, 2600 Wenhua South Road, Kuiwen District, Weifang City, Shandong Province Patentee after: Weifang Zhongke Zhishi Information Technology Co., Ltd Address before: 100190 1044, room 10, research complex, 6 South Road, Haidian District Science Academy, Beijing. Patentee before: Beijing Kejiyin Technology Co. Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20061018 Termination date: 20201204 |