US9632790B2 - Select logic for the instruction scheduler of a multi strand out-of-order processor based on delayed reconstructed program order - Google Patents

Select logic for the instruction scheduler of a multi strand out-of-order processor based on delayed reconstructed program order Download PDF

Info

Publication number
US9632790B2
US9632790B2 US13/993,794 US201213993794A US9632790B2 US 9632790 B2 US9632790 B2 US 9632790B2 US 201213993794 A US201213993794 A US 201213993794A US 9632790 B2 US9632790 B2 US 9632790B2
Authority
US
United States
Prior art keywords
instructions
rpo
values
value
strand
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US13/993,794
Other languages
English (en)
Other versions
US20150301831A1 (en
Inventor
Jayesh Iyer
Nikolay KOSAREV
Sergey Y. Shishlov
Alexey Y. Sivtsov
Yuriy V Baida
Alexander V Butuzov
Bob Babayan
Vladimir Pentkovski
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intel Corp
Original Assignee
Intel Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intel Corp filed Critical Intel Corp
Assigned to INTEL CORPORATION reassignment INTEL CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BABAYAN, BOB, BUTUZOV, ALEXANDER V., BAIDA, YURIY V., IYER, JAYESH, KOSAREV, Nikolay, SHISHLOV, Sergey Y., SIVTSOV, ALEXEY Y.
Assigned to INTEL CORPORATION reassignment INTEL CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: PENTKOVSKI, VLADIMIR
Publication of US20150301831A1 publication Critical patent/US20150301831A1/en
Application granted granted Critical
Publication of US9632790B2 publication Critical patent/US9632790B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/38—Concurrent instruction execution, e.g. pipeline or look ahead
    • G06F9/3836—Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/38—Concurrent instruction execution, e.g. pipeline or look ahead
    • G06F9/3836—Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution
    • G06F9/3851—Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution from multiple instruction streams, e.g. multistreaming
    • G06F9/3855—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/38—Concurrent instruction execution, e.g. pipeline or look ahead
    • G06F9/3854—Instruction completion, e.g. retiring, committing or graduating
    • G06F9/3856—Reordering of instructions, e.g. using queues or age tags

Definitions

  • This disclosure relates to the field of processing devices and, in particular, to select logic for an instruction scheduler of a multi-stand out-of-order processor using a reconstructed program order.
  • ILP instruction level parallelism
  • critical loops In such processors, in order to achieve desired performance goals (in terms of instructions per cycle (IPC)), some pieces of logic, which may be referred to as critical loops, evaluate in a single execution cycle or else they could be obstacles to the deeper pipelining described above.
  • One such critical loop includes instruction scheduling logic, which may be made up of wakeup logic and select logic.
  • Wakeup logic which includes tracking data dependencies and checking if source operands needed by instructions are available, determines when instructions are ready to be sent for execution. Select logic determines, based on some policy, which of these ready instructions should be sent for execution. Select logic may only be applicable when there are more ready instructions than a number of available execution resources. As an instruction cannot be “qualified as ready” by the wakeup logic until all the instructions which it is dependent on are selected and sent for execution, the wakeup logic and the select logic form a critical loop for performance. In addition, the select logic needs to select the right instructions to schedule to the execution units, such that it could move instructions forward on the critical-path (i.e., wake up the dependent instructions). Thus, for every processor with out-of-order instruction execution, select logic which determines which instruction to select first if there are multiple instructions which are ready to occupy a single execution resource and which fits into the timing budget of single-cycle scheduling loop is important for achieving the desired performance.
  • a multi-strand out-of-order processor which implements an out-of-order fetching technique (i.e. it is capable of fetching instructions out-of-order from different strands of a multi-strand program representation generated by the compiler).
  • a multi-strand out-of-order processor is not aware of the program order of the instructions within a strand with respect to instructions from other strands also allocated in the waiting buffer.
  • scheduling policies currently used by the select logic in conventional out-of-order processors, such as for example, age-based policies, location-based policies, round robin policies, compiler-aided priority policies, split scheduling window approaches, and select free scheduling.
  • age-based policies that schedules instructions for execution based on when they are allocated is not applicable, since the instructions are allocated out-of-order.
  • Location-based policies and round robin policies which prioritize instructions based on their location in the waiting buffer, have lower hardware costs but suffer from decreased performance.
  • Select free scheduling which removes the select logic from the critical path by pipelining select logic into a 1-cycle wakeup loop and a 2-cycle select loop, increases the clock frequency, but at the cost of compromising IPC.
  • a select logic scheduling policy is needed that can make use of ILP to achieve higher performance in terms of IPC while still meeting the timing requirements of the critical single-cycle scheduling loop, without growing the complexity of the select logic in a multi-strand out-of-order processor.
  • FIG. 1 is a block diagram illustrating an instruction scheduling flow in a multi-strand out-of-order processor using a delayed reconstructed program order, according to an embodiment.
  • FIG. 2 is a block diagram illustrating a flow for appending program order information to program instructions, according to an embodiment.
  • FIG. 3 is a flow diagram illustrating a method for scheduling instructions using a delayed reconstructed program order, according to an embodiment.
  • FIG. 4 is a flow diagram illustrating a method for reconstructing a program order, according to an embodiment.
  • FIG. 5 is a block diagram illustrating a hardware arrangement for reconstructing a program order, according to an embodiment.
  • FIG. 6 is a flow diagram illustrating a method for ordering instructions based on the reconstructed program order, according to an embodiment.
  • FIG. 7 is a block diagram illustrating a system for ordering instructions based on the reconstructed program order, according to an embodiment.
  • FIG. 8 is a block diagram illustrating a computer system, according to an embodiment.
  • Described herein is a method and system for scheduling of instructions in a multi-strand out-of-order processor using a delayed reconstructed program order.
  • an ordered list of ready to dispatch instructions is created for execution, using a delayed reconstructed program order (RPO) of strands in a multi-strand out-of-order processor.
  • RPO delayed reconstructed program order
  • This list may govern the order in which the instructions will be dispatched for execution every cycle.
  • a delay of a certain number of execution cycles e.g., 2 or 3 cycles
  • This approach may be similar in complexity to prevalent select logic policies (e.g., age-based policies) but leads to better performance (i.e., higher instructions executed per cycle (IPC)) in out-of-order fetching processors.
  • the scheduling of instructions in a multi-strand out-of-order processor using a delayed reconstructed program order makes better use of the instruction level parallelism (ILP) that characterizes the inherent parallelism of a program algorithm than other select logic policies.
  • the policy by using delayed reconstructed program order value, also optimizes the scheduling (i.e., wakeup and select) loop by removing the part of select logic for creating the ordered list from the critical path of the loop without compromising on IPC performance (as compared to select free scheduling implemented in some out-of-order machines, which optimizes the loop but with a drop in IP performance).
  • the policy also relieves the compiler from the need to explicitly and statically specify special additional priorities for strands (as compared to policies like “branch path,” “chain length,” and “maximum dependents,” which use compiler support to provide priority information).
  • the delayed RPO policy adapts instruction scheduling to dynamic situations and allows for the handling of nondeterministic latencies, such as cache misses and delay variations, much better than static schedule policies like “chain-length,” “max dependents,” and “branch path,” which rely totally on the compiler to set priorities.
  • EIP executed instruction pointer
  • the processor is able to achieve a very large instruction window. To be able to fully make use of the ILP and maintain a high sustained execution width, it is essential to be able to select the right strands (and therefore instructions), which might be on the critical path to dispatch from the available ready to dispatch strands in the scheduler.
  • the strand select algorithm is responsible for identifying the right strands to dispatch every cycle, which in turn would facilitate the wakeup of dependent instructions and hence could achieve a high sustained execution width of the machine.
  • the algorithm creates an ordered list of ready to dispatch instructions based on the delayed RPO of each strand, which governs the order in which the instructions are dispatched for execution every cycle.
  • the processor does not normally have information about the program order of an instruction with respect to other instructions in the program algorithm.
  • the explicit program order information is appended to an instruction in the object code at compilation time. This may be the case only for instructions that would change the state of the system, such as memory instructions (e.g., load or store) or instructions that can cause interrupts.
  • memory instructions e.g., load or store
  • instructions that can cause interrupts e.g., to save space in the instruction size
  • only an increment to the actual program order is encoded in the instruction. This may be referred to as the incremental program order (IPO).
  • the compiler guarantees that the program order values are assigned to the instructions within strand strongly in-order, thus forming a progressing sequence of program order values in the strand.
  • the select logic of the processor reconstructs the actual program order of the strand by adding the IPO encoded in the instruction to the program order (or RPO) of a previously dispatched instruction in the same strand.
  • RPO program order
  • Each of the entries in a waiting buffer in an instruction scheduling unit (ISU) keeps the RPO of the strand being considered for scheduling, as well as IPOs for calculation of the RPO for the next instruction in the strand.
  • the waiting buffer can be presented as strand scheduling flops (SSF).
  • the SSF may include a two-level structure where the first level (SSF 1 ) holds non-ready (including newly allocated) instructions for consideration by the wakeup logic and the second level (SSF 2 ) holds the ready instructions and keeps the RPO of these instructions for consideration by the select logic.
  • the RPO of the strand and IPO encoded in the instruction may be kept in separate buffers and multiplexed separately.
  • a delay buffer may store the updated RPO values for previous cycles, so that the delayed RPO values may be used to create the ordered priority list.
  • FIG. 1 is a block diagram illustrating an instruction flow through an instruction scheduling unit in a multi-strand out-of-order processor using a delayed reconstructed program order, according to an embodiment.
  • the various modules and components may be described in regards to their roles in scheduling instructions for execution using a delayed reconstructed program order.
  • Instructions in a multi-strand out-of-order architecture may be fetched and grouped in strands.
  • a strand is a sequence of instructions that are predominantly data-dependent upon each other and are executed in-order. But instructions of different strands are mostly independent so multi-strand out-of-order architectures may execute instructions from numerous strands in parallel after resolving any data dependency between them through the wakeup logic. As such, the instructions of a second strand may outrun the instructions of a first strand, if there is no data dependency, even though the location of the first strand instructions may precede the location of second strand instructions in the original source code.
  • the processor 100 shown in FIG. 1 is logically divided into a front end unit 110 , an instruction scheduling unit 120 , an execution unit 130 and memory subsystem (not shown in FIG. 1 ).
  • the front end unit 110 includes a number of instruction buffers, each corresponding to a particular strand 112 , 114 , 116 .
  • the instruction buffers may be used for receiving fetched instructions.
  • the instruction buffers may be implemented using a queue (e.g., FIFO queue) or any other container-type data structure. Instructions stored in an instruction buffer may be ordered based on an execution or program order.
  • the processor 100 may be one example of processing device 802 , described below with respect to FIG. 8 , which operates in a computing system 800 .
  • each buffer may be interdependent.
  • instructions may be buffered in an execution order that respects the data dependencies among the instructions of the strand and thus executed strongly in-order. For example, a result of executing a first instruction of a strand may be required to evaluate a second instruction of the strand. As such, the first instruction will precede the second instruction in an instruction buffer dedicated for the strand.
  • an instruction stored in a head of a buffer may be designated as the first or next instruction for dispatch and execution.
  • the instruction scheduling unit 120 may receive an instruction from an instruction buffer in one of strands 112 , 114 , 116 , as its input.
  • the instruction scheduling unit 120 may be organized into strand scheduled flops (SSF).
  • the strand scheduled flops may include a two level hardware structure that holds the instructions at the head of each strand when they are allocated to the instruction scheduling unit 120 .
  • the SSF may include a first level of hardware entries (i.e., SSF 1 140 ) and a second level of hardware entries (i.e., SSF 2 160 ), for storing instructions.
  • the aforementioned hardware entries may include but are not limited to hardware buffers, flops, or any other hardware resource capable of storing instructions and/or data.
  • the instruction scheduling unit 120 includes wakeup logic 150 for checking operand readiness of instructions stored in SSF 1 140 .
  • Wakeup logic which includes tracking data dependencies and checking if source operands needed by instructions are available, determines when instructions are ready to be sent for execution.
  • Wakeup logic 150 may take as its input, an instruction stored in a first level hardware entry (e.g., 142 , 144 or 146 ) and determine whether the operands for the particular instruction are ready. For example, if a particular instruction is designed to add two operands together to determine a result, the operands must be calculated (possibly by other instructions) before the instruction can be executed. An operation is considered “ready” if all the operations it depends on have been executed. If the instruction is ready, wakeup logic 150 may move the instruction to the corresponding entry (e.g., 162 , 164 , 166 ) in SSF 2 160 , so that the instruction may be considered for execution.
  • a first level hardware entry e.g., 142 , 144
  • the instruction scheduling unit 120 may further include select logic 170 .
  • Select logic 170 may take as its input one or more instructions stored in SSF 2 160 and determine the availability of execution ports for those stored instructions. For example, if there are n strands of instructions and x execution ports, select logic 170 may select up to x out of then stored instructions to designate to the x execution ports 132 , 134 , 136 . Once an execution port is designated as available for an operand-ready instruction stored in SSF 2 160 , the instruction is dispatched to the execution port. In one embodiment, select logic 170 creates an ordered list of ready to dispatch instructions for execution, using a delayed reconstructed program order of the strands.
  • select logic 170 may introduce a delay of a certain number of execution cycles (e.g., 2 or 3 cycles) so that the instructions which are dispatched for execution in a given cycle are selected based on the ordered list created by processing the reconstructed program order values from several cycles prior. This removes part of the select logic (i.e., the creation of the ordered list) from the critical path, without decreasing the clock frequency. Additional details regarding select logic 170 are provided below.
  • execution unit 130 includes a number of execution ports 132 , 134 , 136 , to which operand-ready instructions stored in the instruction scheduling unit 120 are dispatched. Once an instruction is dispatched to an execution port, it is executed and generates its result.
  • a front-end instruction buffer, a first level hardware entry, an operand check module, and a second level hardware entry may be dedicated for each strand.
  • a first strand 112 may be associated with a dedicated entry 142 in SSF 1 and a dedicated entry 162 in SSF 2 . Accordingly, these features may be used only with respect to instructions of the first strand.
  • a second strand 114 may be associated with a dedicated entry 144 in SSF 1 and a dedicated entry 164 in SSF 2 , and so on.
  • FIG. 2 is a block diagram illustrating a flow for appending program order information to program instructions, according to an embodiment.
  • program source code 200 includes a number of program instructions 202 , 204 , 206 and 208 .
  • the program instructions may be arranged in a defined execution or program order, such that Instruction 0 202 is to be executed before Instructions 1 204 , which is to be executed before Instruction 2 206 , and so on.
  • complier 210 may assign a sequence number named program order (PO), to each of a subset of instructions named program-ordered instructions (PO instructions), which are defined herein as any instructions that access memory data (e.g., a load or a store instruction), as well as any instructions that may cause an interrupt.
  • PO instructions program-ordered instructions
  • the program order values are natural numbers assigned in progressive order, but the sequence of values need not be consecutive.
  • one sequence of instructions may include a load, an add, a load, a store, a subtract, a store, and a load.
  • the first load may be assigned a PO value of 1
  • the second load may be assigned a PO value of 2
  • the first store a PO value of 4 the second store a PO value of 5
  • the third load a PO value of 8.
  • Compiler 210 may then split this sequence of instructions across a plurality of strands 220 , 230 , 240 .
  • the instructions may be divided between strands based on dependencies between the instructions. For example, each instruction that depends on another instruction may be placed in the same strand as the instructions on which it depends. In addition, independent instructions might be distributed among the available strands 220 , 230 , 240 .
  • compiler appends an incremental program order (IPO) value to each of the PO instructions (e.g., 222 , 224 ) in each strand.
  • the IPO value (e.g., 223 , 225 ) may not encode the entire PO of an instruction. Instead, the IPO may only encode an increment value that is equal to the difference between the PO value of the instruction for which the PO is being calculated and the PO value of the next PO instruction in the same strand.
  • the processor hardware reconstructs the PO value for the PO instructions using the IPO value appended to the PO instruction.
  • IPO information need not be encoded for non-PO instructions, such as 226 , because PO information is only needed for memory instructions and instructions which may cause an interrupt, in order to maintain the correct order for memory accesses.
  • arithmetic instructions may not have an IPO value as part of their instruction format.
  • the compiler 210 may split the program into multiple strands for execution on a multi-strand architecture, while preserving the program order information by attaching the IPO field to each PO instruction.
  • FIG. 3 is a flow diagram illustrating a method for scheduling instructions using a delayed reconstructed program order, according to an embodiment.
  • the method 300 may be performed by processing logic that may comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), or a combination thereof.
  • the processing logic may dispatch instructions for execution based on an ordered list created from delayed reconstructed program order values.
  • the method 300 is performed by select logic 170 , as shown in FIG. 1 .
  • method 300 calculates a reconstructed programming order (RPO) value for each ready instruction selected for dispatch.
  • RPO reconstructed programming order
  • wakeup logic 150 wakes up instructions in SSF 1 140 and moves the instructions to SSF 2 160 .
  • SSF 2 160 includes instructions that are ready for scheduling.
  • select logic 170 may calculate an RPO value.
  • the RPO value may be indicative of the actual (or “real”) program order of the instructions.
  • select logic 170 adds an incremental program order value appended to the instruction, when it is selected for dispatch, to the RPO value of the previously selected instruction from the same strand (which may be stored in a buffer in select logic 170 ).
  • the result of this operation may be the reconstructed program order (RPO) value of this instruction.
  • method 300 creates an ordered list of instructions based on the delayed RPO values (i.e. delayed RPO values are used to determine which instructions should be dispatched for scheduling).
  • select logic 170 may create an ordered list of instructions based on the RPO values that were calculated Z cycles (e.g., 2 or 3 cycles) previously. For example, if the RPO values are updated for dispatched instructions in cycle T, those updated RPO values may be used for scheduling instructions Z cycles later. Normally the system would expect that if the RPO is updated in cycle T, the select logic would use this updated RPO to create an ordered list in cycle T+1 (in order to satisfy the 1-cycle schedule).
  • the updated RPO will be used in the cycle (T+1)+Z. This removes part of the select logic (i.e., the creation of the ordered list based on the RPO values) from the critical path without a significant loss in performance.
  • the instruction having the oldest RPO value may be given the highest priority.
  • method 300 dispatches the highest priority instructions based on the ordered list of delayed RPO values.
  • select logic 170 dispatches the instructions with the oldest RPO values to the execution unit 130 . For example, if there are five execution ports, then the first five instructions from ordered list may be dispatched.
  • FIG. 4 is a flow diagram illustrating a method for reconstructing a program order, according to an embodiment.
  • the method 400 may be performed by processing logic that may comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), or a combination thereof.
  • the processing logic may reconstruct the program order of ready instructions in an out-of-order processor.
  • the method 400 is performed by select logic 170 , as shown in FIG. 1 , using the hardware arrangement 500 described below with respect to FIG. 5 .
  • method 400 reads an incremental program order (IPO) value appended to an instruction that is dispatched to execution unit.
  • SSF 2 160 may store a number of entries 162 , 164 , 166 , each of which, if it is a PO instruction, keeps the corresponding IPO value for the instruction.
  • the RPO value of the strand which is calculated by the previous instruction dispatched from the same strand, is either kept in the corresponding SSF 2 entries or stored in a separate RPO buffer.
  • the IPO value may encode an increment value that is equal to the difference between the RPO value of the previous instruction in the same strand and the RPO value of the instruction to be calculated for which the IPO is encoded.
  • the IPO value represents the difference between instructions in the real program order that is attributable to the reorganization of the instructions into strands, as described above with respect to FIG. 2 .
  • method 400 adds the IPO value to the corresponding strand PO or RPO value.
  • the PO value for a strand (representing the PO value of the most recently selected instruction from that strand) and the IPO value for the instruction are applied to an arithmetic logic unit (ALU) that adds the IPO value to the PO value (which may also be referred to as RPO value, if previously reconstructed) to form an updated RPO value of the strand.
  • ALU arithmetic logic unit
  • the PO and IPO may be combined using binary addition. However, other arithmetic operations may alternatively be used depending on how the compiler has calculated the IPOs in relation to the POs.
  • method 400 stores the updated strand RPO value either in the corresponding SSF 2 entry or in a separate RPO buffer. Since the newly calculated RPO value at block 420 necessarily represents the most recently dispatched instruction, the updated RPO value may be used as the strand PO value. Thus, the RPO value may be written back to the RPO buffer or stored as the strand PO value in a corresponding entry of SSF 2 . The updated RPO value may be used to order instructions for scheduling, as will be further described below, and to calculate (i.e., reconstruct) the POs of subsequent instructions in corresponding entries in the waiting buffer. For purposes of illustration, the blocks of FIG. 4 show the process for a single instruction. However, it should be understood that in some implementations, the actions at block 410 - 430 may be performed for each dispatched instruction in SSF 2 in parallel.
  • FIG. 5 is a block diagram illustrating a hardware arrangement 500 for reconstructing a program order in a multi-strand out-of-order processor, according to an embodiment of the present invention.
  • the arrangement 500 may be implemented as part of an instruction scheduling unit (ISU), such as ISU 120 within a multi-strand out-of-order processor, such as processor 100 .
  • ISU instruction scheduling unit
  • the arrangement 500 may be a standalone circuit that communicates with an ISU in a multi-strand out-of-order processor to perform reconstruction of the program order.
  • the arrangement 500 may include a waiting buffer 510 and logic for program order reconstruction (e.g. program order reconstruction units 530 , 532 and a write demultiplexer 540 ).
  • the waiting buffer 510 may store a number of entries, each of which keeps the program order (PO) of the strand, calculated through dispatched instruction in the previous cycle, being considered for execution, as well as corresponding IPOs for the instructions.
  • the waiting buffer 510 may be analogous to a reservation station in a superscalar processor, in that it allocates every instruction that needs to be executed, with the difference being in the out-of-order allocation. Also in this waiting buffer instructions are de-allocated as soon as they are dispatched for execution.
  • the waiting buffer is split into two levels and the second level of waiting buffer 510 holds only ready instructions.
  • each entry of the waiting buffer 510 may comprise a PO-IPO pair that is associated with a head instruction in a strand.
  • the waiting buffer 510 may store up to n PO-IPO pairs.
  • the waiting buffer 510 may output the PO-IPOs to a read multiplexer 520 .
  • the entries in the waiting buffer 510 may also correspond to non-PO instructions which, as previously explained, may not include an IPO encoding.
  • the waiting buffer 510 entry may only include a PO value for the non-PO instruction, along with a zero or null value in the IPO field. Since the PO value of the next PO instruction in the strand has been calculated by the previous PO instruction (i.e., by adding the IPO of the previous PO instruction dispatched with the strand PO value), the PO value of the next PO instruction in the strand can be set equal to the PO value of non-PO instructions when dispatched. Additionally, in clock cycles when there is no instruction in the waiting buffer 510 for a strand (e.g. the previous instruction has been executed, and the next one has not yet been fetched so that the buffer entry for that strand is empty), the PO field continues to contain the PO of the previous PO instruction, which continues to be used in the calculation of the RPO.
  • the read multiplexer 520 is part of the select logic and may be implemented as a single (n to x) multiplexer that receives the PO-IPO pairs in the waiting buffer 510 . During each execution cycle, the read multiplexer 520 may forward up to x entries out of the possible n entries, based on the instructions selected by the select logic from the ordered list which will be described later, to x number of dispatch ports.
  • the schedule vector 522 is an input into the read multiplexer 120 .
  • the schedule vector 522 may be the output from the select logic and may indicate which of the x instructions out of the possible n ready instructions in waiting buffer 150 are selected for dispatch in the current cycle. So the schedule vector 522 controls the x entries to be forwarded from the read multiplexer 520 .
  • the number of entries selected as being scheduled for execution in any given clock cycle may vary from cycle to cycle but could be a maximum up to x.
  • the PO and IPO values may be kept in separate buffers (i.e., the waiting buffer 510 is split into two buffers), so the read multiplexer 520 may be implemented using two buffers, one for dispatching the PO value and one for dispatching the IPO value, but both of these multiplexers would still be controlled by the schedule vector 522 as described above.
  • the selected PO-IPO pairs may be output to the PO reconstruction units 530 , 532 .
  • the reconstruction logic may include a number of PO reconstruction units 530 , 532 , each of which is configured to reconstruct the PO of an instruction coming into the waiting buffer 510 , by using a corresponding PO-IPO pair output from the read multiplexer 520 .
  • a reconstruction unit 530 may be provided for each of the outputs of the read multiplexer 520 (i.e., x dedicated reconstruction units).
  • each reconstruction unit 530 may include arithmetic logic that combines the PO and the IPO to form the reconstructed programming order (RPO).
  • RPO reconstructed programming order
  • the PO and IPO may be combined using binary addition. However, other arithmetic operations may alternatively be used depending on how the compiler has calculated the IPOs in relation to the POs.
  • the write demultiplexer 540 may be implemented using one or more multiplexers (e.g., a single x to n multiplexer) that together receive the RPOs from the reconstruction units 530 , 532 . As with the read multiplexer 520 , the write demultiplexer 540 may be controlled by the schedule vector 522 from the ISU. Thus, the write demultiplexer 540 may select, RPO for which x strands out of total n strands need to be updated in the waiting buffer 510 .
  • multiplexers e.g., a single x to n multiplexer
  • a feedback loop is formed in which the POs of previous instructions in the waiting buffer 510 are used to calculate (i.e., reconstruct) the POs of subsequent instructions in corresponding entries in the waiting buffer 510 , where the reconstructed POs correspond to the real program order of the PO instructions that have selected to be scheduled for execution in a given cycle.
  • the updated PO values which are written back to the waiting buffer 510 may ultimately be provided to an ordering logic.
  • the ordering logic which is part of the select logic, may compare the RPO values and reorder instructions in an increasing RPO value, resulting in an ordered list.
  • the ordering logic may implement ordering the instructions by using the RPO values updated a number of cycles prior to the current cycle. For example, the ordering logic may form the ordered list using the RPO values from 2 cycles prior. This ordered list may then be used to prioritize instructions for dispatch to the execution ports.
  • the waiting buffer 510 or the RPO buffer may store the RPO values from prior cycles and those values may be provided directly to the ordering logic.
  • FIG. 6 is a flow diagram illustrating a method for ordering instructions based on the reconstructed program order, according to an embodiment.
  • the method 600 may be performed by processing logic that may comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device to perform hardware simulation), or a combination thereof.
  • the processing logic may create an ordered list of instructions based on delayed RPO values associated with the strands.
  • the method 600 is performed by select logic 170 , as shown in FIG. 1 , using the system 700 described below with respect to FIG. 7 .
  • method 600 compares the updated RPO value to all other updated strand RPO values.
  • the selection logic compares one RPO value to the RPO values for all other strands in parallel.
  • the selection logic may implement a matrix based logic to generate a comparison matrix storing the results of the parallel comparisons.
  • method 600 determines if the updated RPO value (RPOi) is less than the RPO value of all other ready instructions in waiting buffer 510 to which it is currently being compared (RPOj). If RPOi is less than (or equal to) RPOj, at block 630 , method 600 sets a corresponding matrix cell (cell ij ) to a first value (e.g., 1). If RPOi is not less than or equal to RPOj (i.e., RPOi is greater than RPOD, at block 640 , method 600 sets the corresponding matrix cell (cell ij ) to a second value (e.g., 0). In FIG. 6 , i is from 0 to n ⁇ 1 and j is from 0 to n ⁇ 1, which means RPO value of each entry in waiting buffer is compared in parallel to each of the other entries in waiting buffer.
  • method 600 converts all the matrix cells associated with the updated RPO value (RPOi) to a single representative value.
  • one's counting logic counts the number of 1's in each row and produces a 4-bit representative value (i.e., from 1111 indicating the instruction with least RPO to 0000 indicating the instruction with highest RPO).
  • the one's counting logic may be implemented as tree of adders (for each row) operating in parallel.
  • method 600 may order the instructions and set the instruction priority based on the representative values for each instruction.
  • ordering logic takes each of the 4-bit representative values and orders them in an ordered list based on their value (e.g., 1111 may occupy the highest position in the ordered list, while 0000 occupies the lowest position). This way, the ordered list is created based on increasing RPO value (i.e. oldest ready instruction occupying the first position while the youngest ready instruction in SSF 2 occupying the last position). In one embodiment, the ordered list will hold only the representative value (i.e., 4-bit value). The instructions may then be dispatched for execution based on the ordered list.
  • FIG. 7 is a block diagram illustrating a system for ordering instructions based on the reconstructed program order, according to an embodiment.
  • the system 700 includes a comparison matrix 710 , one's counting logic 720 , and ordering logic 730 .
  • Comparison matrix 710 may receive as inputs, the updated RPO values calculated as described above.
  • delayed RPO values are used. For example, if the RPO values are updated for dispatched instructions in cycle T, those updated RPO values may be used for scheduling instructions Z cycles later than expected. Normally the system would expect that if the RPO is updated in cycle T, the select logic would use this updated RPO to create an ordered list in cycle T+1.
  • the updated RPO will be used in the cycle (T+1)+Z. This removes part of the select logic (i.e., the creation of the ordered list based on the RPO values) from the critical path without a significant loss in performance.
  • the system 700 can use the delay of Z cycles to create the ordered list, as will be described below.
  • each RPO value is compared to every each of the other RPO values at the same time.
  • the results of these comparisons may be stored in comparison matrix 710 .
  • all comparisons may be performed in parallel and be implemented by a matrix based logic, where every strand RPO is compared with all other strands RPO in parallel.
  • RPOi indicates the RPO of the instruction stored in entry i of SSF 2 .
  • Each cell of the matrix 710 indicates the result of the comparison of two RPOs.
  • Row 0 has the result for the RPO of strand 0 (RPO 0 ) being compared with RPO of all other strands (strand 0 to 15 indicated by RPO 0 to RPO 15 ).
  • the result of each comparison may be a single bit according to a defined condition.
  • the condition may be defined in some other manner.
  • RPO 0 if RPO 0 is lesser than the RPOs of twelve other strands, then row 0 of the matrix will have twelve 1's in it (indicating that instruction of strand 0 is older than the 12 other ready strands in SSF 2 ). This means that in the matrix, there will also be a row with all 1's, indicating that instruction in that way is the oldest amongst the 16 entries of SSF 2 . In addition, there will also be a row with all 0's, indicating it is the youngest instruction among the 16 entries. If there are two strands with equal RPO values, then the selection logic may choose based on position of the strand (e.g. priority in decreasing way number). Also, the RPO comparison of a way with itself is indicated by an X in the diagram (as no comparison required). In one embodiment, X can be treated as 1. The resulting values in comparison matrix 710 may be provided to one's counting logic 720 .
  • each row of the matrix will have certain number of 1's, which indicates, how many ready instructions in SSF 2 are younger than the instruction corresponding to that row. This information may be used to order the instructions based on the RPO values.
  • one's counting logic 720 counts the number of 1's in each row in parallel and produces a 4-bit representative value (i.e., from 1111 indicating the instruction with least RPO to 0000 indicating the instruction with highest RPO).
  • the one's counting logic 720 may be implemented as tree of adders (for each row) operating in parallel.
  • Ordering logic 730 then takes each of these 16, 4-bit representative values and orders them in an ordered list 735 based on their value (1111 may occupy the highest position in the ordered list, while 0000 occupies the lowest position). This way, the ordered list is created, based on increasing RPO value (i.e., oldest ready instruction occupying the first position while the youngest ready instruction in SSF 2 occupying the last position). In one embodiment, the ordered list 735 will hold only the representative value (i.e., 4-bit value). In one embodiment, the ordering logic 730 may be implemented as 1-to-16 demultiplexer (one for each way to be ordered in the ordered list). In one embodiment, based on the ordered list 735 , the instructions may be dispatched for execution. For example, if there are five execution ports, then the first five instructions from ordered list 735 may be dispatched.
  • FIG. 8 illustrates a diagrammatic representation of a machine in the exemplary form of a computer system 800 within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed.
  • the machine may be connected (e.g., networked) to other machines in a local area network (LAN), an intranet, an extranet, or the Internet.
  • the machine may operate in the capacity of a server or a client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.
  • the machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a server, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
  • PC personal computer
  • PDA Personal Digital Assistant
  • STB set-top box
  • WPA Personal Digital Assistant
  • a cellular telephone a web appliance
  • server a server
  • network router network router, switch or bridge
  • computer system 800 may be representative of a computing device, such as a computing device including processor 100 .
  • the exemplary computer system 800 includes a processing device 802 , a main memory 804 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory 806 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage device 818 , which communicate with each other via a bus 830 .
  • ROM read-only memory
  • DRAM dynamic random access memory
  • SDRAM synchronous DRAM
  • RDRAM Rambus DRAM
  • static memory 806 e.g., flash memory, static random access memory (SRAM), etc.
  • SRAM static random access memory
  • Any of the signals provided over various buses described herein may be time multiplexed with other signals and provided over one or more common buses.
  • the interconnection between circuit components or blocks may be shown as buses or as single signal lines. Each of the buses may alternatively be one or more single signal lines and each of the
  • Processing device 802 represents one or more general-purpose processing devices such as a microprocessor, central processing unit, or the like.
  • the processing device implements a multi-strand architecture which allows out-of-order fetching and execution of instructions arranged by the compiler into multiple strands.
  • the processing device may implement a complex instruction set computing (CISC) architecture, a reduced instruction set computer (RISC) architecture, a very long instruction word (VLIW) architecture, or other instruction sets, or a combination of instruction sets, through translation of binary codes in the abovementioned instruction sets into the multi-strand architecture by a compiler.
  • CISC complex instruction set computing
  • RISC reduced instruction set computer
  • VLIW very long instruction word
  • Processing device 802 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like.
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • DSP digital signal processor
  • the processing device 802 is configured to execute processing logic 826 for performing the operations and steps discussed herein.
  • the computer system 800 may further include a network interface device 808 .
  • the computer system 800 also may include a video display unit 810 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 812 (e.g., a keyboard), a cursor control device 814 (e.g., a mouse), and a signal generation device 816 (e.g., a speaker).
  • a video display unit 810 e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)
  • an alphanumeric input device 812 e.g., a keyboard
  • a cursor control device 814 e.g., a mouse
  • a signal generation device 816 e.g., a speaker
  • the data storage device 818 may include a machine-accessible storage medium 828 , on which is stored one or more set of instructions 822 (e.g., software) embodying any one or more of the methodologies of functions described herein.
  • the instructions 822 may also reside, completely or at least partially, within the main memory 804 and/or within the processing device 802 during execution thereof by the computer system 800 ; the main memory 804 and the processing device 802 also constituting machine-accessible storage media.
  • the instructions 822 may further be transmitted or received over a network 820 via the network interface device 808 .
  • the instructions may be executed in hardware of processing device 802 . In other embodiments, however, the instructions may be stored in memory 804 or in storage 818 and executed by software.
  • the machine-readable storage medium 828 may also be used to store instructions for scheduling instructions in a processor based on a delayed reconstructed processing order, as described herein. While the machine-readable storage medium 828 is shown in an exemplary embodiment to be a single medium, the term “machine-readable storage medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions.
  • a machine-readable medium includes any mechanism for storing information in a form (e.g., software, processing application) readable by a machine (e.g., a computer).
  • the machine-readable medium may include, but is not limited to, magnetic storage medium (e.g., floppy diskette); optical storage medium (e.g., CD-ROM); magneto-optical storage medium; read-only memory (ROM); random-access memory (RAM); erasable programmable memory (e.g., EPROM and EEPROM); flash memory; or another type of medium suitable for storing electronic instructions.
  • magnetic storage medium e.g., floppy diskette
  • optical storage medium e.g., CD-ROM
  • magneto-optical storage medium e.g., magneto-optical storage medium
  • ROM read-only memory
  • RAM random-access memory
  • EPROM and EEPROM erasable programmable memory
  • flash memory or another type of medium suitable for storing electronic instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Advance Control (AREA)
US13/993,794 2012-12-26 2012-12-26 Select logic for the instruction scheduler of a multi strand out-of-order processor based on delayed reconstructed program order Active 2033-04-22 US9632790B2 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/IB2012/003079 WO2014102563A1 (en) 2012-12-26 2012-12-26 Select logic using delayed reconstructed program order

Publications (2)

Publication Number Publication Date
US20150301831A1 US20150301831A1 (en) 2015-10-22
US9632790B2 true US9632790B2 (en) 2017-04-25

Family

ID=51019952

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/993,794 Active 2033-04-22 US9632790B2 (en) 2012-12-26 2012-12-26 Select logic for the instruction scheduler of a multi strand out-of-order processor based on delayed reconstructed program order

Country Status (4)

Country Link
US (1) US9632790B2 (ko)
KR (1) KR101730282B1 (ko)
CN (1) CN104781786B (ko)
WO (1) WO2014102563A1 (ko)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10983799B1 (en) 2017-12-19 2021-04-20 Apple Inc. Selection of instructions to issue in a processor
US11422821B1 (en) 2018-09-04 2022-08-23 Apple Inc. Age tracking for independent pipelines

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2514618B (en) * 2013-05-31 2020-11-11 Advanced Risc Mach Ltd Data processing systems
US10255075B2 (en) * 2013-07-18 2019-04-09 Nvidia Corporation System, method, and computer program product for managing out-of-order execution of program instructions
CN107567614B (zh) * 2015-06-01 2022-05-24 英特尔公司 用于对根据关键度被分组的指令的缕程的执行的多核处理器
US11106467B2 (en) 2016-04-28 2021-08-31 Microsoft Technology Licensing, Llc Incremental scheduler for out-of-order block ISA processors
US20180032344A1 (en) * 2016-07-31 2018-02-01 Microsoft Technology Licensing, Llc Out-of-order block-based processor
US10241801B2 (en) 2016-12-23 2019-03-26 Intel Corporation Method and apparatus to create register windows for parallel iterations to achieve high performance in HW-SW codesigned loop accelerator
US10241789B2 (en) 2016-12-27 2019-03-26 Intel Corporation Method to do control speculation on loads in a high performance strand-based loop accelerator
US10235171B2 (en) 2016-12-27 2019-03-19 Intel Corporation Method and apparatus to efficiently handle allocation of memory ordering buffers in a multi-strand out-of-order loop processor
CN109918134B (zh) * 2019-03-06 2023-05-30 湖南科技大学 用于vliw基本块调度的组合启发式指令选择优化方法
CN111552366B (zh) * 2020-04-07 2021-10-22 江南大学 一种动态延迟唤醒电路及乱序指令发射架构
CN115269008B (zh) * 2022-09-29 2023-02-28 苏州浪潮智能科技有限公司 一种数据处理方法、装置、介质及电子设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997013193A1 (en) 1995-10-06 1997-04-10 Advanced Micro Devices, Inc. Instruction buffer organization method and system
US5765037A (en) * 1985-10-31 1998-06-09 Biax Corporation System for executing instructions with delayed firing times
US5859992A (en) 1997-03-12 1999-01-12 Advanced Micro Devices, Inc. Instruction alignment using a dispatch list and a latch list
US6219778B1 (en) 1997-06-25 2001-04-17 Sun Microsystems, Inc. Apparatus for generating out-of-order results and out-of-order condition codes in a processor
US20040268335A1 (en) * 2003-06-24 2004-12-30 International Business Machines Corporaton Modulo scheduling of multiple instruction chains
US7065635B1 (en) 2003-12-17 2006-06-20 Sun Microsystems, Inc. Method for handling condition code modifiers in an out-of-order multi-issue multi-stranded processor
US20100274972A1 (en) 2008-11-24 2010-10-28 Boris Babayan Systems, methods, and apparatuses for parallel computing

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010060084A2 (en) * 2008-11-24 2010-05-27 Intel Corporation Systems, methods, and apparatuses to decompose a sequential program into multiple threads, execute said threads, and reconstruct the sequential execution

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5765037A (en) * 1985-10-31 1998-06-09 Biax Corporation System for executing instructions with delayed firing times
WO1997013193A1 (en) 1995-10-06 1997-04-10 Advanced Micro Devices, Inc. Instruction buffer organization method and system
US5859992A (en) 1997-03-12 1999-01-12 Advanced Micro Devices, Inc. Instruction alignment using a dispatch list and a latch list
US6219778B1 (en) 1997-06-25 2001-04-17 Sun Microsystems, Inc. Apparatus for generating out-of-order results and out-of-order condition codes in a processor
US20040268335A1 (en) * 2003-06-24 2004-12-30 International Business Machines Corporaton Modulo scheduling of multiple instruction chains
US7065635B1 (en) 2003-12-17 2006-06-20 Sun Microsystems, Inc. Method for handling condition code modifiers in an out-of-order multi-issue multi-stranded processor
US20100274972A1 (en) 2008-11-24 2010-10-28 Boris Babayan Systems, methods, and apparatuses for parallel computing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
International search report and written opinion for PCT application No. PCT/IB2012/003079, mailed on Sep. 26, 2013, 6 pages.

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10983799B1 (en) 2017-12-19 2021-04-20 Apple Inc. Selection of instructions to issue in a processor
US11422821B1 (en) 2018-09-04 2022-08-23 Apple Inc. Age tracking for independent pipelines

Also Published As

Publication number Publication date
CN104781786A (zh) 2015-07-15
WO2014102563A1 (en) 2014-07-03
US20150301831A1 (en) 2015-10-22
KR20150065865A (ko) 2015-06-15
KR101730282B1 (ko) 2017-04-25
CN104781786B (zh) 2018-05-08

Similar Documents

Publication Publication Date Title
US9632790B2 (en) Select logic for the instruction scheduler of a multi strand out-of-order processor based on delayed reconstructed program order
CN107810480B (zh) 根据性能度量的指令块分配
US8332854B2 (en) Virtualized thread scheduling for hardware thread optimization based on hardware resource parameter summaries of instruction blocks in execution groups
US9811340B2 (en) Method and apparatus for reconstructing real program order of instructions in multi-strand out-of-order processor
US9830156B2 (en) Temporal SIMT execution optimization through elimination of redundant operations
US8914805B2 (en) Rescheduling workload in a hybrid computing environment
US9529596B2 (en) Method and apparatus for scheduling instructions in a multi-strand out of order processor with instruction synchronization bits and scoreboard bits
US9652243B2 (en) Predicting out-of-order instruction level parallelism of threads in a multi-threaded processor
KR101594502B1 (ko) 바이패스 멀티플 인스턴스화 테이블을 갖는 이동 제거 시스템 및 방법
US20140208074A1 (en) Instruction scheduling for a multi-strand out-of-order processor
US10268519B2 (en) Scheduling method and processing device for thread groups execution in a computing system
CN107810479A (zh) 用于处理器控制传送的目标位置的确定
KR20180021812A (ko) 연속하는 블록을 병렬 실행하는 블록 기반의 아키텍쳐
Wang et al. Simultaneous multikernel: Fine-grained sharing of gpus
US11275561B2 (en) Mixed precision floating-point multiply-add operation
Sanjuan-Estrada et al. Adaptive parallel interval branch and bound algorithms based on their performance for multicore architectures
US20210191757A1 (en) Sub-idle thread priority class
Zhou et al. A performance analysis framework for exploiting GPU microarchitectural capability
CN111258655A (zh) 融合计算方法及可读存储介质
CN116670644A (zh) 通用计算核上的交错处理的方法
US20200278868A1 (en) Method to execute successive dependent instructions from an instruction stream in a processor
Mohammadi et al. A dynamic special-purpose scheduler for concurrent kernels on gpu
Iliakis et al. Decoupled mapreduce for shared-memory multi-core architectures
Li et al. A fine-grained prefetching scheme for DGEMM kernels on GPU with auto-tuning compatibility
US11416261B2 (en) Group load register of a graph streaming processor

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTEL CORPORATION, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PENTKOVSKI, VLADIMIR;REEL/FRAME:033845/0224

Effective date: 19930201

Owner name: INTEL CORPORATION, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:IYER, JAYESH;KOSAREV, NIKOLAY;SHISHLOV, SERGEY Y.;AND OTHERS;SIGNING DATES FROM 20130903 TO 20130906;REEL/FRAME:033845/0173

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4