US20010014918A1 - Method and apparatus for synchronized message passng using shared resources - Google Patents
Method and apparatus for synchronized message passng using shared resources Download PDFInfo
- Publication number
- US20010014918A1 US20010014918A1 US08/883,997 US88399797A US2001014918A1 US 20010014918 A1 US20010014918 A1 US 20010014918A1 US 88399797 A US88399797 A US 88399797A US 2001014918 A1 US2001014918 A1 US 2001014918A1
- Authority
- US
- United States
- Prior art keywords
- message
- list
- message buffer
- buffer
- free
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Definitions
- This invention relates generally to communications in a computer system and more particularly to performing synchronized message passing in a computer system using shared resources.
- message passing is one means of communicating between two or more processes. Messages can be sent between two or more processes executing in the same computer system, or between processes executing in different computer systems. Using message passing, a first process wishing to communicate with a second process sends the second process a message. The message is typically a quantity or packet of data sent from the first process to the second process.
- One class of techniques for communicating messages between two processes uses a single shared list in which there is always a buffer of one element in the list.
- a receiving process reads and processes a first message sent from a sending process only when a second message has been added to the list, such as that which occurs when the sending process sends a second message to the receiving process.
- the receiving process does not read, process, and delete a message until the message being accessed by the receiving process is not the same as the one being sent by the sending process.
- Such a technique only provides for message communication in one direction between two processes.
- Another class of techniques for communicating messages between two processes uses a portion of memory that is shared between processes A and B and contains two shared message lists.
- the first shared list includes messages sent from process A to process B
- the second shared list includes messages sent from process B to process A.
- Both shared lists are accessed by each of the processes A and B either to send a message to the other process, or receive its own messages.
- the first shared list is accessed by process A to send a message to process B by adding the message to the first shared list.
- the first shared list is also accessed by process B to read messages it receives by removing a message from the first shared list.
- an operating system usually employs a locking mechanism by which a shared resource, such as the foregoing two shared lists included in the portion of memory, is exclusively accessed by one of multiple processes. For example, when one of the two processes, A and B, executes a data operation in a critical section to access the first shared list, a locking mechanism insures that the other process is excluded from also accessing the first shared list.
- a typical example of such locking mechanisms is the semaphore.
- One of the drawbacks of the foregoing technique of a critical section with a locking mechanism is the performance overhead incurred by using the locking mechanism to implement the mutual exclusion.
- the additional overhead includes requesting, obtaining, and releasing a lock on a shared resource each time a process is accessing the shared resource. This additional overhead adversely impacts system performance, for example, by increasing execution time and requiring computer system resources to instantiate the locking mechanism.
- process A sends more messages to process B than process B sends to process A, there is an uneven flow of messages between process A and process B. If processes A and B each allocate their own message buffers, as used for storing data of a message, process A may request additional memory for a message buffer to send a message to process B while process B has unused message buffers. Thus, process A may require an excessive amount of memory for message buffers while process B has message buffers available for use.
- a third shared list including “free” or available memory buffers is used.
- a third shared list of available message buffers is used by both process A and process B.
- this third shared list is an improvement over the previously described technique in that use of the third shared list provides for adaptive message buffer allocation that minimizes memory allocation in the event of uneven message traffic between processes.
- a drawback of the use of a third shared list is that it represents an additional resource that is shared by process A and process B.
- a critical section is additionally required for data operations performed to obtain an available message buffer from the shared free list.
- using this third shared list further degrades system performance due to the additional use of the locking mechanism.
- access to shared resources is synchronized through discipline imposed upon updating and accessing the shared resources without requiring a global locking mechanism for synchronization.
- the method includes initializing a first message buffer containing message data, and adding the first message buffer to a list of incoming messages to the second process.
- Initializing the first message buffer includes setting a first flag indicating that the first message buffer contains a message not yet completely processed by the second process, and setting a first link field indicating that the first message buffer is the last message in the list of incoming messages to the second process.
- the list of incoming messages to the second process includes a second message buffer.
- the step of adding the first message buffer includes updating a second link field associated with the second mesage buffer to identify the first message buffer being added.
- the apparatus includes a first data structure, a second data structure, and a messsage buffer.
- the first data structure is associated with the first process and includes a remote tail pointer identifying an incoming message associated with the second process.
- the second data structure is associated with the second process and includes a head pointer to the incoming message list associated with the second process.
- the message buffer includes a link field identifying a next message in the incoming message list, and a state field indicating whether the message buffer has been processed by the second process.
- a method for allocating a message buffer from a free list when sending a message between a first and second process includes determining if the free list includes at least three message buffers and removing a message buffer from one end of the free list if there are at least three message buffers.
- FIG. 1 is a block diagram of a computer system
- FIG. 2A is a block diagram of an embodiment of data structures and processes used in message communications between processes executing in the computer system of FIG. 1;
- FIG. 2B is a flowchart of an embodiment of a method of sending messages between processes executing in the computer system of FIG. 1;
- FIG. 2C is a flowchart of an embodiment of a method of receiving messages between processes executing in the computer system of FIG. 1;
- FIG. 3 is a block diagram of an embodiment of the data structures of FIG. 2 used in interprocess message communication;
- FIG. 4 is a flowchart of an embodiment of a method for allocating a buffer from a free list of available message buffers
- FIG. 5 is a flowchart of an embodiment of a method for determining if there are more than two free message buffers in the free list of available message buffers and removing the message buffer from an end of the free list;
- FIG. 6A is a flowchart of an embodiment of a method for sending messages from process A to process B;
- FIG. 6B is a flowchart of an embodiment of a method depicting detailed steps of FIG. 6A for initializing a new message buffer and updating process A's pointer information;
- FIG. 7A is a flowchart of an embodiment of a method for retrieving a message from process A's message list
- FIG. 7B is a flowchart of an embodiment of a method for performing more detailed steps from FIG. 7A for retrieving a message from A's message list;
- FIG. 7C is a flowchart of an embodiment of a method for performing more detailed steps from FIG. 7B for returning a message buffer to the free list;
- FIG. 7D is a flowchart of an embodiment of a method for performing more detailed steps from FIG. 7C for returning a message buffer to the free list;
- FIG. 8A is a flowchart of an embodiment of a method performed upon completion of processing a message buffer for handling message buffers after message processing is complete;
- FIG. 8B is a flowchart of an embodiment of a method depicting more detailed steps of FIG. 8A for handling message buffers after data processing is complete.
- an embodiment of the computer system 10 is shown to include a plurality of host computers 12 a - 12 c , a communications medium 14 , storage subsystems 24 a - 24 b , and shared memory 30 .
- Each host 12 a - 12 c is directly connected through a communication medium 14 to storage subsystems 24 a and 24 b , and shared memory 30 .
- Host computer 12 a includes a processor 16 a , local memory 18 a and communications interface 20 a by which host computer 12 a is connected to communications medium 14 .
- each host 12 b - 12 c respectively, includes a processor 16 b - 16 c , local memory 18 b - 18 c and communications interface 20 b - 20 c .
- Each of the host systems 12 b - 12 c are respectively connected to communication medium 14 by communications interface 20 b - 20 c .
- the communication medium 14 may be, for example, a system bus.
- Storage subsystem 24 a includes storage device 28 a which is controlled by device controller 26 a .
- Each of the host computers 12 a - 12 c can communicate over communications medium 14 with storage subsystems 24 a and 24 b .
- storage subsystem 24 b also includes a device controller 26 b and device 28 b .
- Each of the storage subsystems 24 a - 24 b includes a device, such as a disk, and device controller, such as a disk controller, particular for that device included in the storage subsystem.
- Each of the host computer systems 12 a - 12 c typically perform I/O operations such as read and write operations to devices 28 a and 28 b .
- shared memory 30 which can be accessed by any of the host processors 12 a - 12 c .
- shared memory 30 is a shared resource for use by any of host computers 12 a - 12 c.
- FIG. 1 It should be noted that other resources may be included in an embodiment of a computer system as shown in FIG. 1 for use between host computer systems 12 a - 12 c . Additionally, a computer system may also contain more or fewer host computer systems than included in the embodiment of FIG. 1.
- FIG. 2A a block diagram of an embodiment of the data structures and processes used in message communication between processes executing in a computer system of FIG. 1 is shown. Included in FIG. 2A are process A 32 and process B 34 .
- process A 32 executes in one of the host computer systems 12 a - 12 c of FIG. 1.
- process B 34 also executes in one of the host computer systems 12 a - 12 c of FIG. 1.
- Process A 32 communicating with process B 34 .
- Communication between process A and process B may include, for example, process A sending a message to process B, or process B sending a message to process A.
- shared memory 30 of FIG. 2 are data structures used to facilitate communication between process A and process B. Included in shared memory 30 of FIG. 2 is a process A message list 38 a , a process B message list 38 b and a free message buffer list 38 c .
- Process A message list 38 a contains those messages which process A receives from process B.
- process B message list 38 b contains those messages received from process A.
- the free message buffer list 38 c contains available message buffers which a process sending a message to another process may obtain for sending a message.
- Process A 32 includes a data processing portion 32 a (DP) and a queue manager portion 32 b .
- the queue manager 32 b and DP 32 a interact to manage process A's message list 38 a , and process the messages included in this message list.
- the queue manager 32 b performs management functions of process A's message list 38 a such as adding and deleting message buffers from process A's message list.
- DP portion 32 a is responsible for processing the contents of a message included in process A's message list.
- process B 34 contains a DP portion 34 a and a queue manager 34 b . Both process A and process B when executing in a computer system of FIG. 1 access the shared memory 30 over communications medium 14 .
- process A message management information (MMI) 40 Also included in the local memory in which process A 32 is loaded is process A message management information (MMI) 40 .
- MMI message management information
- Process A's MMI includes information enabling proper management of the various data structures included in shared memory 30 for process A to send and receive messages.
- process B 34 contains process B message management information (MMI) 42 which is included in the local memory in which process B is loaded.
- Process B's MMI includes information enabling proper management of the data structures included in shared memory 30 for process B to send and receive messages.
- FIG. 2B shown is a flowchart of an embodiment of a method of sending a message from a sending process to a receiving process.
- A is the sending process sending a message to process B, the receiving process.
- a message buffer is allocated, as from the free message buffer list 38 a , and initialized.
- the message from step 33 a is sent to the receiving process B, as by including the message in Process B Message List 38 b .
- Control returns to step 33 a to send additional messages as needed to process B.
- process B receives messages from a sending process, process A.
- process B determines that it has received a message from process A.
- process B can determine that it has received a message from process A using one of a variety of techniques. For example, process B may use a polling mechanism in which process B continually checks for incoming messages after a predetermined amount of time. Process B may also use an interrupt mechanism by which process B receives an interrupt when a new message is received.
- step 33 d the newly received message is retrieved and processed by process B.
- end of message processing is performed, as in step 33 e , in which the message buffer including the message just processed may be returned to the free message buffer list 38 a in accordance with other conditions and principles of the invention that are described in following paragraphs.
- Process A's MMI 40 includes a local head pointer 40 a , a remote tail pointer 40 b and a free list pointer 40 c .
- the local head pointer 40 a points to the head, or first message, of the message list for process A 38 a .
- Remote tail pointer 40 b identifies the tail end or the last element on process B's message list 38 b .
- the free list pointer 40 c identifies the first message buffer in the list of free message buffers 38 c available for use by process A when sending a message to process B.
- process B's MMI 42 contains a local head pointer 42 a identifying the first message on process B's list of incoming messages 38 b .
- the remote tail pointer 42 b identifies the last element on process A's message list 38 a .
- Free list pointer 42 c identifies the first message buffer in the list of free pointers 38 c available for use by process B when sending a message to process A.
- Message lists in this embodiment such as message lists 38 a and 38 b , always include at least one element.
- each message list is initialized with a dummy message buffer element marked as “already processed”.
- this “already processed” state is indicated by setting the dummy element's state field to EMPTY.
- Other implementations may indicate this processing state in a variety of ways dependant upon the implementation.
- methods for subsequently managing the data structures of FIG. 3 guarantee that there will always be at least one element in each of the message lists 38 a and 38 b . Using these methods, processing may or may not have completed on the first element of each message list. If there is more than one element on one of the message lists, the methods guarantee that processing has not begun on the second and subsequent elements, if any, included in message list.
- Process A's message list 38 a shown in FIG. 3 includes two message buffers 44 and 46 . Note that, as previously described, this is a “snapshot” of the data structures, such as message list 38 a . Taking a “snapshot” of the data structures at other points of execution may result in different data structure states.
- the first message buffer 44 included in process A's message list 38 a is identified by the local head pointer 40 a of process A's MMI 40 .
- Message buffer 44 contains an AB link field 44 a , a BA link field 44 b , a work link field 44 c and a state field 44 d followed by any additional data for the message to be sent to process A.
- the work link field 44 c and the state field 44 d are used in managing the process A message list 38 a .
- the AB link field 44 a and the BA link field 44 b will be described in conjunction with the free message buffer list since they are used in conjunction with the management of this data structure. Since fields 44 a and 44 b are not used in the management of process A's message list 38 a , values for these fields are not shown.
- the work link field 44 c identifies the next message buffer in the linked list of messages to be processed by process A. In FIG. 3 work link field 44 c identifies the next message buffer in process A's message list 38 a , which is message buffer 46 .
- a message buffer such as 46 is the last message buffer included in process A's message list 38 a
- its work link field 46 c has the value NULL to indicate the end of the list.
- the state field, such as 44 d has the value of either EMPTY or FULL indicating whether the message included in the corresponding message buffer has been completely processed.
- Message buffer 44 has its state field 44 d set to EMPTY indicating that process A has completed processing of the message data included in message buffer 44 .
- message buffer 46 contains state field 46 d which is set to FULL indicating that the message in message buffer 46 has not yet been completely processed, for example, by the DP portion 32 a of process A 32 .
- Process B's message list 38 b contains message buffers similar to those described previously in conjunction with process A's message list 38 a.
- Fields 40 a and 40 b of process A's MMI 40 are used in managing process A's message list 38 a for processing A's incoming messages, and also for sending a message from process A to process B by adding a message to process B's message list 38 b .
- process A's MMI 40 contains the remote tail field 40 b identifying the last element 50 of process B's message list 38 b .
- Process A sends a message to Process B by adding it to the end of Process B's message list 38 b using remote tail pointer 40 b .
- Fields 42 a and 42 b of Process B's MMI 42 are used similarly to Process A's fields 40 a and 40 b.
- Used in managing the free message buffer list 38 c are the free list fields, 40 c and 42 c , respectively, of the MMI data structures 40 and 42 , and the AB link and the BA link fields of the message buffers, such as previously described in conjunction with message buffers 44 , 46 , 48 and 50 .
- process A When process A is attempting to allocate a message buffer from the free message buffer list 38 c for use in sending a message to process B, process A examines the free message buffer list using the AB link field to identify a next consecutive message buffer.
- the AB link fields of the message buffers contained in the free message buffer list form a chain of message buffers whose order is privately used by process A in allocating message buffers.
- the BA link fields of the message buffers contained in the free message buffer list 38 c form a chain of message buffers whose ordering is privately used by process B in allocating message buffers.
- Process B reads the BA link field of a first message buffer to determine whether that message buffer is available for use by process B.
- Process A does not read the BA link field in determining whether that message buffer is available for its use.
- the BA link field is private for use only in memory buffer allocation by process B.
- the AB link field is a private field available for use by process A in memory buffer allocation, and is not used by process B when determining the next element available for use on the free message buffer list 38 c.
- FIG. 4 a flowchart of an embodiment of a method for allocating a buffer from the free list 38 c is shown. Shown in FIG. 4 are the method steps for removing a message buffer from the A process end of the free list 38 c , such as when process A requires a message buffer to send a message to process B. One skilled in the art will note that these steps may be generalized and modified for also sending a message from process B to process A.
- step 66 a determination is made as to whether there are more than two free message buffers in the free list 38 c . If there are more than two free message buffers in the free message buffer list 38 c , a message buffer from the A end of the free list is removed, as in step 76 .
- the message buffer removed from the free message buffer list 38 c is used by process A.
- the queue manager portion 32 b performs the steps of FIG. 4 to remove the message buffer from the free message buffer list 38 c and return it for use by the DP portion 32 a.
- step 66 If a determination is made at step 66 that there are not more than two free message buffers in the free message buffer list, control proceeds to step 68 where a determination is made as to whether or not there are sufficient system resources to allocate an additional message buffer. If there are insufficient system resources to allocate an additional message buffer, control proceeds to step 70 where an indication is made that insufficient system resources are available to allocate a message buffer. If a determination is made at step 68 that there are sufficient system resources available to allocate an additional message buffer, control proceeds to step 72 where the memory for the new additional message buffer is allocated. This new message buffer is returned to process A for use in step 74 .
- FIG. 5 a flowchart of an embodiment of a method for performing steps 66 and 76 as described in FIG. 4 are shown.
- FIG. 5 depicts performing steps 66 and 76 of FIG. 4 upon the particular embodiment of the data structures previously shown in conjunction with FIG. 3.
- a temp pointer receives the contents of the AB_link field of the first message buffer that indicates the second message buffer, if any, of the free message buffer list 38 c .
- a determination is made at step 66 b as to whether or not the temporary pointer is NULL. If the temporary pointer is NULL, it indicates that there is one message buffer in the free message buffer list 38 b .
- step 66 c Prior to executing step 66 c , it is known that the free message buffer list contains at least two elements.
- step 66 c a determination is made as to whether or not the free message buffer list contains at least three message buffers. This is determined by examining the AB link field of the message buffer identified by the temp pointer, since temp points to the second element of the list. If the AB link field of the message buffer identified by the temp pointer is not NULL, it indicates that a third message buffer is contained in the message buffer list.
- step 66 c if a determination is made that the AB link field of the message buffer identified by the temp pointer is not NULL, control proceeds to step 76 a to remove the first message buffer from the free message buffer list 38 c . If step 66 c determines that there are not at least three message buffers in the free message buffer list, control proceeds to step 68 .
- the BA link field of the message buffer pointed to by temp is assigned a value of NULL to remove the element from the free message buffer list 38 c .
- the temp pointer is assigned to point to the message buffer identified by A.freelist which is the message buffer just allocated or removed from the free message buffer list 38 c .
- process A's MMI freelist field 40 c is updated to point to the next message buffer of the list 38 c available for use by process A.
- FIG. 6A a flowchart of an embodiment of the method steps for sending a message from process A to process B is shown.
- a new message buffer is initialized.
- the new message buffer initialized in step 80 is allocated from the free message buffer list using the steps previously described in conjunction with FIGS. 4 and 5.
- the new message initialized in step 80 is added to process B's message list 38 b using A's MMI information 40 .
- FIG. 6B a flowchart of an embodiment of a method for performing the steps previously described in conjunction with FIG. 6A are shown in more detail when using the data structures previously described in conjunction with FIG. 3 of this particular embodiment.
- steps 80 a - 80 c the new message buffer is initialized.
- step 80 a its work link field is initialized to NULL.
- step 80 b its state field is set to FULL.
- the data of the message is added to the new message buffer.
- steps 82 a - 82 b add the new message buffer to process B's message list 38 b using process A's MMI 40 .
- the new message buffer is added to the tail of process B's message list.
- the work link field of the last message buffer contained in process B message list 38 b is updated to point to the new message buffer.
- A's MMI information is updated such that the remote tail of process A's MMI 40 points to the new message buffer just added to the tail of process B's message list 38 b.
- new messages as sent from process A to process B, are added or appended to the tail of the receiving process's message list by the sending process using its MMI.
- FIG. 7A a flowchart of an embodiment of a method for retrieving a message from a message list is shown.
- the method steps of FIG. 7A depict retrieving a message received by process A from process B.
- a determination is made as to whether or not there is more than one message buffer in process A's message list 38 a .
- each process's message list such as 38 a and 38 b , always contains at least one element. If a determination at step 84 indicates that there is more than one message buffer included in A's message list 38 a , then a message is returned for processing and an attempt is made to free a message buffer from A's message list 38 a . However, if there is only a single message buffer in process A's message list 38 a , control proceeds to step 85 where the method terminates.
- step 84 If a determination is made at step 84 that process A's message list 38 a contains more than one message buffer, control proceeds to step 86 .
- step 86 the message buffer from the head of process A's message list is disconnected from the remaining portion of A's message list by updating A's MMI.
- step 88 a determination is made as to whether or not the message buffer disconnected in step 86 has been processed to completion. If a determination is made at step 88 that the message has not been processed to completion, control proceeds to step 91 where the new head of A's message list is passed to process A's DP portion. In other words, if A is not done processing the message buffer, it cannot yet be returned to the free list as available for use.
- step 88 If a determination is made at step 88 that the message has been processed, control proceeds to step 90 where the dequeued prior head message of A's message list 38 a is returned to the free message buffer list 38 c .
- step 91 the new head message buffer of A's message list is passed to A's DP for processing.
- FIG. 7B a flowchart of an embodiment of a method for retrieving a message received by process A from process B is shown in which the method uses the data structures previously described in conjunction with FIG. 3.
- a determination is made using the data structures of FIG. 3 as to whether or not there is more than one message buffer in process A's message list 38 a . This is determined by examining the work link field of the message buffer identified by the local head field 40 a of process A's MMI data structure 40 . If this work link field is NULL, control proceeds to step 85 .
- step 86 a a temp pointer is used to identify the first message buffer on process A's message list 38 a .
- the first message buffer in process A's message list is disconnected from the remainder of A's message list by updating A's MMI field local head 40 a to point to the next consecutive message buffer in process A's message list.
- step 88 a a determination is made as to whether or not the disconnected first message buffer identified by the temp pointer has been processed. This is done by examining the state field of the message buffer identified by the temp pointer.
- step 90 a the message buffer pointed to by temp can be returned to the free message buffer list 38 c .
- control proceeds to step 91 in which, as previously described in conjunction with FIG. 7A, the new head message buffer is returned to the DP portion of process A for processing.
- step 90 a for returning a message buffer identified by the temp pointer of step 90 a to the free message buffer list.
- FIG. 7C a flowchart of an embodiment of a method for performing step 90 a of FIG. 7B is shown in which a message buffer pointed to by temp is returned to the message buffer free list.
- the link fields i.e., AB-link and BA-link
- the message buffer is added to the free message buffer list, as in step 106 .
- FIG. 7D a flowchart of an embodiment of a method depicting more detail of steps 104 and 106 from FIG. 7C for returning a message buffer to the A end of the free message buffer list is shown.
- the message buffer identified by temp is initialized. This initialization includes initializing the BA link field to NULL, as in step 104 a , and setting the AB link field to identify the first message buffer currently on the free message buffer list at the A end of the free message buffer list, as in step 104 b .
- the message buffer identified by temp is added to the free message buffer list.
- the BA link field of the message buffer identified by the freelist field of A's MMI is set to point to the message buffer identified by temp, as in step 106 a .
- the freelist field 40 c of process A's MMI is set to identify the new first message buffer available for use by assigning it the value contained in temp.
- FIG. 8A a flowchart of an embodiment of a method for handling message buffers returned after data processing is complete is shown. As will be described in paragraphs that follow, this method includes steps for returning a message buffer to the message buffer free list in accordance with conditions and principles of the invention.
- step 96 a determination is made as to whether or not the message buffer which will possibly be returned to the free list is still the head of the process A message list 38 a . If a determination is made at step 96 that an attempt is being made to return the first message buffer being processed by process A to the free message buffer list 38 c , control proceeds to step 98 . At step 98 , a flag is set to indicate that the first message buffer has been read and processed, yet the first message buffer is not yet removed from process A's message list 38 a .
- step 96 If a determination is made at step 96 that the first message buffer of process A's message list 38 a is not the message attempting to be returned to the free list 38 c , control proceeds to step 100 where the message buffer link fields (i.e, AB link, and BA_link) are reinitialized and added to the free message buffer list 38 c in step 102 .
- the message buffer link fields i.e, AB link, and BA_link
- step 96 a a determination is made as to whether an attempt is being made to return to the free message buffer list 38 c , the head of process A's message list 38 a . This is done at step 96 a by comparing the address identified by a buf_ptr, a pointer, which identifies the address of a message buffer attempting to be returned to the free message buffer list, to the address identified by the local head field 40 a of process A's MMI. If a determination is made that these addresses are not the same, control proceeds to step 100 a.
- the message buffer identified by buf_ptr is initialized.
- the BA link field is initialized to NULL, and at step 100 b the AB link field is set to identify the first message buffer currently on the free message buffer list 38 c from the A end of the free message buffer list.
- the BA link field of the message buffer identified by the freelist field of A's MMI is set to point to the message buffer identified by buf_ptr.
- the freelist field 40 c of process A's MMI is set to identify the new first message buffer available for use by assigning it the value contained in buf_ptr.
- FIGS. 7A, 7B, 7 C, 7 D, 8 A and 8 B may be performed by the queue manager in this embodiment, such as queue manager 32 b to retrieve a message for processing by the DP portion 32 a , or after a message is returned by the DP portion 32 a to the queue manager 32 b .
- the method steps of the foregoing FIGs. provide for two particular points, as summarized below, at which a message buffer may be returned to the message buffer free list.
- FIGS. 7 A- 7 D may be performed when a message is retrieved by the queue manager for processing by DP.
- a message buffer may be returned to the free list at a first particular processing point when performing these method steps.
- a preferred embodiment may have multiple messages being processed resulting in multiple outstanding message buffers. Thus, in these foregoing FIGs a determination is made to see if a message buffer which was previously being processed may be returned to the free list provided that there is more than one element in a process' message buffer list.
- FIGS. 8A and 8B may be performed when the DP portion of a process has completed processing of a message.
- a message buffer may also be returned to the message buffer free list if certain conditions are satisfied at a second particular processing point.
- the message buffer which is the only one on the process message list is not returned to the free message buffer list. If the message being returned is not the head of the process message list, then the message buffer which contains the message just processed by DP is returned to the free message buffer list.
- freeList temp; ⁇ return (A_side.localHead); ⁇ else return (NULL); ⁇ A_end free (WORK_ENTRY *buf_ptr) - if buf_ptr is not the head of the work queue, push onto the free list, else mark it EMPTY ⁇ /** FIGS.
- each of processes A and B include a DP portion and a queue manager portion. Allocated to each of these portions are various tasks accomplished by performing method steps previously described.
- Other implementations applying the principles of the invention described in the foregoing embodiment may include a process with different portions and different task allocations in accordance therewith as known to those skilled in the art.
- the foregoing description affords a flexible and efficient technique for providing bidirectional message communication as between processes using a shared pool of message buffers in a synchronized manner.
- the foregoing technique does not require expensive, global locking mechanisms as those of the prior art, and includes adaptive buffer allocation techniques which minimize memory requirements while simultaneously avoiding the problem of uneven message flow as between processes A and B.
- the foregoing technique can easily be tuned for a variety of particular implementations, operating systems, and computer systems.
- Techniques used in the previously described embodiment generally promote efficient use of system resources, such as memory and processor time, and are scalable for use in a system having many user processes, as well as a single user process, and a single processor or a multiprocessor computer system.
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
Abstract
Description
- This invention relates generally to communications in a computer system and more particularly to performing synchronized message passing in a computer system using shared resources.
- As it is known in the art, message passing is one means of communicating between two or more processes. Messages can be sent between two or more processes executing in the same computer system, or between processes executing in different computer systems. Using message passing, a first process wishing to communicate with a second process sends the second process a message. The message is typically a quantity or packet of data sent from the first process to the second process.
- One class of techniques for communicating messages between two processes uses a single shared list in which there is always a buffer of one element in the list. A receiving process reads and processes a first message sent from a sending process only when a second message has been added to the list, such as that which occurs when the sending process sends a second message to the receiving process. In other words, the receiving process does not read, process, and delete a message until the message being accessed by the receiving process is not the same as the one being sent by the sending process. Such a technique only provides for message communication in one direction between two processes.
- Another class of techniques for communicating messages between two processes, process A and process B, uses a portion of memory that is shared between processes A and B and contains two shared message lists. The first shared list includes messages sent from process A to process B, and the second shared list includes messages sent from process B to process A. Both shared lists are accessed by each of the processes A and B either to send a message to the other process, or receive its own messages. For example, the first shared list is accessed by process A to send a message to process B by adding the message to the first shared list. The first shared list is also accessed by process B to read messages it receives by removing a message from the first shared list.
- Typically, data operations requiring access to one of the shared lists are included in a “critical section” of machine executable code providing exclusive access to a shared resource, such as a list, to an accessing process. Thus, when a process executes an operation in a critical section, it is guaranteed exclusive access to the shared resource.
- To provide for synchronization of a shared resource within a critical section, an operating system usually employs a locking mechanism by which a shared resource, such as the foregoing two shared lists included in the portion of memory, is exclusively accessed by one of multiple processes. For example, when one of the two processes, A and B, executes a data operation in a critical section to access the first shared list, a locking mechanism insures that the other process is excluded from also accessing the first shared list. A typical example of such locking mechanisms is the semaphore.
- One of the drawbacks of the foregoing technique of a critical section with a locking mechanism is the performance overhead incurred by using the locking mechanism to implement the mutual exclusion. The additional overhead includes requesting, obtaining, and releasing a lock on a shared resource each time a process is accessing the shared resource. This additional overhead adversely impacts system performance, for example, by increasing execution time and requiring computer system resources to instantiate the locking mechanism.
- Another drawback of the foregoing technique occurs in the event of an uneven flow of message traffic between processes A and B. If process A sends more messages to process B than process B sends to process A, there is an uneven flow of messages between process A and process B. If processes A and B each allocate their own message buffers, as used for storing data of a message, process A may request additional memory for a message buffer to send a message to process B while process B has unused message buffers. Thus, process A may require an excessive amount of memory for message buffers while process B has message buffers available for use.
- To provide for message buffer allocation while minimizing memory requirements in the event of uneven message traffic between process A and process B, a third shared list including “free” or available memory buffers is used. In other words, rather than have process A and process B each individually allocate memory for a message buffer when needed, a third shared list of available message buffers is used by both process A and process B. When process A or process B requires a message buffer to send a message, it first attempts to obtain an available message buffer from the third shared list of free message buffers.
- Using this third shared list is an improvement over the previously described technique in that use of the third shared list provides for adaptive message buffer allocation that minimizes memory allocation in the event of uneven message traffic between processes. However, a drawback of the use of a third shared list is that it represents an additional resource that is shared by process A and process B. Thus, a critical section is additionally required for data operations performed to obtain an available message buffer from the shared free list. Generally, using this third shared list further degrades system performance due to the additional use of the locking mechanism.
- Thus, there is required an efficient technique for bidirectional message communication between processes which does not require global locking mechanism for synchronization, and which provides adaptive message buffer allocation in the event of uneven message traffic.
- In accordance with principles of the present invention, to facilitate communication between a first and second process, access to shared resources is synchronized through discipline imposed upon updating and accessing the shared resources without requiring a global locking mechanism for synchronization.
- In accordance with one aspect of the invention is a method for sending messages from the first process to the second process. The method includes initializing a first message buffer containing message data, and adding the first message buffer to a list of incoming messages to the second process. Initializing the first message buffer includes setting a first flag indicating that the first message buffer contains a message not yet completely processed by the second process, and setting a first link field indicating that the first message buffer is the last message in the list of incoming messages to the second process. The list of incoming messages to the second process includes a second message buffer. The step of adding the first message buffer includes updating a second link field associated with the second mesage buffer to identify the first message buffer being added.
- Further in accordance with another aspect of the invention is an apparatus for sending a message from a first process to a second process. The apparatus includes a first data structure, a second data structure, and a messsage buffer. The first data structure is associated with the first process and includes a remote tail pointer identifying an incoming message associated with the second process. The second data structure is associated with the second process and includes a head pointer to the incoming message list associated with the second process. The message buffer includes a link field identifying a next message in the incoming message list, and a state field indicating whether the message buffer has been processed by the second process.
- Further in accordance with yet another aspect of the invention is a method for allocating a message buffer from a free list when sending a message between a first and second process. The method includes determining if the free list includes at least three message buffers and removing a message buffer from one end of the free list if there are at least three message buffers.
- Thus, there is provided an efficient technique for bidirectional message communication between processes which does not require global locking mechanism for synchronization, and which provides for adaptive message buffer allocation in the event of uneven message traffic.
- The above-mentioned and other features of the invention will now become apparent by reference to the following description taken in connection with the accompanying drawings, in which:
- FIG. 1 is a block diagram of a computer system;
- FIG. 2A is a block diagram of an embodiment of data structures and processes used in message communications between processes executing in the computer system of FIG. 1;
- FIG. 2B is a flowchart of an embodiment of a method of sending messages between processes executing in the computer system of FIG. 1;
- FIG. 2C is a flowchart of an embodiment of a method of receiving messages between processes executing in the computer system of FIG. 1;
- FIG. 3 is a block diagram of an embodiment of the data structures of FIG. 2 used in interprocess message communication;
- FIG. 4 is a flowchart of an embodiment of a method for allocating a buffer from a free list of available message buffers;
- FIG. 5 is a flowchart of an embodiment of a method for determining if there are more than two free message buffers in the free list of available message buffers and removing the message buffer from an end of the free list;
- FIG. 6A is a flowchart of an embodiment of a method for sending messages from process A to process B;
- FIG. 6B is a flowchart of an embodiment of a method depicting detailed steps of FIG. 6A for initializing a new message buffer and updating process A's pointer information;
- FIG. 7A is a flowchart of an embodiment of a method for retrieving a message from process A's message list;
- FIG. 7B is a flowchart of an embodiment of a method for performing more detailed steps from FIG. 7A for retrieving a message from A's message list;
- FIG. 7C is a flowchart of an embodiment of a method for performing more detailed steps from FIG. 7B for returning a message buffer to the free list;
- FIG. 7D is a flowchart of an embodiment of a method for performing more detailed steps from FIG. 7C for returning a message buffer to the free list;
- FIG. 8A is a flowchart of an embodiment of a method performed upon completion of processing a message buffer for handling message buffers after message processing is complete; and
- FIG. 8B is a flowchart of an embodiment of a method depicting more detailed steps of FIG. 8A for handling message buffers after data processing is complete.
- Referring now to FIG. 1, an embodiment of the
computer system 10 is shown to include a plurality of host computers 12 a-12 c, acommunications medium 14, storage subsystems 24 a-24 b, and sharedmemory 30. Each host 12 a-12 c is directly connected through acommunication medium 14 tostorage subsystems memory 30.Host computer 12 a includes aprocessor 16 a,local memory 18 a and communications interface 20 a by whichhost computer 12 a is connected tocommunications medium 14. Similarly, each host 12 b-12 c, respectively, includes aprocessor 16 b-16 c,local memory 18b-18 c andcommunications interface 20 b-20 c. Each of the host systems 12 b-12 c are respectively connected tocommunication medium 14 bycommunications interface 20 b-20 c. Thecommunication medium 14 may be, for example, a system bus. -
Storage subsystem 24 a includesstorage device 28 a which is controlled bydevice controller 26 a. Each of the host computers 12 a-12 c can communicate over communications medium 14 withstorage subsystems storage subsystem 24 b also includes adevice controller 26 b and device 28 b. Each of the storage subsystems 24 a-24 b includes a device, such as a disk, and device controller, such as a disk controller, particular for that device included in the storage subsystem. Each of the host computer systems 12 a-12 c typically perform I/O operations such as read and write operations todevices 28 a and 28 b. Also included in FIG. 1 is sharedmemory 30 which can be accessed by any of the host processors 12 a-12 c. Generally, sharedmemory 30 is a shared resource for use by any of host computers 12 a-12 c. - It should be noted that other resources may be included in an embodiment of a computer system as shown in FIG. 1 for use between host computer systems12 a-12 c. Additionally, a computer system may also contain more or fewer host computer systems than included in the embodiment of FIG. 1.
- Referring now to FIG. 2A, a block diagram of an embodiment of the data structures and processes used in message communication between processes executing in a computer system of FIG. 1 is shown. Included in FIG. 2A are
process A 32 andprocess B 34. For example,process A 32 executes in one of the host computer systems 12 a-12 c of FIG. 1. Similarly,process B 34 also executes in one of the host computer systems 12 a-12 c of FIG. 1. - One or more of the executing processes in the
computer system 10 may be required to communicate with another process also executing in thecomputer system 10. Shown in FIG. 2 isprocess A 32 communicating withprocess B 34. Communication between process A and process B may include, for example, process A sending a message to process B, or process B sending a message to process A. Included in sharedmemory 30 of FIG. 2 are data structures used to facilitate communication between process A and process B. Included in sharedmemory 30 of FIG. 2 is a processA message list 38 a, a processB message list 38 b and a freemessage buffer list 38 c. Process Amessage list 38 a contains those messages which process A receives from process B. Similarly, processB message list 38 b contains those messages received from process A. The freemessage buffer list 38 c contains available message buffers which a process sending a message to another process may obtain for sending a message. The use of these data structures and the communication between process A and process B will be described in more detail in paragraphs that follow. -
Process A 32 includes adata processing portion 32 a (DP) and aqueue manager portion 32 b. Generally, thequeue manager 32 b andDP 32 a interact to manage process A'smessage list 38 a, and process the messages included in this message list. In particular, thequeue manager 32 b performs management functions of process A'smessage list 38 a such as adding and deleting message buffers from process A's message list. Generally,DP portion 32 a is responsible for processing the contents of a message included in process A's message list. Similarly,process B 34 contains aDP portion 34 a and aqueue manager 34 b. Both process A and process B when executing in a computer system of FIG. 1 access the sharedmemory 30 overcommunications medium 14. - Also included in the local memory in which
process A 32 is loaded is process A message management information (MMI) 40. Process A's MMI includes information enabling proper management of the various data structures included in sharedmemory 30 for process A to send and receive messages. Similarly,process B 34 contains process B message management information (MMI) 42 which is included in the local memory in which process B is loaded. Process B's MMI includes information enabling proper management of the data structures included in sharedmemory 30 for process B to send and receive messages. Each of the data structures included in sharedmemory 30 and the MMI are described in more detail in following paragraphs. - Referring now to FIG. 2B, shown is a flowchart of an embodiment of a method of sending a message from a sending process to a receiving process. In this example, A is the sending process sending a message to process B, the receiving process. At
step 33 a, a message buffer is allocated, as from the freemessage buffer list 38 a, and initialized. Atstep 33 b, the message fromstep 33 a is sent to the receiving process B, as by including the message in ProcessB Message List 38 b. Control returns to step 33 a to send additional messages as needed to process B. - Referring now to FIG. 2C, shown is a flowchart of an embodiment of a method for receiving messages from a sending process. In this example, process B receives messages from a sending process, process A. At
step 33 c, process B determines that it has received a message from process A. As known to those skilled in the art, process B can determine that it has received a message from process A using one of a variety of techniques. For example, process B may use a polling mechanism in which process B continually checks for incoming messages after a predetermined amount of time. Process B may also use an interrupt mechanism by which process B receives an interrupt when a new message is received. - At
step 33 d, the newly received message is retrieved and processed by process B. When process B has completed processing the message, end of message processing is performed, as instep 33 e, in which the message buffer including the message just processed may be returned to the freemessage buffer list 38 a in accordance with other conditions and principles of the invention that are described in following paragraphs. - Referring now to FIG. 3, shown is a block diagram of an embodiment of data structures included in FIG. 2. It should be noted that these data structures represent a “snapshot” of the data structures during the execution of process A and process B. Process A's
MMI 40 includes alocal head pointer 40 a, aremote tail pointer 40 b and afree list pointer 40 c. Thelocal head pointer 40 a points to the head, or first message, of the message list for process A 38 a.Remote tail pointer 40 b identifies the tail end or the last element on process B'smessage list 38 b. Thefree list pointer 40 c identifies the first message buffer in the list of free message buffers 38 c available for use by process A when sending a message to process B. Similarly, process B'sMMI 42 contains alocal head pointer 42 a identifying the first message on process B's list ofincoming messages 38 b. Theremote tail pointer 42 b identifies the last element on process A'smessage list 38 a.Free list pointer 42 c identifies the first message buffer in the list offree pointers 38 c available for use by process B when sending a message to process A. - Message lists in this embodiment, such as message lists38 a and 38 b, always include at least one element. At initialization prior to sending or receiving messages, each message list is initialized with a dummy message buffer element marked as “already processed”. In the instant case, this “already processed” state is indicated by setting the dummy element's state field to EMPTY. Other implementations may indicate this processing state in a variety of ways dependant upon the implementation.
- As will be described in paragraphs that follow, methods for subsequently managing the data structures of FIG. 3 guarantee that there will always be at least one element in each of the message lists38 a and 38 b. Using these methods, processing may or may not have completed on the first element of each message list. If there is more than one element on one of the message lists, the methods guarantee that processing has not begun on the second and subsequent elements, if any, included in message list.
- Process A's
message list 38 a shown in FIG. 3 includes two message buffers 44 and 46. Note that, as previously described, this is a “snapshot” of the data structures, such asmessage list 38 a. Taking a “snapshot” of the data structures at other points of execution may result in different data structure states. Thefirst message buffer 44 included in process A'smessage list 38 a is identified by thelocal head pointer 40 a of process A'sMMI 40.Message buffer 44 contains an AB link field 44 a, aBA link field 44 b, awork link field 44 c and astate field 44 d followed by any additional data for the message to be sent to process A. It should be noted that all message buffers appearing in any of the data structures—process A message list, process B message list or the freemessage buffer list 38 c—include these same fields. However, different fields are used in conjunction with different data structures as will be described in paragraphs that follow. - The
work link field 44 c and thestate field 44 d are used in managing the process Amessage list 38 a. The AB link field 44 a and theBA link field 44 b will be described in conjunction with the free message buffer list since they are used in conjunction with the management of this data structure. Sincefields 44 a and 44 b are not used in the management of process A'smessage list 38 a, values for these fields are not shown. Thework link field 44 c identifies the next message buffer in the linked list of messages to be processed by process A. In FIG. 3work link field 44 c identifies the next message buffer in process A'smessage list 38 a, which ismessage buffer 46. If a message buffer such as 46 is the last message buffer included in process A'smessage list 38 a, itswork link field 46 c has the value NULL to indicate the end of the list. The state field, such as 44 d, has the value of either EMPTY or FULL indicating whether the message included in the corresponding message buffer has been completely processed.Message buffer 44 has itsstate field 44 d set to EMPTY indicating that process A has completed processing of the message data included inmessage buffer 44. In contrast,message buffer 46 containsstate field 46 d which is set to FULL indicating that the message inmessage buffer 46 has not yet been completely processed, for example, by theDP portion 32 a ofprocess A 32. Process B'smessage list 38 b contains message buffers similar to those described previously in conjunction with process A'smessage list 38 a. -
Fields MMI 40 are used in managing process A'smessage list 38 a for processing A's incoming messages, and also for sending a message from process A to process B by adding a message to process B'smessage list 38 b. In this example, process A'sMMI 40 contains theremote tail field 40 b identifying thelast element 50 of process B'smessage list 38 b. Process A sends a message to Process B by adding it to the end of Process B'smessage list 38 b usingremote tail pointer 40 b.Fields MMI 42 are used similarly to Process A'sfields - Used in managing the free
message buffer list 38 c are the free list fields, 40 c and 42 c, respectively, of theMMI data structures message buffer list 38 c for use in sending a message to process B, process A examines the free message buffer list using the AB link field to identify a next consecutive message buffer. In other words, the AB link fields of the message buffers contained in the free message buffer list form a chain of message buffers whose order is privately used by process A in allocating message buffers. Similarly, the BA link fields of the message buffers contained in the freemessage buffer list 38 c form a chain of message buffers whose ordering is privately used by process B in allocating message buffers. Process B reads the BA link field of a first message buffer to determine whether that message buffer is available for use by process B. Process A does not read the BA link field in determining whether that message buffer is available for its use. The BA link field is private for use only in memory buffer allocation by process B. Similarly, the AB link field is a private field available for use by process A in memory buffer allocation, and is not used by process B when determining the next element available for use on the freemessage buffer list 38 c. - A more detailed description of how these data structures and their fields are used in message passing between process A and process B is described in paragraphs that follow.
- Referring now to FIG. 4, a flowchart of an embodiment of a method for allocating a buffer from the
free list 38 c is shown. Shown in FIG. 4 are the method steps for removing a message buffer from the A process end of thefree list 38 c, such as when process A requires a message buffer to send a message to process B. One skilled in the art will note that these steps may be generalized and modified for also sending a message from process B to process A. Instep 66 a determination is made as to whether there are more than two free message buffers in thefree list 38 c. If there are more than two free message buffers in the freemessage buffer list 38 c, a message buffer from the A end of the free list is removed, as instep 76. The message buffer removed from the freemessage buffer list 38 c is used by process A. In this embodiment ofprocess A 32, thequeue manager portion 32 b performs the steps of FIG. 4 to remove the message buffer from the freemessage buffer list 38 c and return it for use by theDP portion 32 a. - If a determination is made at
step 66 that there are not more than two free message buffers in the free message buffer list, control proceeds to step 68 where a determination is made as to whether or not there are sufficient system resources to allocate an additional message buffer. If there are insufficient system resources to allocate an additional message buffer, control proceeds to step 70 where an indication is made that insufficient system resources are available to allocate a message buffer. If a determination is made atstep 68 that there are sufficient system resources available to allocate an additional message buffer, control proceeds to step 72 where the memory for the new additional message buffer is allocated. This new message buffer is returned to process A for use instep 74. - Referring now to FIG. 5, a flowchart of an embodiment of a method for performing
steps steps step 66 a, a temp pointer receives the contents of the AB_link field of the first message buffer that indicates the second message buffer, if any, of the freemessage buffer list 38 c. A determination is made at step 66 b as to whether or not the temporary pointer is NULL. If the temporary pointer is NULL, it indicates that there is one message buffer in the freemessage buffer list 38 b. If the temp pointer is not NULL, control proceeds to step 66 c. Prior to executingstep 66 c, it is known that the free message buffer list contains at least two elements. Atstep 66 c, a determination is made as to whether or not the free message buffer list contains at least three message buffers. This is determined by examining the AB link field of the message buffer identified by the temp pointer, since temp points to the second element of the list. If the AB link field of the message buffer identified by the temp pointer is not NULL, it indicates that a third message buffer is contained in the message buffer list. Thus, atstep 66 c if a determination is made that the AB link field of the message buffer identified by the temp pointer is not NULL, control proceeds to step 76 a to remove the first message buffer from the freemessage buffer list 38 c. Ifstep 66 c determines that there are not at least three message buffers in the free message buffer list, control proceeds to step 68. - At
step 76 a the BA link field of the message buffer pointed to by temp is assigned a value of NULL to remove the element from the freemessage buffer list 38 c. Atstep 76 b the temp pointer is assigned to point to the message buffer identified by A.freelist which is the message buffer just allocated or removed from the freemessage buffer list 38 c. Atstep 76 c process A'sMMI freelist field 40 c is updated to point to the next message buffer of thelist 38 c available for use by process A. - Referring now to FIG. 6A, a flowchart of an embodiment of the method steps for sending a message from process A to process B is shown. At
step 80 a new message buffer is initialized. The new message buffer initialized instep 80 is allocated from the free message buffer list using the steps previously described in conjunction with FIGS. 4 and 5. Atstep 82 the new message initialized instep 80 is added to process B'smessage list 38 b using A'sMMI information 40. - Referring now to FIG. 6B, a flowchart of an embodiment of a method for performing the steps previously described in conjunction with FIG. 6A are shown in more detail when using the data structures previously described in conjunction with FIG. 3 of this particular embodiment.
- In
steps 80 a-80 c, the new message buffer is initialized. Instep 80 a, its work link field is initialized to NULL. Instep 80 b, its state field is set to FULL. At step 80 c, the data of the message is added to the new message buffer.Steps 82 a-82 b add the new message buffer to process B'smessage list 38 b using process A'sMMI 40. The new message buffer is added to the tail of process B's message list. Using theremote tail field 40 b, the work link field of the last message buffer contained in processB message list 38 b is updated to point to the new message buffer. Atstep 82 b, A's MMI information is updated such that the remote tail of process A'sMMI 40 points to the new message buffer just added to the tail of process B'smessage list 38 b. - Using the data structures of FIG. 3 as previously described, new messages, as sent from process A to process B, are added or appended to the tail of the receiving process's message list by the sending process using its MMI.
- Referring now to FIG. 7A, a flowchart of an embodiment of a method for retrieving a message from a message list is shown. In particular, the method steps of FIG. 7A depict retrieving a message received by process A from process B. At
step 84 a determination is made as to whether or not there is more than one message buffer in process A'smessage list 38 a. In accordance with principles of the invention, each process's message list, such as 38 a and 38 b, always contains at least one element. If a determination atstep 84 indicates that there is more than one message buffer included in A'smessage list 38 a, then a message is returned for processing and an attempt is made to free a message buffer from A'smessage list 38 a. However, if there is only a single message buffer in process A'smessage list 38 a, control proceeds to step 85 where the method terminates. - If a determination is made at
step 84 that process A'smessage list 38 a contains more than one message buffer, control proceeds to step 86. Atstep 86, the message buffer from the head of process A's message list is disconnected from the remaining portion of A's message list by updating A's MMI. Atstep 88 a determination is made as to whether or not the message buffer disconnected instep 86 has been processed to completion. If a determination is made atstep 88 that the message has not been processed to completion, control proceeds to step 91 where the new head of A's message list is passed to process A's DP portion. In other words, if A is not done processing the message buffer, it cannot yet be returned to the free list as available for use. If a determination is made atstep 88 that the message has been processed, control proceeds to step 90 where the dequeued prior head message of A'smessage list 38 a is returned to the freemessage buffer list 38 c. Atstep 91, the new head message buffer of A's message list is passed to A's DP for processing. - Referring now to FIG. 7B, a flowchart of an embodiment of a method for retrieving a message received by process A from process B is shown in which the method uses the data structures previously described in conjunction with FIG. 3. At
step 84 a, a determination is made using the data structures of FIG. 3 as to whether or not there is more than one message buffer in process A'smessage list 38 a. This is determined by examining the work link field of the message buffer identified by thelocal head field 40 a of process A'sMMI data structure 40. If this work link field is NULL, control proceeds to step 85. If this work link field is not NULL, control proceeds to step 86 a where a temp pointer is used to identify the first message buffer on process A'smessage list 38 a. Atstep 86 b, the first message buffer in process A's message list is disconnected from the remainder of A's message list by updating A's MMI fieldlocal head 40 a to point to the next consecutive message buffer in process A's message list. Atstep 88 a, a determination is made as to whether or not the disconnected first message buffer identified by the temp pointer has been processed. This is done by examining the state field of the message buffer identified by the temp pointer. If the state field is EMPTY, it indicates that this message has been processed and control proceeds to step 90 a where the message buffer pointed to by temp can be returned to the freemessage buffer list 38 c. In either case, control proceeds to step 91 in which, as previously described in conjunction with FIG. 7A, the new head message buffer is returned to the DP portion of process A for processing. - Described in conjunction with FIG. 7C in following text are details of
step 90 a for returning a message buffer identified by the temp pointer ofstep 90 a to the free message buffer list. - Referring now to FIG. 7C, a flowchart of an embodiment of a method for performing
step 90 a of FIG. 7B is shown in which a message buffer pointed to by temp is returned to the message buffer free list. Atstep 104, the link fields (i.e., AB-link and BA-link) are reinitialized and the message buffer is added to the free message buffer list, as instep 106. - Referring now to FIG. 7D, a flowchart of an embodiment of a method depicting more detail of
steps steps step 104 a, and setting the AB link field to identify the first message buffer currently on the free message buffer list at the A end of the free message buffer list, as instep 104 b. The message buffer identified by temp is added to the free message buffer list. The BA link field of the message buffer identified by the freelist field of A's MMI is set to point to the message buffer identified by temp, as instep 106 a. Atstep 106 b, thefreelist field 40 c of process A's MMI is set to identify the new first message buffer available for use by assigning it the value contained in temp. - Referring now to FIG. 8A, a flowchart of an embodiment of a method for handling message buffers returned after data processing is complete is shown. As will be described in paragraphs that follow, this method includes steps for returning a message buffer to the message buffer free list in accordance with conditions and principles of the invention.
- At
step 96 a determination is made as to whether or not the message buffer which will possibly be returned to the free list is still the head of the process Amessage list 38 a. If a determination is made atstep 96 that an attempt is being made to return the first message buffer being processed by process A to the freemessage buffer list 38 c, control proceeds to step 98. Atstep 98, a flag is set to indicate that the first message buffer has been read and processed, yet the first message buffer is not yet removed from process A'smessage list 38 a. If a determination is made atstep 96 that the first message buffer of process A'smessage list 38 a is not the message attempting to be returned to thefree list 38 c, control proceeds to step 100 where the message buffer link fields (i.e, AB link, and BA_link) are reinitialized and added to the freemessage buffer list 38 c instep 102. - Referring now to FIG. 8B, the method steps of FIG. 8A are shown as used in this particular embodiment with the data structures of FIG. 3. At
step 96 a, a determination is made as to whether an attempt is being made to return to the freemessage buffer list 38 c, the head of process A'smessage list 38 a. This is done atstep 96 a by comparing the address identified by a buf_ptr, a pointer, which identifies the address of a message buffer attempting to be returned to the free message buffer list, to the address identified by thelocal head field 40 a of process A's MMI. If a determination is made that these addresses are not the same, control proceeds to step 100 a. - At
steps step 100 b the AB link field is set to identify the first message buffer currently on the freemessage buffer list 38 c from the A end of the free message buffer list. Atstep 102 a, the BA link field of the message buffer identified by the freelist field of A's MMI is set to point to the message buffer identified by buf_ptr. Atstep 102 b, thefreelist field 40 c of process A's MMI is set to identify the new first message buffer available for use by assigning it the value contained in buf_ptr. - In summary, the methods depicted in FIGS. 7A, 7B,7C, 7D, 8A and 8B may be performed by the queue manager in this embodiment, such as
queue manager 32 b to retrieve a message for processing by theDP portion 32 a, or after a message is returned by theDP portion 32 a to thequeue manager 32 b. The method steps of the foregoing FIGs. provide for two particular points, as summarized below, at which a message buffer may be returned to the message buffer free list. - The method steps of FIGS.7A-7D may be performed when a message is retrieved by the queue manager for processing by DP. A message buffer may be returned to the free list at a first particular processing point when performing these method steps. A preferred embodiment may have multiple messages being processed resulting in multiple outstanding message buffers. Thus, in these foregoing FIGs a determination is made to see if a message buffer which was previously being processed may be returned to the free list provided that there is more than one element in a process' message buffer list.
- The method steps of FIGS. 8A and 8B may be performed when the DP portion of a process has completed processing of a message. In performing these method steps, a message buffer may also be returned to the message buffer free list if certain conditions are satisfied at a second particular processing point. In accordance with these methods previously described, if there is only one message buffer remaining on the process message list (such as message lists38 a and 38 b) when DP has complete processing, the message buffer which is the only one on the process message list is not returned to the free message buffer list. If the message being returned is not the head of the process message list, then the message buffer which contains the message just processed by DP is returned to the free message buffer list.
- Included below is a C-style pseudo code description of the methods previously described in conjunction with FIGS. 4, 5,6A, 6B, 7A-7D, 8A and 8B using the data structures described in conjunction with FIG. 3.
/**data declarations**/ typedef volatile struct work WORK_ENTRY; struct work ( int state; /* EMPTY or FULL */ WORK_ENTRY *workLink; /* next on work queue */ WORK_ENTRY *ABfreeLink; /* next free entry in AB direction */ WORK_ENTRY *BAfreeLink; /* next free entry in BA direction */ uint32 body[BODY_SIZE]; /* for some specified BODY_SIZE */ }WORK_ENTRY typedef struct queue_head { WORK_ENTRY *localHead; WORK_ENTRY *remoteTail; WORK_ENTRY *freeList; } HEADER; HEADER A_side, B_side; AllocateAB() - remove the buffer at the A-end of the free list if it points to another free entry. /* FIG. 4 and 5 method steps */ } WORK_ENTRY *temp; /* use temp to avoid possible conflict from reading ABfreeLink twice */ temp = A_side.freelist->ABfreeLink; if (temp != NULL) /* not a 1 element list */ if (temp ->ABfreeLink ! = NULL) { /* not a 2 element list */ temp ->BAfreeLink = NULL; /* remove from list*/ temp = A_side.freelist; A_side.freelist = A_side.freelist->ABfreeLink; return(temp) ; } /* at this point, an additional buffer can be allocated and it /* will augment the free buffer pool, or several can be /* allocated and all but one added to the free list. Lastly, /* if sufficient resources have already been allocated, a NULL /* can be returned to tell the requestor to try again later. */ return (get_buffer_fromsystem() ); } Send_to_B (WORK_ENTRY *buf_ptr) - enqueue full buffer on B-workq /* FIG. 6A, 6B method steps **/ { buf_ptr->workLink = NULL; buf_ptr->state = FULL; A_side.remoteTail->workLink = buf_ptr; A_side.remoteTail = buf_ptr; } Receive_from_B() - if the local work queue contains more than one buffer, dequeue the buffer at the head of the queue; if that buffer was EMPTY, return it to the free queue; return the address of the buffer now at the head of the work queue. Otherwise, return NULL. { WORK_ENTRY *temp; /** FIGS. 7A, 7B, 7C, 7D method steps **/ if (A_side.localHead->workLink != NULL) { temp =A_side.localHead; A_side.localHead = A_side.localHead->workLink; if (temp->state == EMPTY) { temp->BAfreeLink = NULL; temp->ABfreeLink = A_side.free_list; A_side.freeList->BAfreeLink = temp; A_side. freeList=temp; } return (A_side.localHead); } else return (NULL); } A_end free (WORK_ENTRY *buf_ptr) - if buf_ptr is not the head of the work queue, push onto the free list, else mark it EMPTY { /** FIGS. 8A, 8B method steps **/ if (bufptr <> A_side.localHead) { buf_ptr->BAfreeLink = NULL; /* initialize pointer fields in buffer */ buf_ptr->ABfreeLink = A_side.freeList; A_side.freeList->BAfreeLink =buf_ptr; /* append to list */ A_side.freeList =buf_ptr; } else A_side.localHead->state = EMPTY; } - It should be noted that various aspects of the foregoing description may vary with implementation of the principles of the invention. For example, in the foregoing embodiment each of processes A and B include a DP portion and a queue manager portion. Allocated to each of these portions are various tasks accomplished by performing method steps previously described. Other implementations applying the principles of the invention described in the foregoing embodiment may include a process with different portions and different task allocations in accordance therewith as known to those skilled in the art.
- The foregoing description affords a flexible and efficient technique for providing bidirectional message communication as between processes using a shared pool of message buffers in a synchronized manner. The foregoing technique does not require expensive, global locking mechanisms as those of the prior art, and includes adaptive buffer allocation techniques which minimize memory requirements while simultaneously avoiding the problem of uneven message flow as between processes A and B. The foregoing technique can easily be tuned for a variety of particular implementations, operating systems, and computer systems.
- Techniques used in the previously described embodiment generally promote efficient use of system resources, such as memory and processor time, and are scalable for use in a system having many user processes, as well as a single user process, and a single processor or a multiprocessor computer system.
- Having described preferred embodiments of the invention, it will now become apparent to those of skill in the art that other embodiments incorporating its concepts may be provided. It is felt, therefore, that this invention should not be limited to the disclosed embodiment, but rather should be limited only by the scope of the appended claims.
Claims (7)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/883,997 US6385658B2 (en) | 1997-06-27 | 1997-06-27 | Method and apparatus for synchronized message passing using shared resources |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/883,997 US6385658B2 (en) | 1997-06-27 | 1997-06-27 | Method and apparatus for synchronized message passing using shared resources |
Publications (2)
Publication Number | Publication Date |
---|---|
US20010014918A1 true US20010014918A1 (en) | 2001-08-16 |
US6385658B2 US6385658B2 (en) | 2002-05-07 |
Family
ID=25383760
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US08/883,997 Expired - Fee Related US6385658B2 (en) | 1997-06-27 | 1997-06-27 | Method and apparatus for synchronized message passing using shared resources |
Country Status (1)
Country | Link |
---|---|
US (1) | US6385658B2 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6553438B1 (en) * | 2000-04-24 | 2003-04-22 | Intel Corporation | Methods and system for message resource pool with asynchronous and synchronous modes of operation |
US20040003007A1 (en) * | 2002-06-28 | 2004-01-01 | Prall John M. | Windows management instrument synchronized repository provider |
US7596563B1 (en) * | 1999-10-28 | 2009-09-29 | Hewlett-Packard Development Company, L.P. | Computerized file system and method |
US20100269120A1 (en) * | 2009-04-17 | 2010-10-21 | Nokia Corporation | Method, apparatus and computer program product for sharing resources via an interprocess communication |
CN107783845A (en) * | 2016-08-25 | 2018-03-09 | 阿里巴巴集团控股有限公司 | Message delivery system, method and apparatus |
CN109408159A (en) * | 2017-08-14 | 2019-03-01 | 阿里巴巴集团控股有限公司 | Interface message is shown and processing method, device and electronic equipment |
Families Citing this family (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6477584B1 (en) * | 1997-03-21 | 2002-11-05 | Lsi Logic Corporation | Message FIFO empty early warning method |
DE19957594B4 (en) * | 1999-11-30 | 2004-08-26 | OCé PRINTING SYSTEMS GMBH | Method for synchronizing threads of a computer program |
AU2001257567A1 (en) * | 2000-05-08 | 2001-11-20 | Transilica, Inc. | Distributed applications in a portable thread environment |
US6715059B2 (en) * | 2000-07-26 | 2004-03-30 | Tas Holdings, Inc. | Methods and systems for a shared memory unit with extendable functions |
US20020174258A1 (en) * | 2001-05-18 | 2002-11-21 | Dale Michele Zampetti | System and method for providing non-blocking shared structures |
US7240089B2 (en) * | 2001-12-10 | 2007-07-03 | International Business Machines Corporation | Message queuing method, system, and program product with reusable pooling component |
US7107413B2 (en) * | 2001-12-17 | 2006-09-12 | Intel Corporation | Write queue descriptor count instruction for high speed queuing |
US7269179B2 (en) * | 2001-12-18 | 2007-09-11 | Intel Corporation | Control mechanisms for enqueue and dequeue operations in a pipelined network processor |
US7895239B2 (en) | 2002-01-04 | 2011-02-22 | Intel Corporation | Queue arrays in network devices |
US7181573B2 (en) * | 2002-01-07 | 2007-02-20 | Intel Corporation | Queue array caching in network devices |
US7149226B2 (en) * | 2002-02-01 | 2006-12-12 | Intel Corporation | Processing data packets |
US20040107240A1 (en) * | 2002-12-02 | 2004-06-03 | Globespan Virata Incorporated | Method and system for intertask messaging between multiple processors |
US8181182B1 (en) * | 2004-11-16 | 2012-05-15 | Oracle America, Inc. | Resource allocation brokering in nested containers |
US20070011687A1 (en) * | 2005-07-08 | 2007-01-11 | Microsoft Corporation | Inter-process message passing |
US7949815B2 (en) * | 2006-09-27 | 2011-05-24 | Intel Corporation | Virtual heterogeneous channel for message passing |
CN103455380A (en) * | 2012-06-05 | 2013-12-18 | 上海斐讯数据通信技术有限公司 | Multi-process communication system and establishment and communication method thereof |
US10599494B2 (en) * | 2016-05-27 | 2020-03-24 | Huawei Technologies Co., Ltd. | Method and system of performing inter-process communication between OS-level containers in user space |
CN107515785A (en) * | 2016-06-16 | 2017-12-26 | 大唐移动通信设备有限公司 | A kind of EMS memory management process and device |
US11226852B2 (en) | 2016-11-25 | 2022-01-18 | Genetec Inc. | System for inter-process communication |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5742793A (en) * | 1991-12-18 | 1998-04-21 | Intel Corporation | Method and apparatus for dynamic memory management by association of free memory blocks using a binary tree organized in an address and size dependent manner |
US5315707A (en) * | 1992-01-10 | 1994-05-24 | Digital Equipment Corporation | Multiprocessor buffer system |
EP0551789A1 (en) * | 1992-01-17 | 1993-07-21 | International Business Machines Corporation | Apparatus for recovering lost buffers in a data processing system |
US5797005A (en) * | 1994-12-30 | 1998-08-18 | International Business Machines Corporation | Shared queue structure for data integrity |
US5644575A (en) * | 1995-12-06 | 1997-07-01 | Dsc Communications Corporation | Managing memory within a local telecommunications network |
-
1997
- 1997-06-27 US US08/883,997 patent/US6385658B2/en not_active Expired - Fee Related
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7596563B1 (en) * | 1999-10-28 | 2009-09-29 | Hewlett-Packard Development Company, L.P. | Computerized file system and method |
US6553438B1 (en) * | 2000-04-24 | 2003-04-22 | Intel Corporation | Methods and system for message resource pool with asynchronous and synchronous modes of operation |
US20040003007A1 (en) * | 2002-06-28 | 2004-01-01 | Prall John M. | Windows management instrument synchronized repository provider |
US20100269120A1 (en) * | 2009-04-17 | 2010-10-21 | Nokia Corporation | Method, apparatus and computer program product for sharing resources via an interprocess communication |
US8220004B2 (en) * | 2009-04-17 | 2012-07-10 | Nokia Corporation | Method, apparatus and computer program product for sharing resources via an interprocess communication |
TWI486878B (en) * | 2009-04-17 | 2015-06-01 | Nokia Corp | Method, apparatus and computer program product for sharing resources via an interprocess communication |
CN107783845A (en) * | 2016-08-25 | 2018-03-09 | 阿里巴巴集团控股有限公司 | Message delivery system, method and apparatus |
CN109408159A (en) * | 2017-08-14 | 2019-03-01 | 阿里巴巴集团控股有限公司 | Interface message is shown and processing method, device and electronic equipment |
Also Published As
Publication number | Publication date |
---|---|
US6385658B2 (en) | 2002-05-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6385658B2 (en) | Method and apparatus for synchronized message passing using shared resources | |
US5652885A (en) | Interprocess communications system and method utilizing shared memory for message transfer and datagram sockets for message control | |
US5448698A (en) | Inter-processor communication system in which messages are stored at locations specified by the sender | |
US6823472B1 (en) | Shared resource manager for multiprocessor computer system | |
US6889269B2 (en) | Non-blocking concurrent queues with direct node access by threads | |
US6023712A (en) | Method and apparatus for brokering memory resources | |
US5905889A (en) | Resource management system using next available integer from an integer pool and returning the integer thereto as the next available integer upon completion of use | |
US7069373B2 (en) | USB endpoint controller flexible memory management | |
US6557056B1 (en) | Method and apparatus for exchanging data between transactional and non-transactional input/output systems in a multi-processing, shared memory environment | |
US7152231B1 (en) | High speed interprocess communication | |
US7337275B2 (en) | Free list and ring data structure management | |
US5987550A (en) | Lock mechanism for shared resources in a data processing system | |
US5961606A (en) | System and method for remote buffer allocation in exported memory segments and message passing between network nodes | |
EP0343820A2 (en) | Temporary state preservation for a distributed file service | |
US5781741A (en) | Message communications system in a parallel computer | |
US20110078214A1 (en) | Managing message queues | |
EP1741023B1 (en) | Communicating protected data within locking messages | |
US7770177B2 (en) | System for memory reclamation based on thread entry and release request times | |
JPS6292061A (en) | Series arrangement of access to common source | |
US7353515B1 (en) | Method and apparatus for dynamic allocation and management of semaphores for accessing shared resources | |
US6886031B2 (en) | Efficient connection and memory management for message passing on a single SMP or a cluster of SMPs | |
US6865741B1 (en) | Determining completion of transactions processing in a dynamically changing network | |
EP0694850A2 (en) | Method and apparatus for distributing control messages between interconnected processing elements | |
US5386514A (en) | Queue apparatus and mechanics for a communications interface architecture | |
EP0618709A2 (en) | Memory manager for a multichannel network interface |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: DIGITAL EQUIPMENT CORPORATION, MASSACHUSETTS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HARTER, PAUL K., JR.;FRASER, JAMES I., JR.;REEL/FRAME:008630/0124 Effective date: 19970627 |
|
AS | Assignment |
Owner name: COMPAQ COMPUTER CORPORATION, TEXAS Free format text: MERGER;ASSIGNOR:DIGITAL EQUIPMENT CORPORATION;REEL/FRAME:011776/0688 Effective date: 19991209 |
|
AS | Assignment |
Owner name: COMPAQ INFORMATION TECHNOLOGIES GROUP, L.P., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:COMPAQ COMPUTER CORPORATION;REEL/FRAME:012468/0308 Effective date: 20010620 |
|
FEPP | Fee payment procedure |
Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FEPP | Fee payment procedure |
Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS Free format text: CHANGE OF NAME;ASSIGNOR:COMPAQ INFORMATION TECHNOLOGIES GROUP, L.P.;REEL/FRAME:019028/0515 Effective date: 20021001 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees | ||
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20140507 |