IES981034A2

IES981034A2 - An inter-computer communications apparatus

Info

Publication number: IES981034A2
Application number: IE981034A
Authority: IE
Inventors: John Power
Original assignee: Scp Powersoft Ltd
Priority date: 1997-12-09
Filing date: 1998-12-09
Publication date: 1999-08-11
Also published as: IE981035A1; GB9827138D0; GB2333868A; GB2333868B

Abstract

An apparatus connects to a legacy host computer system of the type having a non-hierarchical file structure for controlling simultaneous communication between it and local processing devices. Provided that the server is awaiting requests, a data retrieval request from a local processor is detected 3 on a preset communications port, and a check is made 4 to determine whether a pool session is available from a queue QA. If a session is available the request is assigned 6 to that session. The server reads 7 the data stream associated with the request, and parses 8 the stream. Connection details are extracted 9 from the requests, and it is determined 10 if the request forms part of an existing conversation. The apparatus improves the efficiency and transparency of communications between computer systems to release legacy based host information for Internet and Intranet technologies. The method and apparatus described use remain connected headers, size adjustable multiplexed memory blocks and pool sessions to optimise use of system resources and make the information available.

Description

An Inter-Computer Communications Apparatus The present invention relates to an inter-computer communications apparatus and more particularly to methods and apparatus for improving the efficiency and transparency of communications between computer systems.

The term inter-computer communications refers to communication between a number of data processing entities. >b AND TULE 23 In many data processing systems, it is common to transfer data from between a number of disparate and often geographically remote sources to a local or target computer system. These sources frequently use different hardware and software platforms to comparable data sources and to the local target computer. The data may be transferred for storage, used to obtain information or for processing to provide a result, which is then re-transferred to the source. Such transfers have become extremely common with the emergence of both Internet and Intranet applications. When the hardware and software platforms of the remote sources are configured for this purpose or are identical this is relatively simple and error free. As the numbers of different hardware and software platforms increase so do the problems increase. When there are a large number of variations it becomes virtually impossible to transfer data in a transparent manner, in that the data must be converted at each source, into a format suitable for use by the local target computer. Even this is not suitable in all situations, for example when the source is not designed or configured for this type of operation having been developed over a long period of time. Such legacy systems contain large quantities of information, which may be required for the purposes outlined by the local system. Obviously owners of such source systems wish to unlock the information stored to enable users to fully exploit the new technologies. However, as the systems were not designed to operate in this i. manner this is problematic. This is particularly the case when it is necessary’ to connect df rocessing elements running on the local target to application processing elements written t n the legacy system using the open standards. o V... z .u a.

O ^| '^’{Specialist intervention by skilled operatives may be used to transfer this information to /stems where communication and data transfer is possible however, the cost of replacing the j ardware and/or software becomes prohibitive. Additionally, such a transfer would require Oo;C a-going supervision and implementation of update procedures as the legacy systems ****** I ?d ascribed are frequently in use by large numbers of users continuously updating the information stored. Not withstanding tire cost implications ull transfer of operations to such INT CL I' /folrl IE 981034 Λ a system to overcome these problems is frequently impossible due to functional differences. Staffs working on the legacy servers have developed huge experience and skill in developing applications, the move to a newer technology would require impossible levels of retraining and would render the skill and experience developed over many years redundant.

In situations where it is possible to retrieve the data from the source for use by the local target computer the processing demands placed on the source are such that the number of local systems which can access the information simultaneously is limited and delays are frequently excessive.

One solution has been proposed by International Business Machines Corporation (IBM) and marketed as an Internet Connectivity Secured Server (ICSS). While ICSS provides a server solution for transferring static text and provides the ability to run Common Gateway Interface (CGI) programs the environment in which it runs on the legacy mainframe is however alien to mainframe personnel requiring extensive retraining. Therefore, while the solution proposed is very suitable in proprietary systems where uniformity of operation can be guaranteed it is not universally applicable across information networks where hardware and software differences are normal rather than exceptional. Additionally to run certain program applications on the host ICSS requires the user to manually compile the application in a legacy batch before place it in an accessible environment and testing. This methodology is alien to legacy system staff requiring extensive retraining or recruitment of new staff either of which significantly increases costs.

There is therefore a need for a communication apparatus, which will provide communications between disparate data sources, which will overcome the aforementioned problems.

Accordingly, there is provided an apparatus for connection to a legacy host computer system of the type having a non hierarchical file structure for controlling simultaneous communication between a plurality of local processing devices and the legacy host computer system using an interface protocol, the apparatus having data retrieval means for interaction with the legacy host computer system, the data retrieval means performing the steps of:detecting a server active signal from the legacy host computer system indicating data retrieval requests may be processed; IE 981034 detecting a data retrieval request from a local processing device on a pre-set communications port of the apparatus; automatically accessing a pool session stack to determine availability status and generating a pool session ready signal in response to the existence of a pool session in the pool session stack; ί assigning the detected data retrieval request to the pool session and removing the pool session from the pool session stack; and retrieving a data stream associated with the data retrieval request and parsing the data stream to automatically extract connection details.

Preferably, the data retrieval means performs the further the steps oftprocessing the connection details to detect an existing conversation identifier in the data stream; accessing a conversation pool session register in response to a detected existing conversation identifier and generating a conversation pool session address associated with the existing conversation identifier; indexing a pool session stack using the generated conversation pool session address and retrieving a conversation pool session associated with the existing conversation identifier; transferring the detected data retrieval request to the conversation pool session and removing the conversation pool session from the conversation pool register; and automatically detecting a redundant pool session following the transfer and accessing the pool session stack to update availability status and append the redundant pool session.

Ideally, the data retrieval means performs the additional steps of the steps oftexecuting a Common Gateway Interface (CGI) associated with the data retrieval IE 981034 request; generating an output for data retrieval request in response to the executed CGI and buffering the output; accessing a predefined data segment of the buffered output to locate a conversation request identifier; accessing the conversation pool session register in response to a located conversation request identifier, generating a new conversation pool session address associated with the conversation request identifier; and re-indexing the pool session stack to incorporate the new conversation pool session address.

In one arrangement the apparatus incorporates means for identifying a requested data type from the data retrieval request, means for identifying a requested file location and translation means for identifying a sequential dataset associated with the request.

Preferably, the apparatus incorporates a converter for converting a legacy based character set.

Ideally, the apparatus incorporates means for establishing a connection between a local processing device and the legacy host computer system and in response to a successful connection signal having means for attaching the request to an apparatus defined legacy work unit, the work unit being operable by the apparatus to simultaneously and asynchronously process a plurality of data retrieval requests from the local processing, each request being assigned to a thread of the host.

Preferably, the apparatus incorporates means for reading a header associated with the data retrieval request and means for normalising and indexing a list of received headers to identify a pointer position for assignment to a given thread .

According to another aspect of the invention there is provided a method for controlling simultaneous communication between a plurality of local processing devices and a legacy host computer system of the type having a non hierarchical file structure by performing the steps of:IE 981034 detecting a server active signal from the legacy host computer system indicating data retrieval requests may be processed; detecting a data retrieval request from a local processing device on a pre-set communications port of the apparatus; » automatically accessing a pool session stack to determine availability status and generating a pool session ready signal in response to the existence of a pool session in the pool session stack; assigning the detected data retrieval request to the pool session and removing the pool session from the pool session stack; and retrieving a data stream associated with the data retrieval request and parsing the data stream to automatically extract connection details.

Preferably, the method includes the further steps of:processing the connection details to detect an existing conversation identifier in the data stream; accessing a conversation pool session register in response to a detected existing conversation identifier and generating a conversation pool session address associated with the existing conversation identifier; indexing a pool session stack using the generated conversation pool session address and retrieving a conversation pool session associated with the existing conversation identifier; transferring the detected data retrieval request to the conversation pool session and removing the conversation pool session from the conversation pool register; and automatically detecting a redundant pool session following the transfer and accessing the pool session stack to update availability status and append the redundant pool session.

IE 981034 Ideally, the method performs the additional steps of:executing a Common Gateway Interface (CGI) associated with the data retrieval 5 ' request; generating an output for data retrieval request in response to the executed CGI and ί buffering the output; accessing a predefined data segment of the buffered output to locate a conversation request identifier; accessing the conversation pool session register in response to a located conversation request identifier, generating a new conversation pool session address associated with the conversation request identifier; and re-indexing the pool session stack to incorporate the new conversation pool session address.

According to another aspect of the invention there is provided, an intercomputer communications apparatus for connection to a legacy computer system allowing communications between a number of local computers using an interface protocol.

Preferably, the apparatus provides Internet or Intranet services on the legacy or mainframe systems.

Ideally, the apparatus incorporates means for running applications and preferably said means operate without the overhead and configuration problems posed by open edition standards.

In one arrangement, the apparatus connects to local systems using a server having means for processing requests in a defined protocol, received from the local system and ideally the apparatus further incorporates means for returning a response to said requests in a suitable format.

Preferably, the apparatus has means for identifying a requested data type from the request.

IE 981034 Preferably, the apparatus further incorporates means for identifying a requested file location.

In one arrangement of the invention, the apparatus incorporates a translation mechanism for identifying a sequential dataset associated with the request.

Preferably, the apparatus provides on-line read access means for accessing data on the legacy source and means for updating local data files.

In a particularly preferred arrangement, the apparatus incorporates means for terminating the ' request from the local system to run a program to access source data and means for building an output data stream to be sent to the local system. This is particularly useful in processing requests received to run programs under standards such as HTTP, DCOM or CORBA.

Preferably, the means for terminating the request uses a Common Gateway Interface.

Preferably, the apparatus incorporates a converter for converting a legacy based character set to one readable by the local system thus problems associated with a non-heterogeneous network of machines using different character sets is overcome.

Preferably, the apparatus has means for establishing a connection between the legacy and the local system and in response to a successful connection message for attaching the request to internal unit of work within an apparatus address space of the host.

Ideally, the apparatus incorporates means for simultaneously and asynchronously processing a plurality' of requests form local systems, each request being assigned to a thread of the host.

In one preferred arrangement, the apparatus includes means for reading a header associated with a request and means for normalising and indexing a list of received headers to identify a pointer position prior to thread assignment.

Preferably, tire means for reading the header has a remain connected header identifier.

According to one aspect of the invention, there is provided means for processing multiple requests without reference to the number of task control blocks of the host using a multiplexer and segmented legacy address space. This is particularly advantageous when processing requests relating to a function or program call, as necessary' data resources, may be IE 981034 provided locally within a thread.

An inter-computer communications apparatus formed in accordance with provides a HTTP middleware broker to service local system requests using standard interfaces to provide standard Common Gateway Interface (CGI) support. Additionally it allows extensions to enable a variety of computer languages to be interpreted to use these interfaces and to support CGI requests. The apparatus allows for the communication of new technology formats between the local and legacy systems such as may be delivered via the Internet or an Intranet in a manner, which is integrated with normal legacy or mainframe security systems. To facilitate such security accounting facilities are provided as well as storage protection to protect from rogue applications.

The invention will be more clearly understood from the following description of an embodiment thereof with reference to the accompanying drawing, given by way of example only, in which: Fig. 1 is a flow diagram illustrating operation of an inter-computer communications apparatus formed in accordance with the invention.

For the purposes of this description, specific system architectures, processors, memory devices, timing and performance details have been omitted in order not to unnecessarily obscure the present invention. Thus, the constituent components of the invention have been described in terms of functionality, as many ways of achieving said functionality will be readily apparent to those skilled in the art.

An inter-computer communications apparatus according to the invention is connected to a legacy computer system to allow communications between a number of local computers using Transmission Control Protocol/Intemet Protocol. (TCP/IP). In addition to allowing the local systems to access stored information based on internet or intranet services on such legacy or mainframe systems it runs applications without the restrictions imposed by Customer Information Control System (CICS) and IMS/DC and without the overhead and configuration problems posed by open edition standards.

The apparatus connects to local systems using a server which processes Hypertext Transport Protocol (HTTP) requests received from the local system. HTTP is the protocol that is used to submit these requests from the local system to the server which in tum uses the same protocol to return the results of that request to the local system or client. HTTP is the IE 981034 foundation of the World Wide Web (WWW) where the simplest through to the most complex of browsers use HTTP to issue requests to WWW servers and to receive and to display the response to those requests.

The server delivers information using this protocol to the local system and the data may be one of a number of types depending on the request received, for example static data which might include text, graphics' pr other forms of binary data used to build images on the local system. This data is typically stored in a hierarchical file system in UNIX and is identified using a Uniform Resource Locator (URL). For data of this type the URL identifies the file which is to be transmitted to the local system. Interpretation of the information communicated to the local system is performed by that system and does not form part of the current invention.

A translation mechanism of the apparatus is used to identify a sequential dataset to which the URL relates. For example, the URL /w/x/y/z will be taken to refer to the dataset w.x.y member z. Similarly the URL /a/b/c/ will be taken to refer to the sequential dataset a.b.c. Once translated, the apparatus locates the dataset or dataset/member combination and sends the appropriate information to the local system in response to the request. The contents of the located data are identified using either a logical member type or the last level of the dataset name identified by the URL. This translation forms an important part of the current invention in that it allows the invention to operate on non-hierarchical file handling platforms.

The apparatus allows on-line read access to data of the source or legacy system and the ability to update local data files on-line terminating the request from the local system to run a program to access source data and build the output data stream to be sent to the user. Generally speaking any given WWW output will consist of static content, such as graphics, to make the result look more presentable, combined with the dynamic information provided by this program and both are transferred to the local system.

The interface for running these programs uses the Common Gateway Interface (CGI) and is not dependent on the use of the C language. CGI is a remote procedure call from an HTTP client to a program running on an HTTP server. The HTTP client, for example a WWW browser, includes information about the WWW client as well as data provided by a user entering data into an HTML page displayed on the browser. The URL used for the request identifies the request as a CGI request to the HTTP server, which in turn launches the IE 981034 ίο program, identified in the URL. The apparatus of the invention uses functional extensions to enable any mainframe or legacy language to execute as CGI program thus enabling mainframe application operatives to utilise current skills. It will however be appreciated that the invention is equally suited to executing such C language programs under such standards as the Portable Operating System Interface (POSIX), the Distributed Common Object Model (DCOM) or the Common Object Request Broker Architecture (CORBA).

The previously known method of processing such requests is now described to facilitate understanding of the current invention and to highlight the important technical advantages associated therewith.

The HTTP client attempts to connect to the machine address where an HTTP server is running based on an address provided by the user. The HTTP server is normally listening for incoming requests on a TCP/IP port.

The HTTP server normally accepts the connection at which time the client is free to send data. The HTTP server may also elect not to accept the request at which time the connection will be broken. This may occur where the HTTP server only wishes to service requests from certain TCP/IP addresses.

The HTTP client sends the HTTP request to the HTTP server encapsulating various levels of information in standard HTTP headers. The client may also send some ‘content’ to the server depending on the type of request. If request content is sent, the client will normally send ‘CONTENT-LENGTH’ and ‘CONTENT-TYPE’ HTTP headers to enable the HTTP server to interpret the content correctly.

The HTTP server receives the request over the TCP/IP connection and begins processing the request. The most important part of the request is the URL, which identifies the resource, which the client wishes to access. The string ‘cgi’ at the start of the URL will, normally identify a CGI request.

Prior to giving control to the CGI program, various environment variables are set which contain all of the information about the request. The CGI program may query these environment variables. The environment is so configured that any data written will become part of the response to the request and any attempt to access this will result in the content being returned to the user.

IE 981034 CGI programs simply provide a content type followed by the content and lets the HTTP server look after the HTTP header data.

When the CGI program terminates, the HTTP server sends the response back to the client, which will be waiting to receive the data, and closes the connection with the client.

I The client on receiving the data, process the result which, in the case of a browser, means displaying the output for the user, and closes it’s end of the TCP/IP connection. This process is repeated for each URL or resource the HTTP client wishes to access.

The corresponding portion of the current invention is now described with reference to Fig. 1. In step 1 the client initiates a connection request and this request is routed to the server in step 2. Providing the server is awaiting requests this request is detected in step 3 on a known port. A check is then conducted in step 4 to determine whether a pool session is available from a queue QA. If no session is available then a new session is produced in step 5. If however, if a session is available from the queue QA the request Is assigned to that session in step 6. The server then reads the data stream associated with the requests in step 7 and parses the stream in step 8. Connection details are extracted from the requests in step 9 and a check is conducted to determine whether the request forms part of an existing conversation in step 10. If the request forms part of an existing conversation the request is passed to an existing conversation session from a conversation session queue QC in step 11. The session to which the request was passed in step 6 is then returned to the queue QA in step 12 and the CGI is executed in step 14. If the check conducted in step 10 determined that the request does not relate to any existing conversation, the request type is determined in step 15. If the request is a CGI then this is processed as before in step 14. If the request is for static information as discussed above then this is processed in step 16 the static content is returned in step 17 and the session is returned to the queue QA in step 18.

The CGI executed in step 14 generates an output built for the response in step 20. The response is buffered in step 21 and delivered in step 22. A check is then conducted in step 23 to determine whether the response incorporates a conversation request. If no conversation request is detected the conversation session is returned to the queue QA in step 24.

However, if the check done in step 23 determines that a conversation has been requested the conversation session is returned to the queue QC in such as way that it awaits further requests in the same session in step 25.

IE 981034 Legacy systems such as those described frequently use different character sets to those used by the local systems making requests and this can pose a major problem in a heterogeneous network where machines using the different character sets must exchange data. Content type of the data to be communicated is used to correctly format an incoming or outgoing piece of data by the apparatus. This may involve the automatic translation of the data for example between ASCII and EBCDIC? in a consistent way using a user configurable table.

The mainframe legacy, source or mainframe system receives a request from the local system using the TCP/IP protocol mentioned above. A connection is established as dictated by the TCP standard and using the API sockets. Providing the apparatus on the source is awaiting requests when the request is generated from the local system a connection is established. The request is attached to an internal unit of work within the apparatus address space. To avoid the source blocking accept mode while for example running a program as a result of a local request a second utility is run concurrently by the apparatus to interact with requests from the local system.

When the request is received it is assigned to a thread which provides a dedicated, size adjustable portion of memory to allow processing of the request. This memory allocation is mapped into a working memory block containing all of the reads currently being processed by the legacy system. If a large number of requests are being simultaneously processed by the system the individual or a group of threads may be swapped out to a memory device of the mainframe. Typical requests are of the order of 80K bytes and threads can be of the order of 8k bytes to 1 Mbyte. Threads once assigned are allocated to a task control block (TCB) of the source.

When a connection is accepted by the source or legacy system an accept acknowledge signal is returned by the apparatus to the local system. The local system generates the request on receipt of this signal. The request typically contains seven pieces of header information however there may be as many as 50. These headers may provide information relating to the source of the request and information relating to the request itself. For example content type, or an authorisation indicator for use by the legacy system. These headers are arranged in a normalised list as they are received and in a structured format. The list of headers is then indexed and referred to by the apparatus to identify a pointer position before being loaded into a thread. The authorisation header if present is communicated to the legacy system to determine whether standard operating system default security should be applied or whether IE 981034 |j alternative criteria should be applied. if the received request is for static data such as that described above by parsing and identifying the URL information of the request and security considerations allow the data is returned to the local system having first had headers applied. These headers are of the standard HTTP type and may optionally include a remain connected header. This header is applied by the apparatus for-situations where it is deemed that further requests will issue from the local system within a short period of time. For example in the discussion above relating to the use of the communications apparatus for internet or intranet applications the initial request might relate to the construction of a homepage. Such homepages typically will generate five or more requests and by maintaining the connection following the initial request the overhead associated with continuous connect - reconnects significantly enhances system performance.

Historically legacy systems have assigned a block of memory for a request from a local system and dedicated a task control block to each memory block. This has significant implications for the number of requests that can be concurrently processed by the source, as TCB resources for a given system are limited. The current invention allows for multiple requests to be processed independently of the number of TCBs in any given system by multiplexing a single TCB between a number of threads, each thread in turn being shared by a number of users and their requests. This is achieved on initialisation by segmenting address spaces into blocks of on average 300K bytes. This segmentation therefore determines the maximum number of threads and therefore concurrent users providing a significant improvement.

This is particularly effective when the local request relate to a function or program call for dynamic information. By assigning the thread in this way the program whose execution was requested may have all data resources provided locally within the thread. This request type is identified from the URL as discussed above and the name of the function called is identified in the next eight bytes of the request. In this way access to information may be obtained irrespective of programming language used and returned using HTTP to the request origin.

It will of course be understood that the invention is not limited to the specific details described herein, which are given by way of example only, and that various modifications and alterations are possible within the scope of the invention.

Claims

CLAIMS:

1. An apparatus for connection to a legacy host computer system of the type having a non hierarchical file structure for controlling simultaneous communication between a plurality of local processing devices and the legacy host computer system using an interface protocol, the-apparatus having data retrieval means for interaction with the legacy host computer system, the data retrieval means performing the steps of:detecting a server active signal from the legacy host computer system indicating data retrieval requests may be processed; detecting a data retrieval request from a local processing device on a preset communications port of the apparatus; automatically accessing a pool session stack to determine availability status and generating a pool session ready signal in response to the existance of a pool session in the pool session stack; assigning the detected data retrieval request to the pool session and removing the pool session from the pool session stack; and retrieving a data stream associated with the data retrieval request and parsing the data stream to automatically extract connection details; the data retrieval means optionally performing the further steps oftprocessing the connection details to detect an existing conversation identifier in the data stream; accessing a conversation pool session register in response to a detected existing conversation identifier and generating a conversation pool session address associated IE 981034 I5 with the existing conversation identifier; indexing a pool session stack using the generated conversation pool session address and retrieving a conversation pool session associated with the existing conversation 5 identifier; i transferring the detected data retrieval request to the conversation pool session and removing the conversation pool session from the conversation pool register; and IO automatically detecting a redundant pool session following the transfer and accessing the pool session stack to update availability status and append the redundant pool session; the data retrieval means optionally further performs the additional steps of :15 executing a Common Gateway Interface (CGI) associated with the data retrieval request; generating an output for data retrieval reqeust in response to the executed CGI and 20 buffering the output; accessing a predefined data segment of the buffered output to locate a conversation request identifier; 25 accessing the conversation pool session register in response to a located conversation request identifier, generating a new conversation pool session address associated with the conversation request identifier; and re-indexing the pool session stack to incorporate the new conversation pool session 30 address. IE 981034 I6

2. An apparatus as claimed in any preceding claim incorporating means for identifying a requested data type from the data retrieval request, means for identifying a requested file location and translation means for identifying a sequential dataset associated with the request; optionally in which the apparatus incorporates a converter for converting a legacy j based character set; and optionally incorporating means for establishing a connection between a local processing device and the legacy host computer system and in response to a successful connection signal having means for attaching the request to an apparatus defined legacy work unit, the work unit being operable by the apparatus to simultaneously and asynchronously process a plurality of data retrieval requests from the local processing, each request being assigned to a thread of the host; and further optionally in which the apparatus incorporates means for reading a header associated with the data retrieval request and means for normalising and indexing a list of received headers to identify a pointer position for assignment to a given thread.

3. An apparatus substantially as hereinbefore described, with reference to and as illustrated in the accompanying drawing.

4. A method for controlling simultaneous communication between a plurality of local processing devices and a legacy host computer system of the type having a non hierarchical file structure by performing the steps of:detecting a server active signal from the legacy host computer system indicating data retrieval requests may be processed; detecting a data retrieval request from a local processing device on a preset communications port of the apparatus; IE 981034 I7 automatically accessing a pool session stack to determine availability status and generating a pool session ready signal in response to the existance of a pool session in the pool session stack; assigning the detected data retrieval request to the pool session and removing the pool ί session from the pool session stack; and retrieving a data stream associated with the data retrieval request and parsing the data io stream to automatically extract connection details; the method optionally performing the further steps of:processing the connection details to detect an existing conversation identifier in the 15 data stream; accessing a conversation pool session register in response to a detected existing conversation identifier and generating a conversation pool session address associated with the existing conversation identifier; indexing a pool session stack using the generated conversation pool session address and retrieving a conversation pool session associated with the existing conversation identifier; 25 transferring the detected data retrieval request to the conversation pool session and removing the conversation pool session from the conversation pool register; and automatically detecting a redundant pool session following the transfer and accessing the pool session stack to update availability status and append the 30 redundant pool session; and IE 981034 optionally performing the steps of:executing a Common Gateway Interface (CGI) associated with the data retrieval request; generating an output for data retrieval request in response to the executed CGI and ί buffering the output; accessing a predefined data segment of the buffered output to locate a conversation request identifier; accessing the conversation pool session register in response to a located conversation request identifier, generating a new conversation pool session address associated with the conversation request identifier; and re-indexing the pool session stack to incorporate the new conversation pool session address.

5. A method substantially as hereinbefore described, with reference to and as illustrated in the accompanying drawing.