WO2003062993A2 - A highly redundant, high-reliability and high-performance platform logging/billing generation and collection subsystem - Google Patents

A highly redundant, high-reliability and high-performance platform logging/billing generation and collection subsystem Download PDF

Info

Publication number
WO2003062993A2
WO2003062993A2 PCT/US2003/001300 US0301300W WO03062993A2 WO 2003062993 A2 WO2003062993 A2 WO 2003062993A2 US 0301300 W US0301300 W US 0301300W WO 03062993 A2 WO03062993 A2 WO 03062993A2
Authority
WO
WIPO (PCT)
Prior art keywords
clients
logging
information
criteria
client
Prior art date
Application number
PCT/US2003/001300
Other languages
French (fr)
Other versions
WO2003062993A3 (en
Inventor
Jeremy S. De Bonet
Original Assignee
Idetic, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US10/342,113 external-priority patent/US7073178B2/en
Application filed by Idetic, Inc. filed Critical Idetic, Inc.
Priority to AU2003236530A priority Critical patent/AU2003236530A1/en
Priority to EP03731937A priority patent/EP1474762A2/en
Publication of WO2003062993A2 publication Critical patent/WO2003062993A2/en
Publication of WO2003062993A3 publication Critical patent/WO2003062993A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/3476Data logging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3006Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/302Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3065Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3072Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/3495Performance evaluation by tracing or monitoring for systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising

Definitions

  • This invention generally relates to the reporting of log data from computer systems. More particularly, this invention is a method for creating a subsystem, which can be implemented in software or hardware, that enables the selective dissemination of system logs to multiple networked machines.
  • the secondary mechanism In most large systems, there exists the need for a secondary mechanism that can be used to monitorthe performance, activity and behavior of the primary function of the system. It is common in the art for the secondary mechanism to use a method called logging. Generally, the method of logging involves the generation of a series of log messages that correspond to transactions or events that occur in the system. These log messages typically use some sort of formal structure or format in order to facilitate the extraction of information by other systems, but they may be less structured in some systems.
  • Another problem is the risk of losing the logging information. Since the information is all dumped to a single location, it may be at risk due to destination device malfunctions, or even bottlenecks in getting the information to the destination device. The problem of bottlenecks is particularly relevant in the context of generating logging information for Web servers, network proxies or the like, as the bandwidth available to transmit the information over the network may be limited.
  • the invention comprises systems and methods for selectively distributing logging hformation for a system to a plurality of clients to achieve improved reliability and performance.
  • One embodiment of the invention comprises a system in which a logging subsystem monitors transactions of a transaction system, compares the transactions to criteria specified for one or more clients, and delivers logging information for transactions that meet the specified criteria to the respective clients.
  • the logging subsystem is implemented in a network proxy server.
  • the logging subsystem is configured to receive subscription information from each client, wherein the subscription information specifies the criteria for selecting logging information to transmit to the corresponding client.
  • the subscription information may also specify the level of detail of the long information to be transmitted to the client.
  • One embodiment of the invention comprises a method in which transactions within a transaction system are monitored, and logging information corresponding to the transactions is selectively distributed to one or more clients.
  • each loggable transaction is compared to the subscription criteria of each of the clients. For each client, it is determined whether the transaction meets the specified criteria or not. If the criteria are not met, no logging information for the transaction is transmitted to the client. If the criteria are met, logging information is generated for the transaction. The information is generated according to criteria that specify a level of detail desired by the client. The generated logging information is then transmitted to the client. This process is repeated for each client, then the next loggable transaction is processed in the same manner.
  • the subscription criteria for a set of clients are identical. As a result, the same logging information is transmitted to each of these clients. The transmission of this redundant information to each of the clients serves to increase their reliability of the system.
  • the subscription criteria for a set of clients define an exclusive subset of the logging information for each client. In this manner, the system performs load balancing that may improve the performance of the system.
  • the subscription criteria for the clients serve as filters for the logging information, so that logging information which is unnecessary or undesired is not transmitted to the clients. This reduces the loading on the system, reducing bandwidth requirements and improving performance.
  • the inventbn comprises a software application.
  • the software application is embodied in a computer-readable medium such as a floppy disk, CD-ROM, DVD-ROM, RAM, ROM, database schemas and the like.
  • the computer readable medium contains instructions which are configured to cause a computer such as a network proxy server to execute a method which is generally as described above.
  • the computer readable medium may comprise a RAM or other memory which forms part of a computer system. The computer system would thereby be enabled to perform a method in accordance with the present disclosure and is believed to be within the scope of the appended claims.
  • FIGURE 1 is a diagram illustrating a network-based system in accordance with one embodiment of the present invention.
  • FIGURE 2 is a diagram illustrating the components of the client and server computers in accordance with one embodiment of the invention.
  • FIGURE 3 is a diagram illustrating the basic architecture of a logging system in accordance with one embodiment of the invention.
  • FIGURE 4 is a top-level flow diagram illustrating a method in accordance with one embodiment of the invention.
  • FIGURE 5 is a flow diagram illustrating a method in accordance with one embodiment of the invention.
  • the invention comprises systems and methods for selectively distributing logging information for a system to a plurality of clients to achieve improved reliability and performance.
  • the present invention comprises a system for logging transactions in a network environment.
  • the system includes a transaction system such as a Web server or network proxy and a logging subsystem that is implemented in the transaction system.
  • the logging subsystem is configured to monitor transactions in the system and to distribute logging information (or subsets thereof) to one or more clients that are subscribed to the logging subsystem.
  • transaction system such as a Web server or network proxy
  • transaction may log events in any type of system in which loggable events occur.
  • Such systems may include virtually any software system, as well as almost any electronically monitorable mechanical system.
  • transaction system should therefore be broadly construed to include any such system, and the term “transaction should be construed to include any such loggable events.
  • the logging subsystem distributes the logging information as requested by the subscribing clients.
  • the clients define the type of information that is desired and the manner in which it is reported by the logging subsystem. For example, a client may request only certain types of messages, or it may request a subset of all of the messages. The client may request that the logging subsystem provide log messages which provide little detail about the corresponding transaction, or a large amount of detail.
  • This system can solve one or more of the problems of prior art systems through the use of appropriate subscriptions to the logging subsystem.
  • redundant storage of logging information may be provided by having two or more clients subscribe to identical sets of logging information. Then, as transactions occur in the system, corresponding log messages are transmitted to each of these clients.
  • Load balancing can be achieved by having each of a plurality of clients subscribe to an exclusive subset of the logging information. For instance, one client may subscribe to the set of all even- numbered log messages, while a second client may subscribe to the set of all odd- numbered log messages. Many other variations are possible.
  • FIGURE 1 a diagram illustrating a network-based system in accordance with one embodiment of the present invention is shown.
  • a transaction system 16 is operated in conjunction with a logging subsystem 18.
  • Transaction system 16 and logging subsystem 18 are coupled to a network 14.
  • a plurality of clients 12 are also coupled to the network.
  • Logging subsystem 18 acts as a server which provides logging information to clients 12. That particular logging information that is provided to each of the clients is determined by subscription criteria that are specified for each of the clients.
  • transaction system 16 comprises a Web server
  • it would be coupled to the Internet in order to provide service to Web clients.
  • Clients of the logging subsystem could be directly coupled to the logging subsystem, or they could be coupled to it through a separate network. Again, this is merely an example and is not intended to be uniting.
  • Network 30 may be an internal network, or an external network, such as the Internet.
  • Client computer 20 may comprise a general purpose desktop computer, a laptop computer, or any other device capable of communicating over nelwork 30 and processing (e.g., receiving and displaying) logging information received from server computer 40.
  • Server computer 40 may also be any of a number of suitable computing devices, such as a general purpose desktop computer, a computer specifically designed as a server, and the like.
  • the client computer 20 can include central processing unit (“CPU") 22, read-only memory (“ROM”) 24, random access memory (“RAM”) 26, hard disk drive (“HD”) or storage memory 28, and input/output device(s) (“I/O") 29.
  • I/O 29 can include a keyboard, monitor, printer, electronic pointing device (e.g., mouse, trackball, stylus, etc.), or the like.
  • server computer 40 can include CPU 42, ROM 44, RAM 46, HD 48, and I/O 49.
  • FIGURE 2 may have more than one CPU, ROM, RAM, HD, I/O, or other hardware components.
  • each computer is illustrated as having one of each of the hardware components.
  • FIGURE 2 is a simplification of an exemplary hardware configuration. Many other alternative hardware configurations are possible and known to persons of skill in the art.
  • Each of computers 20 and 40 is an example of a data processing system.
  • ROM 24 and 44, RAM 26 and 46 and hard disk drives 28 and 48 can be replaced with other media that can be read by CPUs 22 or 42.
  • Each of these types of memories is a medium, readable by a data processing system ("computer-readable media"), in which a software application may be embodied. These memories may be internal or external to computers 20 or 40.
  • Portions of the methods described herein may be implemented in suitable software code that may reside within ROM 24 or 44, RAM 26 or 46, hard disk drives 28 or 48 or other computer-readable media within the system.
  • the software code may also be contained on a separable data storage device, such as a removable hard disk.
  • the instructions may be stored as software code on a removable computer-readable medium such as a DASD array, magnetic tape, floppy diskette, optical storage device, or the like.
  • the software code may comprise lines of compiled C ++ , Java, or other language code.
  • the various software components may reside on a single computer or on any combination of separate computers. Other configurations may also be used.
  • any one of the computers may be performed by a different computer than illustrated in FIGURE 2.
  • a computer program or its software components with such code may be embodied in more than one data processing system readable medium in more than one computer.
  • a single system is shown for each of client computer 20 and server computer 40.
  • some or all of the software components may reside on the same computer.
  • one or more the software component(s) of the server computer 40 could reside on the client computer 20, or vice versa.
  • Communications between the computers in FIGURE 2 can be accomplished using electronic, optical, radio-frequency, or other signals.
  • client computer 20 may convert signals received from server computer 40 to a human understandable form and may convert input from a human understandable form to appropriate electronic, optical, radio-frequency, or other signals to be used by server computer 40.
  • server computer 40 may convert the signals to a human understandable form when sending a communication to the operator and may convert input from a human understandable form to appropriate electronic, optical, radio-frequency, or other signals to be used by client computer 20.
  • FIGURE 3 a diagram illustrating the basic architecture of a logging system in accordance with one embodiment of the invention is shown.
  • logging subsystem 200 is implemented within transaction system 100. As transactions occur in transaction system 100, they are monitored by logging system 200. When a loggable transaction is detected, logging subsystem 200 determines whether the transaction is within the sets of log messages requested by any of the subscribing clients (410, 510, 512, 514, 610, 612, 614). If not, no log message is generated. If so, a log message corresponding to the transaction is generated and transmitted over a network abstraction layer 300 to the requesting client.
  • Network abstraction layer 300 essentially comprises a network path between the logging system and the clients.
  • a single client 410 can subscribe to all or some subset 400 of the log messages.
  • a group of clients (510, 512, 514) can jointly subscribe to the same subsets of the logs (500, 502, 504) and deal with them separately, achieving high levels of redundancy against failure of any one of the clients' processes.
  • Another group of clients (610, 612, 614) can jointly subscribe to different subsets of the logs (600, 602, 604). Each of these clients deals with its own subset in the same way as the others. In this manner, no one client has to deal with all of the logging information by itself, and high levels of performance can be achieved, and none of the clients are overwhelmed.
  • one or more clients may subscribe to the logging subsystem to in order to obtain logging information.
  • the client subscribes to the logging subsystem.
  • the client's subscription to the logging subsystem defines the transactions for which logging information is to be distributed to the client, as well as the form of the logging information.
  • each of the transactions monitored by the logging subsystem is filtered by the client's subscription criteria. If any particular transaction meets these criteria, a log message for that transaction is generated for distribution to the client.
  • the clients subscription may define the level of detail included in the log message. For example, the log message may only indicate that a particular transaction occurred, or it may indicate additional information, such as the time of the transaction or the like. If it is no longer necessary for the client to receive the logging information, the client may unsubscribe to the logging subsystem.
  • the transaction system to which the logging subsystem is coupled comprises a network server.
  • a network server is described in detail in U.S.
  • the network server implements the logging subsystem as well as the transaction system.
  • the network server accepts connections from clients of the logging subsystem and filters the logging information according to the subscriptions of the respective clients.
  • the network server is configured to accept subscription information, including the criteria with which transactions or logging information will be filtered for distribution to client. These criteria may be specified using regular expressions, numerical comparisons, string matches, or other means for filtering the logging information.
  • the network server is also configured to maintain information on the correspondence of clients, filtering criteria and connections, so that the desired logging information can be distributed to each of the clients.
  • the network server may accepts connections from a plurality of clients. Each of these clients may specify its own subscription criteria. Based upon the respective subscription criteria, the network server may distribute to the clients identical sets of logging information, overlapping but non-identical sets of information, exclusive sets of information, or any combination thereof. The system thereby enables such features as redundant logging and load balancing through the selection of appropriate subscription criteria for each of the clients.
  • the logging clients can dynamically subscribe and unsubscribe to the logging subsystem. While the logging subsystem operates continuously, monitoring transactions and processing these transactions according to the client subscriptions, the client subscriptions themselves may change. When the logging subsystem begins operation, there may be no clients subscribed to it. After the logging subsystem is operational, two clients may be subscribed, for example, to each receive half of the logging information (thereby providing load balancing between the two clients). Later, it may be desirable to load balance between three clients, so the first two clients may be unsubscribed and then re-subscribed to receive one third of the logging information. The third client would also be subscribed to receive one third of the logging information. Alternatively, the system may make provisions for modifying the subscription criteria without necessarily having to unsubscribe.
  • FIGURE 5 a flow diagram illustrating a method in accordance with one embodiment is shown.
  • the method is implemented within the logging subsystem.
  • the method essentially comprises detecting an eventfor which logging information can be generated, then, for each of the clients, determining whether the client desires logging information for the event and, if so, generating the appropriate log message and transmitting the log message to the client.
  • the method is implemented on a event-by-event basis. That is, as each event occurs in the system, the logging subsystem detects the event and begins to determine how to distribute log information for that event. First, the logging subsystem determines whether any logging clients are subscribed. If there are no subscribed clients, then no logging information is generated or distributed. If there are clients that are subscribed to the logging subsystem, the event is processed for each of the clients, one by one.
  • the event For each of the subscribed clients, the event is first compared to the subscription criteria corresponding to the client. If, for a given client, the event matches that criteria specified for that client, a log message corresponding to the event will be generated. Thus, the logging information is filtered according to the clients' subscription criteria. The logging subsystem formats the log message according to the requirements of that particular clients subscription. The resulting log message is then transmitted to that client. This is repeated for each of the subscribed clients.
  • the methods of the present invention can be implemented, as an exemplary embodiment, in a network proxy server.
  • clients may communicate with the logging subsystem via an Internet protocol.
  • HTTP hypertext transfer protocol
  • HTTP hypertext transfer protocol
  • other protocols may be used as well.
  • the logging subsystem and its clients be separated by a network.
  • the logging subsystem and clients may each be components of the system that is being logged.
  • some of the clients may be components of the system, while other clients may be resident in systems that are independent of the system.
  • Many other variations are also possible.
  • the embodiments of the invention described above "pull" information from the logging subsystem. That is, the logging subsystem provides logging information to clients in response to requests for the information. For example, a client subscribes to the logging subsystem (i.e., requests logging information), and the logging subsystem provides information to the client responsive to the subscription (request).
  • a client's subscription to the logging subsystem may be considered a standing request to pull information from the subsystem, while in other embodiments a client may make individual requests for logging information.
  • some embodiments of the invention may be configured to push logging information to the clients.
  • one embodiment may be configured to filter the logging information and distribute it to a plurality of clients irrespective of requests by the clients.
  • the logging subsystem may be configured to load-balance among all the available clients by simply distributing information for every nth event to each of the clients (where it is assumed that there are n clients, and the subsets of logging information are offset so that they are exclusive).
  • embodiments of the present invention may enable clients to avoid being overwhelmed by too much logging information, it is possible that a given client may have difficulty keeping up with the amount of information that is distributed to it. It may therefore be desirable to implement a buffer to store the logging information until it can be accepted by the client.
  • the buffer may be implemented in the logging subsystem.
  • the buffer may queue logging information for a corresponding client said that it can be delivered after a configurable delay.
  • Such a buffer may also be used to accumulate logging information so that it can be transmitted to the client in bulk, rather than in small pieces, thereby reducing overhead.
  • without logging subsystem may be configured to provide unique identifiers for logging information that is transmitted to the clients. This may be particularly useful in the case of systems that are configured to perform load balancing.
  • unique identifiers may enable duplfcative information to be eliminated when the subsets of logging information transmitted to different clients are combined.
  • the various embodiments of the present invention may provide a number of advantages over prior art logging systems. As mentioned above, prior art systems merely identified event s and dumped all of the corresponding logging information to a single location. It was not unusual in such a system for the destination device to be overwhelmed by the huge amounts of data that were transmitted to it. Trying to capture all of this information in a single destination device was analogous to trying to take a drink from a fire hose — some portion of it was typically lost. These logging systems were therefore typically somewhat unreliable and had lower performance than was desired. The various embodiments of the present invention may provide the ability to achieve arbitrarily high levels of reliability and performance without having to change the underlying logging subsystem. This is done primarily through the ability to perform load balancing and redundant logging enough information.
  • the present logging subsystem may be configured to distribute the same logging information to multiple clients. This may be achieved by having each of these clients subscribe to the same subset of logging information. This redundant collection of information minimizes the probability that information will be lost as a result of failures within the system. For example, if one of the clients experiences a failure and loses some of the logging information, it is very likely that one of the other clients did not experience a failure and therefore captured the information that was lost by the other client. By increasing the number of clients receiving redundant information, the reliability of the overall system can be increased to an arbitrary level (e.g., 99.9999% reliability).
  • the present logging subsystem may be configured to separate the logging information into exclusive subsets, and to deliver each of the subsets to a different client.
  • the system can perform load balancing among the clients.
  • load i.e., distributing the logging information
  • the likelihood that anyone of the clients will be overwhelmed by the received information is reduced. Again, this results in a lower likelihood of failures and/or data loss, which in turn results in higher reliability.
  • the reduced load also allows each of the clients to operate at a higher level of performance.
  • the logging subsystem itself may also have improved performance, as it need not be concerned with the ability of the clients to keep up with the logging information.
  • the present invention may reduce the bandwidth requirements associated with transmissions of the logging information over a network connection by filtering the information distributed to each client. Because each client receives only information which meets the criteria specified for that client, the network connection between this client and the logging subsystem will have reduced bandwidth requirements, compared to the transmission of all of the logging information to a single destination, as performed in the prior art. Even if all of the logging information is distributed, it will typically be distributed to multiple clients, so the loading on the network connection to any one of these clients will be less than if all of the information were transmitted to a single client. It may therefore be possible to eliminate the problem of a network bottleneck, as encountered in the prior art.
  • That filtering of the logging information is useful, not only in the context of redundancy and the load balancing, but also in the context of eliminating unwanted data.
  • it is desirable to log every event e.g., billable transactions
  • a system may be configured to log certain types of events in case this information is ever needed, but it is not necessary to have this information all the time. This information may be useful for diagnostic purposes, but may be unnecessary if the system is operating properly.
  • Embodiments of the present invention may allow clients to request certain types of logging information that is useful at the time, while discarding other logging information that is not currently necessary.
  • the client may select a level of detail for the lo information, as some details may be useful at some times, but not others. This avoids the problems of collecting and storing the unnecessary information, as well as the problem of sorting through the logging information to identify the portion that is useful and the portion that is not. This may substantially increase the performance of the system.
  • a method and system can comprise a software architecture that allows different applications in the same or different communications protocols to interact with shared resources within a computer. More specifically, code for a computer program may be written to increase the amount of code that is generic to (i.e., shared by) more than one application or communications protocol and reduce the amount of code that handle application-specific or protocol-specific actions.
  • a transaction may be broken down into a set of discrete actions. The discrete actions may include functions that are common to more than one network application. These functions may be performed by the shared resources.
  • code that is specific to a particular protocol or application may be written as part of a software plug-in module with function calls to functions of the shared resources.
  • Each software plug-in module may substantially act similar to a manager for the action, where common tasks are delegated to the shared resources and the module performs specialized functions.
  • Each protocol may have its own set of software plug-in modules for the discrete actions. New applications and support for new protocols can be added by developing a new set of plug-in modules instead of writing an entirely new program. New applications for the same protocol may be developed by replacing or editing as little as one plug-in module from a different application in the same protocol.
  • a network includes an interconnected set of server and client computers over a publicly available medium (e.g., the Internet) or over an internal (company-owned) system.
  • a user at a client computer may gain access to the network using a network access provider.
  • An Internet Service Provider (“ISP") is a common type of network access provider.
  • ISP Internet Service Provider
  • a method, process, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such method, process, article, or apparatus.
  • "or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).
  • the term "software component' is intended to mean at least a portion of a computer program (i.e., a software application). An example includes a software plug-in module or the like.
  • FIG. 1 illustrates such an exemplary hardware architecture and includes client computer 120, proxy computer 140, and server computer 160.
  • Client computer 120 and proxy computer 140 are bi-directionally coupled to network 11
  • proxy computer 140 and server computer 160 are bi-directionally coupled to network 13.
  • Each of networks 11 and 13 may be an internal network or an external network (e.g., the Internet). In one embodiment, networks 11 and 13 may be the same network, such as the Internet.
  • Computers 140 and 160 may be bi-directionally coupled to databases 14 and 16, respectively.
  • Client computer 120 can include a desktop computer, a laptop computer, a personal digital assistant, a cellular phone, or nearly other device capable of communicating over network 11.
  • Other client computers may also be bi- directionally coupled to network 11.
  • the proxy computer 140 can be a server computer, but in another embodiment may be a client computer.
  • Other server computers similar to server computer 160 may be bi-directionally coupled to network 13.
  • each of proxy computer 140 and server computer 160 may be replaced by a plurality of computers (not shown) that may be interconnected to each other over a network or a combination of networks. For simplicity, a single system is shown for each of proxy computer 140 and server computer 160.
  • the client computer 120 can include central processing unit (“CPU”) 122, readonly memory (“ROM”) 124, random access memory (“RAM”) 126, hard drive (“HD”) or storage memory 128, and input output device(s) (“I/O") 129.
  • I/O 129 can include a keyboard, monitor, printer, electronic pointing device (e.g., mouse, trackball, stylus, etc.), or the like.
  • Proxy computer 140 can include CPU 142, ROM 144, RAM 146, HD 148, and I/O 149
  • server computer 160 can include CPU 162, ROM 164, RAM 166, HD 168, and I/O 169.
  • Each of the computers in FIG. 1 may have more than one CPU, ROM, RAM, HD,
  • each computer is illustrated as having one of each of the hardware components, even if more than one is used.
  • FIG. 1 is a simplification of an exemplary hardware configuration. Many other alternative hardware configurations are possible and known to skilled artisans.
  • computers 120, 140, and 160 is an example of a data processing system.
  • ROM 124, 144, and 164; RAM 126, 146, and 166; HD 128, 148, and 168; and databases 14 and 16 can include media that can be read by CPU 122, 142, or 162. Therefore, each of these types of memories includes a data processing system readable medium. These memories may be internal or external to computers 120, 140, or 160. Portions of the methods described herein may be implemented in suitable software code that may reside within ROM 124, 144, or 164, RAM 126, 146, or 166, or HD 128, 148, or 168. The instructions in an embodiment may be contained on a data storage device, such as HD 148. FIG.
  • FIG. 2 illustrates a combination of software code elements 204, 206, and 208 that are embodied within a data processing system readable medium 202, on HD 148.
  • the instructions may be stored as software code elements on a DASD array, magnetic tape, floppy diskette, optical storage device, or other appropriate data processing system readable medium or storage device.
  • the computer-executable instructions may be lines of compiled assembly, C, C ++ , Java, or other language code.
  • Other architectures may be used.
  • the functions of any one of the computers may be performed by a different computer shown in FIG. 1.
  • a computer program or its software components with such code may be embodied in more than one data processing system readable medium in more than one computer.
  • the various software components may reside on a single computer or on any combination of separate computers. In alternative embodiments, some or all of the software components may reside on the same computer. For example, one or more the software component(s) of the proxy computer 140 could reside on the client computer 120, the server computer 160, or both. In still another embodiment, the proxy computer 140 and database 14 may not be required if the functions performed by the proxy computer 140 are merged into client computer 120 or server computer 160. In such an embodiment, the client computer 120 and server computer 160 may be bi-directionally coupled to the same network (not shown in FIG. 1).
  • Communications between any of the computers in FIG. 1 can be accomplished using electronic, optical, radio-frequency, or other signals.
  • client computer 120 may convert the signals to a human understandable form when sending a communication to the user and may convert input from a human to appropriate electronic, optical, radio-frequency, or other signals to be used by, computers 140 or 160.
  • server computer 160 may convert the signals to a human understandable form when sending a communication to the operator and may convert input from a human to appropriate electronic, optical, radio-frequency, or other signals to be used by computers 120, 140, or 160.
  • the method can comprise breaking down a transaction into a set of discrete actions.
  • the actual definitions used for separating the transaction into the discrete actions is variable and may be selected by skilled artisans in manners that best suit their particular transactions, hardware requirements, and software requirements.
  • the method can also include determining which functions within the set of discrete actions are common to more than one application. As more are identified, the number of shared resources can increase and the amount of application-specific code can be decreased. Therefore, skilled artisans are encouraged to examine the software from many different levels of abstraction to discover potential shared resources that may otherwise be missed.
  • the method can further comprise generating software components for the discrete actions.
  • a set of software plug-in modules can correspond to the different discrete actions for the transaction.
  • Each application may have its own set of software plug-in modules.
  • the amount of code within each software plug-in module should be kept relatively low if the identification of shared resources was performed properly. To the extent code for any shared resources does not currently exist, code for the shared resources should be generated to maximize its ability to be used by as many different plug-in modules as possible.
  • At least two of the software plug-in modules for different applications can make function calls to any one or more of the shared resources.
  • a request manipulatbn plug-in module, a content manipulation plug-in module, or both may be the only modules changed. Therefore, creating new application for the same protocol may be simplified because other plug-in modules used for the application may be copied from another application using the same protocol. These other plug-in modules may be substantially the same between the applications.
  • each protocol may have a module that performs substantially the same action as any or all of the similar module(s) for the other protocol(s) though reducing this duplicative code by combining the common functionality is preferable.
  • the software architecture is illustrated in FIGs. 3 and 4 and is directed towards an electronic transaction that can be performed over a network.
  • a basic idea behind the architecture is to allow programming code for shared resources to be commonly used by as many different network applications as possible. Note that all of the resources may or may not be shared by all the applications.
  • the programming code for each application-specific plug-in module may include code to connect the incoming communication in any supported application to the shared resources.
  • each row of boxes 3200, 3400, and 3600 represents different applications in the same or different protocols.
  • row 3200 may represent a first application using HTTP
  • row 3400 may represent a different application using HTTP
  • row 3600 may represent yet another application in a different protocol, such as POP, SNMP, WAP, and the like.
  • the series of dots between rows 3400 and 3600 indicate that many other applications in the same or different protocols may be present.
  • the architecture may be configured to allow the addition of future applications.
  • the software architecture easily supports at least three different and potentially many more protocols.
  • each of the boxes 3202 through 3214 represents different stages (actions) that may occur during an electronic transaction.
  • box 3202 may represent a request reception plug-in module
  • box 3204 may represent an authorization plug-in module
  • box 3206 may represent a request manipulation plug-in module
  • box 3208 may represent a content retrieval plug-in module
  • box 3210 may represents a content manipulation plug-in module
  • box 3212 may represent a content delivery plug-in module
  • box 3214 may represent a post-response communication plug-in module (e.g., acknowledgement, billing, etc.).
  • Each module may correspond to one or more of the discrete actions. Details about the individual plug-in modules are described later in this specification.
  • box 3402 represents an incoming message reception plug-in module for a different application using the same protocol as box 3202
  • box 3602 represents an incoming message reception plug-in module for yet another application using a different protocol compared to box 3202.
  • New applications that make use of already-supported protocols can be developed with a minimum of effort. This is achieved by creating a new row, which makes use of protocol specific plug-ins used in another row and combines them with other plug-ins developed for the specific application at hand.
  • Some plug-in modules may be substantially the same for many different applications in the same protocol. In different protocols, the plug-in modules for at least some of the different applications may provide substantially the same functionality, although the code within those plug-in modules may be different compared to similar modules for the other protocols.
  • shared resources are illustrated as planes 3102, 3104, and 3106 that lie beneath each of the rows 3200, 3400, and 3600.
  • interfaces may be made to each of the shared resources for each plug-in module.
  • functional connectivity 4102 links module 3214 and shared resource 3102.
  • functional connectivity 4104 links module 3214 and shared resource 3104
  • functional connectivity 4106 links module 3214 shared resource 3106. Links 4102, 4104, and 4106 can be achieved by function calls to the shared resources.
  • Examples of the shared resources may include a content cache, a parameter cache, a connection pool, a domain name server cache, a clock, a counter, a database, a global variables space (e.g., a logging database), or the like.
  • a list of potential shared resources is nearly limitless. Note that not all shared resources may be connected to all modules along a row. For example, modules 3202 and 3204 may not need access to the content cache because they may not receive or process content returned for a request.
  • Each connection from a client may be handled independently on its own thread. However in other embodiments, fewer threads or a single thread can be used to operate all connections to a specific row that supports a particular application or protocol. Unless stated to the contrary, the method below is described from the perspective of proxy computer 140.
  • FIG. 5 includes a flow diagram of a method of performing an electronic transaction that corresponds to the software plug-in modules that lie along any of rows 3200, 3400, and 3600. Note that all modules are not required and that functions of some modules may be combined with others (e.g., authorization may be part of processing an initial request). The process flow diagram will be briefly covered followed by a more detailed description of each module.
  • the method can comprise receiving a request from a client computer using a request reception plug-in module (block 502) and performing authorization using an authorization plug-in module (block 504).
  • the method can also comprise manipulating a request using a request manipulation plug-in module (block 512).
  • the method can further comprise retrieving content using a content retrieval plug-in module (block 522).
  • the method can yet further comprise manipulating returned content using a content manipulatbn plug-in module (block 532) and sending the modified content to the client computer using a content delivery plug-in module (block 534).
  • the method can still further comprise processing post-response communications using a post-response plug- in module (block 542).
  • client computer 120 is sending a request for content to proxy computer 140
  • server computer 160 is providing content in response to the request.
  • the flow of information could be in the opposite direction (server computer 160 seeking information from client computer 120).
  • the method can comprise receiving a request from client computer 120 using request reception plug-in module 3202 (block 502 in FIG. 5).
  • Request reception plug-in module 3202 can be used when a request from client computer 120 is received or accessed by proxy computer 140.
  • Module 3202 can initially generate an associative array from portions of the header of the request. Part or all of the associative array may be used by the other modules along the same row.
  • the associative array may provide information that can be part of function calls to the shared resources. Any or all the data (including the associative array) may be passed from any prior plug-in module (e.g., module 3202) to any or all the subsequent plug-in modules along the same row (e.g., 3204, 3206, 3208, 3210, 3212, or 3214).
  • the method can also comprise performing authorization using authorization plug- in module 3204 (block 504).
  • the authorization plug-in module 3204 is optional and can be used for determining whether a user at client computer 120 has proper authorization.
  • the authorization modules may be based on an Internet Protocol ("IP") address or a name and a password.
  • Module 3204 may send the IP address or name and password to a shared resource to determine if the user is allowed access.
  • the method can further comprise manipulating the request using request manipulatbn plug-in module 3206 (block 512).
  • Request manipulatbn plug-in module 3206 may be used to modify, replace, or otherwise manipulate the request.
  • proxy computer 140 may have code to redirect a URL within a request to a different URL.
  • proxy computer 140 may make a function call to that shared resource using the requested URL.
  • the shared resource may pass the different URL back to module 3206.
  • Module 3206 may have the logic to put the different URL in the correct protocol, so that it will be understood by a computer that may receive the redirected request.
  • the method can yet further comprise retrieving content using content retrieval plug-in module 3208 (block 522).
  • Content retrieval plug-in module 3208 may be used to send the request and receive or access content in response to the original request or manipulated request. More specifically, a request originating from client computer 120 may have been processed by proxy computer 140 before being received by server computer 160. Content from server computer 160, in response to the processed request from proxy computer 140, would be processed using module 3208. Similarto module 3202, the code may parse the content from server computer 160 into a header portion and a content portion and append that information onto a previously generated associative array.
  • the method can still further comprise manipulating returned content from the server computer using content manipulation plug-in module 3210 (block 532).
  • Content manipulatbn plug-in module 3210 may be used to add or modify content before sending it to client computer 120. More specifically, proxy computer 140 may add advertisements or supplementary information from third parties to the content provided by server computer 160. In an alternative embodiment, part or all of the content originating from server computer 160 may be deleted or replaced with other content.
  • the method can comprise sending the modified content to the client computer using content delivery plug-in module 3212 (block 534).
  • Content delivery plug-in module 3212 may be used to route the content, after manipulation, if any, to client computer 120. Some of the information in the associative array generated when the original request from client computer 120 was processed may be used by module 3212 when sending the outgoing content to client computer 120.
  • the method can also comprise processing post-response communications using post-response plug-in module 3214 (block 542).
  • Post-response communication plug-in module 3214 may be used for acknowledgement, billing, or other purposes. For example, after content is successfully sent to client computer 120 from module 3212, module 3124 could then charge the user's credit card for that transaction. Alternatively, module 3214 may look for a signal that service to or from client computer 120 or server computer 160 is being terminated for the current transaction. Such post-response processing may be helpful in avoiding invoices or other bills sent to a user at client computer 120 if a product or service was either incomplete or defective or to properly reflect the connect time for a transaction.
  • one of the planes as illustrated in FIG. 3 may include global space variables that may need to be used by other shared resources, proxy computer 140, or the plug-in modules.
  • System statistics are examples of information that may be within a global variable space. This information may be useful to proxy computer 140 or another computer, such as client computer 120 or server computer 160, in monitoring activity. The statistics may include how many computers are connected to proxy computer 140, the amount of time each of those computers are connected to proxy computer 140, the amount of or time lapsed during transactions being processed through proxy computer 140, orthe like.
  • authorization module 3204 may be used in conjunction with a module, such as authorization module 3204. If too many users are currently logged into proxy computer 140, authorization may be denied even if the computer attempting a connection to proxy computer 140 has proper security clearance. After another transaction by another client computer is terminated, a signal from module 3214 can be sent to the logging system within the shared resources. A new client computer may now gain access to the services provided by proxy computer 140 after the connection from the other transaction is terminated.
  • FIGs. 6-8 The process flow diagram illustrated in FIGs. 6-8 is used to describe some of the specific activities. Again, unless stated to the contrary, the method is primarily described from the perspective of proxy computer 140. To aid in understanding the method in FIGs. 6-8, a specific example is used and occasionally referenced.
  • an incoming communication may be a request from client computer 120 sent to proxy computer 140 forwww.yahoo.com.
  • the client computer 120 is communicating using HTTP, using a NetscapeTM browser (of AOL Time Warner, Inc. of New York, New York), and has a MacOS XTM operating system (of Apple Computer, Inc.
  • the method can comprise receiving an incoming communication for a specific application (block 602).
  • the communication can comprise a request, a message, or other form of communication.
  • the communication can be sent by client computer 120 and received or accessed by proxy computer 140 via network 11.
  • Proxy computer 140 can access or read at least a portion of the incoming communication and determine the specific application for the communication.
  • the incoming communication is a request from client computer 120 sent to proxy computer 140 forwww.yahoo.com.
  • the incoming communication will also contain other information within the header of the request. In the example, the other information can include the browser and operating system of client computer 120.
  • proxy computer 140 can determine which row 3200, 3400, 3600, or other or row of plug-in modules will be used for the transaction. At this point in the method, proxy computer 140 may activate any or all of the plug-in modules for the row corresponding to the specific application. In one embodiment, plug-in modules within each row may be activated only as they are first used. Referring to the example, the request is for an application corresponding to row 3200. Therefore, plug-in module 3202 may be activated. If the communication is for another application, plug-in module 3402 or 3602 may be activated for the particular application.
  • the method can further comprise routing the incoming communication to a first software plug-in module for the specific application (block 604).
  • Proxy computer 140 can route the request to request reception software plug-in module 3202 because the incoming request uses the application corresponding to row 3200.
  • the method can comprise parsing the incoming communication into a header portion and a content portion (block 622).
  • the parsing can be performed by module 3202 to obtain information from the request.
  • the method can also comprise generating an associative array using information contained within the header portion (block 624).
  • the associative array can include nearly any finite number of rows. Each row can comprise a key and a value.
  • the key can comprise a parameter within the header portion, and the value can comprise a value for that parameter.
  • the header portbn may include one or more lines of a command followed by a command argument.
  • the command may be a key, and the command argument may be the corresponding value for the key.
  • the associative array may be searched by the key or the value.
  • the associative array is flexible regarding tie number of rows and allows different sizes of associative arrays to be used for different protocols.
  • one of the lines within the header may include a line with "User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.1 ) Gecko.”
  • the key will be "User- Agent,” and the value will be " Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.1 ) Gecko.”
  • a line may include "RETR 27," where 27 is an object identifier for is a particular item to be retrieved.
  • the key will be "COMMAND,” and the value will be “RETR.”
  • a second entry will be made with a key of "ARGUMENT” and a value of "27.”
  • a line may include “get 47.12.112.38,” where 47.12.112.38 corresponds to an object identifier.
  • the key will be "COMMAND”, and the value will be “GET,” and a second entry will have the key “ARGUMENT” and the value "47.12.112.38.”
  • the content may or may not be part of the associative array. If it is, the associative array can include a key of "CONTENT" and the entire content data block as the value. For an image, the content may be a very large amount of data. Alternatively, the associative array may be paired with a data pointer that points to the data block, rather than incorporating it directly into the associative array.
  • the associative array may include information as shown in Table 1 below. Descriptive names are used instead of actual names to aid in understanding the associative array. Also, the associative array may include many more rows. Because the associative array may be searched by key or value, the order of the rows is unimportant.
  • the method can also comprise generating a function call to at least one of the shared resources using data within the associative array (block 702 in FIG. 7).
  • proxy computer 140 can make a function call to a shared resource, more specifically to a clock (shared resource) and a logging system (another shared resource) to get the time and log the beginning of the transaction.
  • the logging information may include the time and a transaction identifier. Note that some of the information within the associative array could be sent with the function call to the shared resource.
  • the method can further comprise receiving data from the function call (block 704).
  • the transaction identifier may be passed back to module 3202.
  • the method can still further comprise processing data from the function call with other code within the first software module (block 706).
  • Module 3202 may be more focused on processing the incoming message rather than processing data coming back from the function call.
  • Other modules such as the content deliver plug-in module 3212, may perform such data processing. Note that the application-specific processing may occur before, during, or after function call(s), if any, are made to the shared resource(s).
  • the next software plug-in module is authorization module 3204.
  • Authorization module 3204 may use some of the information that was collected or generated by module 3202. Passing the information reduces the load on hardware by not sending a communication from proxy computer 140 to another computer (e.g., client computer 120) or making the same or similar function call to a shared resource for the same information.
  • the method can also comprise generating a function call to at least one of the shared resources using data within the associative array (block 822).
  • Authorization module 3204 may make a function call to the parameter system to determine if the user has proper authorization, whether access can be granted (whether number of users currently connected to proxy computer has exceeded its limits), priority of connection (level or speed of service to be provided), etc.
  • Module 3204 may pass user name and password when making the function call to the logging system.
  • Module 3204 may also make a function call to the shared clock to obtain a time for the action.
  • the method can also comprise receiving data from the function call (block 824).
  • the data may include information regarding whether user at client computer 120 has proper security clearance, whether the connection could be made, priority of the connection, and the like.
  • the method can further comprise processing data from the function call with other code within the current software plug-in module (block 826).
  • An example may include sending a communication from proxy computer 140 to client computer 120 informing the user whether the connection was made. Alternatively, no further processing may occur with module 3204.
  • the remaining modules along row 3200 will be addressed to complete the example transaction to give a better understanding of actions within the modules and some function calls that those modules may make. More or fewer modules may be used. Also, more, fewer, or different function calls may be made by the modules.
  • a function call can be made to a shared resource to determine if the request should be changed.
  • the function call may pass information that a request for www.yahoo.com has been received or accessed.
  • the shared resource may include logic to replace the original client request with www.***.com.
  • the associative array may be changed to replace www.yahoo.com with www.***.com or be appended to note that the manipulated request is www.***.com.
  • Module 3208 may perform the content retrieval.
  • a function call can be made to a content cache (shared resource) at proxy computer 140 to determine if the content cache includes a network page for www.***.com specifically formatted for a computer having a NetscapeTM browser and a MacOS XTM operating system. Note that the browser and operating system information can be obtained from the associative array. If the content cache has the network page, it can be passed to module 3208. Otherwise, module 3208 may formulate an HTTP request to server computer 160 requesting the network page for the specific browser and operating system of client computer 120. After proxy computer 140 obtains the proper network page from server computer 160, module 3208 may send a function call to the content cache at proxy computer 140 to cache the network page. The proper network page and other information previously collected may be sent to module 3210.
  • Content manipulation module 3210 may delete, add, or replace some or all of the content within the proper network page returned. For example, when the proper Google network page is received or accessed, module 3210 may add advertisement(s) around the border(s) of the page. A function call can be made to a shared resource to determine which advertisement(s) should be added. The logging system may keep track of which advertisement is being added, whose advertisement it is, and how many times the advertisement has been added during the current billing cycle. The logging system, which is a shared resource, may access the counter (another shared resource) by itself. In other works, some or all of the shared resources may interact with each other without requiring an application-specific software plug-in module to intervene. The manipulated content and other information may be passed to module 3212.
  • Content delivery software plug-in module 3212 may take the Google network page formatted for a NetscapeTM browser and MacOS XTM operating system and the advertisement(s) from module 3210 and prepare a communication using HTTP. The communication can be sent from proxy computer 140 to client computer 120. Function calls can be made to the logging system to note the actual content sent to client computer 120 and time sent. Any or all information collected or generated by modules 3202-3212 may be passed to module 3214.
  • Post-response communications module 3214 may be used to track usage or billing information. At the end of a transaction, module 3214 may make a function call to the clock to determine the current time, and make another function call to the logging system to determine how much time lapsed during the transaction and record any billing information. The billing information may be within a shared resource managed by an accounting department. Billing information for the user at client computer 120 may be passed from one of the shared resources to module 3214, which may return some of the information for the user at client computer 120. Proxy computer 140 may send a message to client computer 120 similar to "You were connected for 2.1 minutes and were charged $1.27. Thank you for using our service.” Alternatively, no message may be sent and the method may end. Note that not all of the activities described in the process flow diagram in FIGs. 6-
  • the power of creating new applications for the same protocol may be better understood with the flow diagram in FIG. 9 and an example.
  • different applications may be generated for different priorities of users for a network site.
  • the communication protocol may use HTTP.
  • the method can comprise developing a first set of plug-in modules for a first application (block 902). The set may correspond to row 3200 and be directed to premium users of a network site.
  • a new application may need to be developed for regular users of the network site.
  • the communication protocol may also use HTTP.
  • the method can comprise copying the first set of plug-in modules to form a second set of plug-in modules (block 922).
  • the request manipulation plug-in module, the content manipulatbn plug-in module, or both may be replaced.
  • the remainder of the plug-in modules may be unchanged and be substantially the same as the remainder of the plug- in modules for the first application.
  • the method may comprise replacing a first request manipulation plug-in module with a second request manipulation plug-in module for a second application (block 924).
  • the premium user may have access to some network pages that the regular user may not. If the regular user requests a premium page, the second request manipulatbn module may direct the regular user to another network page for which the regular user has proper access.
  • the method may also comprise replacing a first content manipulation plug-in module with a second content manipulation plug-in module for the second application (block 926).
  • the premium user may have only 10 percent of his or her window occupied by advertisements, whereas the regular user may have 50 percent of his or her window occupied by advertisements.
  • the second content manipulation module may reformat the retrieved content to allow for more advertising space.
  • the second content manipulation module may also access the shared resources to obtain the advertisements and keep track of which advertisements were used.
  • Device dependentoptimization of network pages can be achieved by plugging in a module which transcodes content using settings developed for the particular device that made the initial request.
  • the method can still further comprise executing the second application using the second set of plug-in modules (block 942).
  • those modules may be generated by editing code within the corresponding modules within the first set for the first application.
  • client computer 120 may make a request for information within a database located at server computer 160.
  • the request may be handled in a manner similar to a request for a network page. If the user does not have proper authorization to all information within a request, the request manipulation module may request only that information for which the user has property access or the content manipulatbn module may add information stating that the user does not have proper access to some or all the information.
  • the multiple-protocol software architecture and plug-in modules may be installed in client computer 120 or server computer 160. Not all modules in proxy computer 140 may be needed by client computer 120 or server computer 160.
  • Authorization modules 3204, 3404, and 3604 may not be used or can be coded to allow authorization (always authorized) at client computer 120.
  • the content manipulation modules 3210, 3410, and 3610 may not be used by the server computer 160.
  • the software components can be designed to maximize their ability use shared resources while minimizing the amount of code used for application-specific operations. Therefore, relatively smaller plug-in modules (compared to the shared resources) may be used to access the shared resources illustrated in the planes below the modules. In this manner, less code needs to be written for a new protocol compared to the prior-art method of writing or copying and modifying an entire program for a specific protocol. For applications in the same protocol, the specific coding requirements may be much less. Furthermore, protocols are more likely to be supported because the coding requirements are less, and therefore, may be generated for protocols that have relatively fewer users compared to other protocols. The method and system are significantly more efficient in both time and cost compared to existing prior-art methods dealing with the problem of many different applications in the same or different protocols.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Computing Systems (AREA)
  • Business, Economics & Management (AREA)
  • Mathematical Physics (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Computer Hardware Design (AREA)
  • Strategic Management (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • General Business, Economics & Management (AREA)
  • Computer And Data Communications (AREA)
  • Hardware Redundancy (AREA)
  • Debugging And Monitoring (AREA)

Abstract

Systems and methods for selectively distributing logging information for a system to a plurality of clients to achieve improved reliability and performance. One embodiment comprises a system in which a logging subsystem monitors transactions of a transaction system, compares the transactions to criteria specified for one or more clients, and delivers logging information for transactions that meet the specified criteria to the respective clients. In one embodiment, the logging subsystem is implemented in a network proxy server. The logging subsystem is configured to receive subscription information from each client, wherein the subscription information specifies the criteria for selecting logging information to transmit to the corresponding client. The subscription information may also specify the level of detail of the long information to be transmitted to the client.

Description

A HIGHLY REDUNDANT, HIGH-RELIABILITY AND HIGH-PERFORMANCE
PLATFORM LOGGING/BILLING GENERATION AND COLLECTION SUBSYSTEM
CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims priority to U.S. Provisional Patent Application No. 60/349,420, filed January 18, 2002 by Jeremy S. de Bonet, entitled "A Highly Redundant, High-Reliability and High-Performance Platform Logging/Biling Generation and Collection Subsystem, United States Provisional Patent Application No. 60/349,424, entitled "Network Proxy Platform that Simultaneously Supports Data Transformation, Storage, and Manipulation for Multiple Protocols" by de Bonet et al., filed on January 18, 2002, United States Provisional Patent Application No. 60/349,344 entitled "Modular Plug-In Transaction Processing Architecture" by de Bonet et al., filed January 18, 2002, which are hereby fully incorporated by reference herein. Additionally, United States Patent Application Serial
No. , entitled "Method and System of Performing Transactions Using Shared
Resources and Different Applications," by de Bonet et al., filed January 14, 2003 is incorporated by reference herein." which is incorporated by reference as if set forth herein in its entirety.
BACKGROUND OF THE INVENTION
Technical Field
This invention generally relates to the reporting of log data from computer systems. More particularly, this invention is a method for creating a subsystem, which can be implemented in software or hardware, that enables the selective dissemination of system logs to multiple networked machines.
Related Art
In most large systems, there exists the need for a secondary mechanism that can be used to monitorthe performance, activity and behavior of the primary function of the system. It is common in the art for the secondary mechanism to use a method called logging. Generally, the method of logging involves the generation of a series of log messages that correspond to transactions or events that occur in the system. These log messages typically use some sort of formal structure or format in order to facilitate the extraction of information by other systems, but they may be less structured in some systems.
High performance transaction systems generate huge volumes of logging information. Because of the volume of the logging information and the speed with which it is generated, the retrieval of the logging information can greatly tax these systems and limit their overall performance. It is important to be able to retrieve this information, however, because in many applications it is used for such purposes as generating billing information. Consequently, speed and reliability are of the utmost importance.
Because the volume of logging information and the speed with which it is generated are so high in these high performance transaction systems, it can be very difficult to handle this information. More specifically, it is generally very difficult to do anything other than simply dump the logging information to a single output device. As a result, prior art technologies do just that - dump the logging information to a single output device, such as a console display, a text file, a database, or a printer. No filtering, selection, distribution, or other processing of log messages is performed. After the logging information is dumped to the output device, it can be processed further.
This way of handling logging information, however, can be problematic. First, there is the problem of post-processing. Since the logging system simply outputs the logging information, additional systems must be employed to perform the post-processing. These systems must be coordinated with the logging system and possibly with each other in order to achieve redundancy, load-balancing, filtering or other processing of the logging information. Deploying these additional systems requires increased development time, increased hardware resources, and more maintenance personnel.
Another problem is the risk of losing the logging information. Since the information is all dumped to a single location, it may be at risk due to destination device malfunctions, or even bottlenecks in getting the information to the destination device. The problem of bottlenecks is particularly relevant in the context of generating logging information for Web servers, network proxies or the like, as the bandwidth available to transmit the information over the network may be limited. SUMMARY OF THE INVENTION
One or more of the problems outlined above may be solved by the various embodiments of the invention. Broadly speaking, the invention comprises systems and methods for selectively distributing logging hformation for a system to a plurality of clients to achieve improved reliability and performance.
One embodiment of the invention comprises a system in which a logging subsystem monitors transactions of a transaction system, compares the transactions to criteria specified for one or more clients, and delivers logging information for transactions that meet the specified criteria to the respective clients. In one embodiment, the logging subsystem is implemented in a network proxy server. The logging subsystem is configured to receive subscription information from each client, wherein the subscription information specifies the criteria for selecting logging information to transmit to the corresponding client. The subscription information may also specify the level of detail of the long information to be transmitted to the client.
One embodiment of the invention comprises a method in which transactions within a transaction system are monitored, and logging information corresponding to the transactions is selectively distributed to one or more clients. In one embodiment, each loggable transaction is compared to the subscription criteria of each of the clients. For each client, it is determined whether the transaction meets the specified criteria or not. If the criteria are not met, no logging information for the transaction is transmitted to the client. If the criteria are met, logging information is generated for the transaction. The information is generated according to criteria that specify a level of detail desired by the client. The generated logging information is then transmitted to the client. This process is repeated for each client, then the next loggable transaction is processed in the same manner.
In one embodiment, the subscription criteria for a set of clients are identical. As a result, the same logging information is transmitted to each of these clients. The transmission of this redundant information to each of the clients serves to increase their reliability of the system. In another embodiment, the subscription criteria for a set of clients define an exclusive subset of the logging information for each client. In this manner, the system performs load balancing that may improve the performance of the system. In another embodiment, the subscription criteria for the clients serve as filters for the logging information, so that logging information which is unnecessary or undesired is not transmitted to the clients. This reduces the loading on the system, reducing bandwidth requirements and improving performance.
Another embodiment of the inventbn comprises a software application. The software application is embodied in a computer-readable medium such as a floppy disk, CD-ROM, DVD-ROM, RAM, ROM, database schemas and the like. The computer readable medium contains instructions which are configured to cause a computer such as a network proxy server to execute a method which is generally as described above. It should be noted that the computer readable medium may comprise a RAM or other memory which forms part of a computer system. The computer system would thereby be enabled to perform a method in accordance with the present disclosure and is believed to be within the scope of the appended claims.
Numerous additional embodiments are also possible.
BRIEF DESCRIPTION OF THE DRAWINGS
Other objects and advantages of the invention may become apparent upon reading the following detailed description and upon reference to the accompanying drawings.
FIGURE 1 is a diagram illustrating a network-based system in accordance with one embodiment of the present invention.
FIGURE 2 is a diagram illustrating the components of the client and server computers in accordance with one embodiment of the invention.
FIGURE 3 is a diagram illustrating the basic architecture of a logging system in accordance with one embodiment of the invention.
FIGURE 4 is a top-level flow diagram illustrating a method in accordance with one embodiment of the invention.
FIGURE 5 is a flow diagram illustrating a method in accordance with one embodiment of the invention.
While the invention is subject to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and the accompanying detailed description. It should be understood, however, that the drawings and detailed description are not intended to limit the inventbn to the particular embodiment which is described. This disclosure is instead intended to cover all modifications, equivalents and alternatives falling within the scope of the present invention as defined by the appended claims.
DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT
A preferred embodiment of the invention is described below. It should be noted friat this and any other embodiments described below are exemplary and are intended to be illustrative of the invention rather than limiting.
Broadly speaking, the invention comprises systems and methods for selectively distributing logging information for a system to a plurality of clients to achieve improved reliability and performance.
In one embodiment, the present invention comprises a system for logging transactions in a network environment. The system includes a transaction system such as a Web server or network proxy and a logging subsystem that is implemented in the transaction system. The logging subsystem is configured to monitor transactions in the system and to distribute logging information (or subsets thereof) to one or more clients that are subscribed to the logging subsystem.
It should be noted that, while the present disclosure describes embodiments of the invention that log transactions in a transaction system such as a Web server or network proxy, alternative embodiments may log events in any type of system in which loggable events occur. Such systems may include virtually any software system, as well as almost any electronically monitorable mechanical system. The term "transaction system" should therefore be broadly construed to include any such system, and the term "transaction should be construed to include any such loggable events.
The logging subsystem distributes the logging information as requested by the subscribing clients. The clients define the type of information that is desired and the manner in which it is reported by the logging subsystem. For example, a client may request only certain types of messages, or it may request a subset of all of the messages. The client may request that the logging subsystem provide log messages which provide little detail about the corresponding transaction, or a large amount of detail.
This system can solve one or more of the problems of prior art systems through the use of appropriate subscriptions to the logging subsystem. For example, redundant storage of logging information may be provided by having two or more clients subscribe to identical sets of logging information. Then, as transactions occur in the system, corresponding log messages are transmitted to each of these clients. Load balancing can be achieved by having each of a plurality of clients subscribe to an exclusive subset of the logging information. For instance, one client may subscribe to the set of all even- numbered log messages, while a second client may subscribe to the set of all odd- numbered log messages. Many other variations are possible.
Referring to FIGURE 1 , a diagram illustrating a network-based system in accordance with one embodiment of the present invention is shown. As depicted in the figure, a transaction system 16 is operated in conjunction with a logging subsystem 18. Transaction system 16 and logging subsystem 18 are coupled to a network 14. A plurality of clients 12 are also coupled to the network. Logging subsystem 18 acts as a server which provides logging information to clients 12. That particular logging information that is provided to each of the clients is determined by subscription criteria that are specified for each of the clients.
It should be noted that, for the purposes of this disclosure, identical items in the figures may be indicated by identical reference numerals followed by a lowercase letter, e.g., 12a, 12b, and so on. The items may be collectively referred to herein simply by the reference numeral.
It should be noted that the network configuration illustrated in FIGURE 1 is intended to be exemplary. Other embodiments may use alternative configurations. For example, if transaction system 16 comprises a Web server, it would be coupled to the Internet in order to provide service to Web clients. Clients of the logging subsystem could be directly coupled to the logging subsystem, or they could be coupled to it through a separate network. Again, this is merely an example and is not intended to be uniting.
Referring to FIGURE 2, a diagram illustrating the components of the client and server computers in accordance with one embodiment is shown. As depicted in the figure, both client computer 20 and server computer 40 are bi-directionally coupled to network 30. Network 30 may be an internal network, or an external network, such as the Internet.
Client computer 20 may comprise a general purpose desktop computer, a laptop computer, or any other device capable of communicating over nelwork 30 and processing (e.g., receiving and displaying) logging information received from server computer 40. Server computer 40 may also be any of a number of suitable computing devices, such as a general purpose desktop computer, a computer specifically designed as a server, and the like.
The client computer 20 can include central processing unit ("CPU") 22, read-only memory ("ROM") 24, random access memory ("RAM") 26, hard disk drive ("HD") or storage memory 28, and input/output device(s) ("I/O") 29. I/O 29 can include a keyboard, monitor, printer, electronic pointing device (e.g., mouse, trackball, stylus, etc.), or the like. Similarly, server computer 40 can include CPU 42, ROM 44, RAM 46, HD 48, and I/O 49.
Each of the computers in FIGURE 2 may have more than one CPU, ROM, RAM, HD, I/O, or other hardware components. For simplicity, each computer is illustrated as having one of each of the hardware components. It should be noted that FIGURE 2 is a simplification of an exemplary hardware configuration. Many other alternative hardware configurations are possible and known to persons of skill in the art.
Each of computers 20 and 40 is an example of a data processing system. ROM 24 and 44, RAM 26 and 46 and hard disk drives 28 and 48 can be replaced with other media that can be read by CPUs 22 or 42. Each of these types of memories is a medium, readable by a data processing system ("computer-readable media"), in which a software application may be embodied. These memories may be internal or external to computers 20 or 40.
Portions of the methods described herein may be implemented in suitable software code that may reside within ROM 24 or 44, RAM 26 or 46, hard disk drives 28 or 48 or other computer-readable media within the system. The software code may also be contained on a separable data storage device, such as a removable hard disk. Alternatively, the instructions may be stored as software code on a removable computer-readable medium such as a DASD array, magnetic tape, floppy diskette, optical storage device, or the like. In one embodiment, the software code may comprise lines of compiled C++, Java, or other language code. In the hardware configuration above, the various software components may reside on a single computer or on any combination of separate computers. Other configurations may also be used. For example, the functions of any one of the computers may be performed by a different computer than illustrated in FIGURE 2. Additionally, a computer program or its software components with such code may be embodied in more than one data processing system readable medium in more than one computer. For simplicity, a single system is shown for each of client computer 20 and server computer 40. In alternative embodiments, some or all of the software components may reside on the same computer. For example, one or more the software component(s) of the server computer 40 could reside on the client computer 20, or vice versa.
Communications between the computers in FIGURE 2 can be accomplished using electronic, optical, radio-frequency, or other signals. For example, when a user is at client computer 20, client computer 20 may convert signals received from server computer 40 to a human understandable form and may convert input from a human understandable form to appropriate electronic, optical, radio-frequency, or other signals to be used by server computer 40. Similarly, when an operator is at server computer 40, server computer 40 may convert the signals to a human understandable form when sending a communication to the operator and may convert input from a human understandable form to appropriate electronic, optical, radio-frequency, or other signals to be used by client computer 20.
Referring to FIGURE 3, a diagram illustrating the basic architecture of a logging system in accordance with one embodiment of the invention is shown. As depicted in this figure, logging subsystem 200 is implemented within transaction system 100. As transactions occur in transaction system 100, they are monitored by logging system 200. When a loggable transaction is detected, logging subsystem 200 determines whether the transaction is within the sets of log messages requested by any of the subscribing clients (410, 510, 512, 514, 610, 612, 614). If not, no log message is generated. If so, a log message corresponding to the transaction is generated and transmitted over a network abstraction layer 300 to the requesting client. Network abstraction layer 300 essentially comprises a network path between the logging system and the clients.
A single client 410 can subscribe to all or some subset 400 of the log messages. A group of clients (510, 512, 514) can jointly subscribe to the same subsets of the logs (500, 502, 504) and deal with them separately, achieving high levels of redundancy against failure of any one of the clients' processes. Another group of clients (610, 612, 614) can jointly subscribe to different subsets of the logs (600, 602, 604). Each of these clients deals with its own subset in the same way as the others. In this manner, no one client has to deal with all of the logging information by itself, and high levels of performance can be achieved, and none of the clients are overwhelmed.
Referring to FIGURE 4, a flow diagram illustrating a method in accordance with one embodiment of the invention is shown. In this embodiment, one or more clients may subscribe to the logging subsystem to in order to obtain logging information. Thus, in the first step of the method, the client subscribes to the logging subsystem. The client's subscription to the logging subsystem defines the transactions for which logging information is to be distributed to the client, as well as the form of the logging information. After the client's subscription is implemented by the logging subsystem, each of the transactions monitored by the logging subsystem is filtered by the client's subscription criteria. If any particular transaction meets these criteria, a log message for that transaction is generated for distribution to the client. The clients subscription may define the level of detail included in the log message. For example, the log message may only indicate that a particular transaction occurred, or it may indicate additional information, such as the time of the transaction or the like. If it is no longer necessary for the client to receive the logging information, the client may unsubscribe to the logging subsystem.
In one embodiment, the transaction system to which the logging subsystem is coupled comprises a network server. An exemplary network server is described in detail in U.S.
Patent Application No. (Attorney Docket No. IDET1130-1), by Jeremy S. de
Bonet, Todd A. Stiers, and Phillip Alvelda VII, filed on January 14, 2003 and entitled entitled "Method And System Of Performing Transactions Using Shared Resources And Different Applications," which is which is incorporated by reference as if set forth herein in its entirety.
The network server implements the logging subsystem as well as the transaction system. The network server accepts connections from clients of the logging subsystem and filters the logging information according to the subscriptions of the respective clients. Accordingly, the network server is configured to accept subscription information, including the criteria with which transactions or logging information will be filtered for distribution to client. These criteria may be specified using regular expressions, numerical comparisons, string matches, or other means for filtering the logging information. The network server is also configured to maintain information on the correspondence of clients, filtering criteria and connections, so that the desired logging information can be distributed to each of the clients.
The network server may accepts connections from a plurality of clients. Each of these clients may specify its own subscription criteria. Based upon the respective subscription criteria, the network server may distribute to the clients identical sets of logging information, overlapping but non-identical sets of information, exclusive sets of information, or any combination thereof. The system thereby enables such features as redundant logging and load balancing through the selection of appropriate subscription criteria for each of the clients.
In one embodiment, the logging clients can dynamically subscribe and unsubscribe to the logging subsystem. While the logging subsystem operates continuously, monitoring transactions and processing these transactions according to the client subscriptions, the client subscriptions themselves may change. When the logging subsystem begins operation, there may be no clients subscribed to it. After the logging subsystem is operational, two clients may be subscribed, for example, to each receive half of the logging information (thereby providing load balancing between the two clients). Later, it may be desirable to load balance between three clients, so the first two clients may be unsubscribed and then re-subscribed to receive one third of the logging information. The third client would also be subscribed to receive one third of the logging information. Alternatively, the system may make provisions for modifying the subscription criteria without necessarily having to unsubscribe.
Referring to FIGURE 5, a flow diagram illustrating a method in accordance with one embodiment is shown. In this embodiment, the method is implemented within the logging subsystem. The method essentially comprises detecting an eventfor which logging information can be generated, then, for each of the clients, determining whether the client desires logging information for the event and, if so, generating the appropriate log message and transmitting the log message to the client.
In this embodiment, the method is implemented on a event-by-event basis. That is, as each event occurs in the system, the logging subsystem detects the event and begins to determine how to distribute log information for that event. First, the logging subsystem determines whether any logging clients are subscribed. If there are no subscribed clients, then no logging information is generated or distributed. If there are clients that are subscribed to the logging subsystem, the event is processed for each of the clients, one by one.
For each of the subscribed clients, the event is first compared to the subscription criteria corresponding to the client. If, for a given client, the event matches that criteria specified for that client, a log message corresponding to the event will be generated. Thus, the logging information is filtered according to the clients' subscription criteria. The logging subsystem formats the log message according to the requirements of that particular clients subscription. The resulting log message is then transmitted to that client. This is repeated for each of the subscribed clients.
As indicated above, the methods of the present invention can be implemented, as an exemplary embodiment, in a network proxy server. In such embodiment, clients may communicate with the logging subsystem via an Internet protocol. HTTP (hypertext transfer protocol) is preferably used, but other protocols may be used as well.
While the embodiments of the invention described herein focus primarily upon network- based implementations, it is not necessarily the case that the logging subsystem and its clients be separated by a network. In one alternative embodiment, the logging subsystem and clients may each be components of the system that is being logged. In yet another alternative, some of the clients may be components of the system, while other clients may be resident in systems that are independent of the system. Many other variations are also possible.
The embodiments of the invention described above "pull" information from the logging subsystem. That is, the logging subsystem provides logging information to clients in response to requests for the information. For example, a client subscribes to the logging subsystem (i.e., requests logging information), and the logging subsystem provides information to the client responsive to the subscription (request). In some embodiments, a client's subscription to the logging subsystem may be considered a standing request to pull information from the subsystem, while in other embodiments a client may make individual requests for logging information. While this particular feature (the pulling of the logging information) is distinct from prior art systems which "push" information from the logging subsystem to the corresponding destination, it should be noted that some embodiments of the invention may be configured to push logging information to the clients. For example, one embodiment may be configured to filter the logging information and distribute it to a plurality of clients irrespective of requests by the clients. In one such embodiment, the logging subsystem may be configured to load-balance among all the available clients by simply distributing information for every nth event to each of the clients (where it is assumed that there are n clients, and the subsets of logging information are offset so that they are exclusive).
While it is contemplated that embodiments of the present invention may enable clients to avoid being overwhelmed by too much logging information, it is possible that a given client may have difficulty keeping up with the amount of information that is distributed to it. It may therefore be desirable to implement a buffer to store the logging information until it can be accepted by the client. In one embodiment, the buffer may be implemented in the logging subsystem. The buffer may queue logging information for a corresponding client said that it can be delivered after a configurable delay. Such a buffer may also be used to accumulate logging information so that it can be transmitted to the client in bulk, rather than in small pieces, thereby reducing overhead.
In some embodiments of the invention, without logging subsystem may be configured to provide unique identifiers for logging information that is transmitted to the clients. This may be particularly useful in the case of systems that are configured to perform load balancing. The use of unique identifiers may enable duplfcative information to be eliminated when the subsets of logging information transmitted to different clients are combined.
The various embodiments of the present invention may provide a number of advantages over prior art logging systems. As mentioned above, prior art systems merely identified event s and dumped all of the corresponding logging information to a single location. It was not unusual in such a system for the destination device to be overwhelmed by the huge amounts of data that were transmitted to it. Trying to capture all of this information in a single destination device was analogous to trying to take a drink from a fire hose — some portion of it was typically lost. These logging systems were therefore typically somewhat unreliable and had lower performance than was desired. The various embodiments of the present invention may provide the ability to achieve arbitrarily high levels of reliability and performance without having to change the underlying logging subsystem. This is done primarily through the ability to perform load balancing and redundant logging enough information.
In one environment, the present logging subsystem may be configured to distribute the same logging information to multiple clients. This may be achieved by having each of these clients subscribe to the same subset of logging information. This redundant collection of information minimizes the probability that information will be lost as a result of failures within the system. For example, if one of the clients experiences a failure and loses some of the logging information, it is very likely that one of the other clients did not experience a failure and therefore captured the information that was lost by the other client. By increasing the number of clients receiving redundant information, the reliability of the overall system can be increased to an arbitrary level (e.g., 99.9999% reliability).
In one environment, the present logging subsystem may be configured to separate the logging information into exclusive subsets, and to deliver each of the subsets to a different client. In this manner, the system can perform load balancing among the clients. By balancing the load (i.e., distributing the logging information) among the clients, the likelihood that anyone of the clients will be overwhelmed by the received information is reduced. Again, this results in a lower likelihood of failures and/or data loss, which in turn results in higher reliability. The reduced load also allows each of the clients to operate at a higher level of performance. The logging subsystem itself may also have improved performance, as it need not be concerned with the ability of the clients to keep up with the logging information.
It should be noted that, in addition to increasing the reliability and performance of the system through redundancy and the load balancing, the present invention may reduce the bandwidth requirements associated with transmissions of the logging information over a network connection by filtering the information distributed to each client. Because each client receives only information which meets the criteria specified for that client, the network connection between this client and the logging subsystem will have reduced bandwidth requirements, compared to the transmission of all of the logging information to a single destination, as performed in the prior art. Even if all of the logging information is distributed, it will typically be distributed to multiple clients, so the loading on the network connection to any one of these clients will be less than if all of the information were transmitted to a single client. It may therefore be possible to eliminate the problem of a network bottleneck, as encountered in the prior art.
That filtering of the logging information is useful, not only in the context of redundancy and the load balancing, but also in the context of eliminating unwanted data. Although, in some instances, it is desirable to log every event (e.g., billable transactions), it is often the case that a portion of the logging information is not needed. For example, a system may be configured to log certain types of events in case this information is ever needed, but it is not necessary to have this information all the time. This information may be useful for diagnostic purposes, but may be unnecessary if the system is operating properly. Embodiments of the present invention may allow clients to request certain types of logging information that is useful at the time, while discarding other logging information that is not currently necessary. Similarly, the client may select a level of detail for the lo information, as some details may be useful at some times, but not others. This avoids the problems of collecting and storing the unnecessary information, as well as the problem of sorting through the logging information to identify the portion that is useful and the portion that is not. This may substantially increase the performance of the system.
The benefits and advantages which may be provided by the present invention have been described above with regard to specific embodiments. These benefits and advantages, and any elements or limitations that may cause them to occur or to become more pronounced are not to be construed as critical, required, or essential features of any or all of the claims. As used herein, the terms 'comprises,' 'comprising,' or any other variatbns thereof, are intended to be interpreted as non-exclusively including the elements or limitations which follow those terms. Accordingly, a system, method, or other embodiment that comprises a set of elements is not limited to only those elements, and may include other elements not expressly listed or inherent to the claimed embodiment.
While the present invention has been described with reference to particular embodiments, it should be understood that the embodiments are illustrative and that the scope of the invention is not limited to these embodiments. Many variations, modifications, additions and improvements to the embodiments described above are possible. It is contemplated that these variations, modifications, additions and improvements fall within the scope of the invention as detailed within the following claims.
APPENDIX NETWORK PROXY PLATFORM AND ITS USE
Reference is made in detail to exemplary embodiments of the network proxy platform and its use, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts (elements).
A method and system can comprise a software architecture that allows different applications in the same or different communications protocols to interact with shared resources within a computer. More specifically, code for a computer program may be written to increase the amount of code that is generic to (i.e., shared by) more than one application or communications protocol and reduce the amount of code that handle application-specific or protocol-specific actions. In one embodiment, a transaction may be broken down into a set of discrete actions. The discrete actions may include functions that are common to more than one network application. These functions may be performed by the shared resources.
For each action, code that is specific to a particular protocol or application may be written as part of a software plug-in module with function calls to functions of the shared resources. Each software plug-in module may substantially act similar to a manager for the action, where common tasks are delegated to the shared resources and the module performs specialized functions. Each protocol may have its own set of software plug-in modules for the discrete actions. New applications and support for new protocols can be added by developing a new set of plug-in modules instead of writing an entirely new program. New applications for the same protocol may be developed by replacing or editing as little as one plug-in module from a different application in the same protocol. The software architecture can reduce development time, increase the likelihood that new applications may be developed quickly with fewer changes from an existing application, more protocols will be properly supported, and reduce the burden on hardware and software resources. A few terms are defined or clarified to aid in understanding the descriptions that follow. A network includes an interconnected set of server and client computers over a publicly available medium (e.g., the Internet) or over an internal (company-owned) system. A user at a client computer may gain access to the network using a network access provider. An Internet Service Provider ("ISP") is a common type of network access provider. As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having" orany other variation thereof, are intended to cover a non-exclusive inclusion. For example, a method, process, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such method, process, article, or apparatus. Further, unless expressly stated to the contrary, "or" refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present). The term "software component' is intended to mean at least a portion of a computer program (i.e., a software application). An example includes a software plug-in module or the like. Different software components may reside in the same computer program or in different computer programs on the same computer or different computers. Before discussing embodiments of the network proxy platform, an exemplary hardware architecture for using network proxy platform is described. FIG. 1 illustrates such an exemplary hardware architecture and includes client computer 120, proxy computer 140, and server computer 160. Client computer 120 and proxy computer 140 are bi-directionally coupled to network 11 , and proxy computer 140 and server computer 160 are bi-directionally coupled to network 13. Each of networks 11 and 13 may be an internal network or an external network (e.g., the Internet). In one embodiment, networks 11 and 13 may be the same network, such as the Internet. Computers 140 and 160 may be bi-directionally coupled to databases 14 and 16, respectively.
Client computer 120 can include a desktop computer, a laptop computer, a personal digital assistant, a cellular phone, or nearly other device capable of communicating over network 11. Other client computers (not shown) may also be bi- directionally coupled to network 11. The proxy computer 140 can be a server computer, but in another embodiment may be a client computer. Other server computers (not shown) similar to server computer 160 may be bi-directionally coupled to network 13. In an alternative embodiment, each of proxy computer 140 and server computer 160 may be replaced by a plurality of computers (not shown) that may be interconnected to each other over a network or a combination of networks. For simplicity, a single system is shown for each of proxy computer 140 and server computer 160.
The client computer 120 can include central processing unit ("CPU") 122, readonly memory ("ROM") 124, random access memory ("RAM") 126, hard drive ("HD") or storage memory 128, and input output device(s) ("I/O") 129. I/O 129 can include a keyboard, monitor, printer, electronic pointing device (e.g., mouse, trackball, stylus, etc.), or the like. Proxy computer 140 can include CPU 142, ROM 144, RAM 146, HD 148, and I/O 149, and server computer 160 can include CPU 162, ROM 164, RAM 166, HD 168, and I/O 169. Each of the computers in FIG. 1 may have more than one CPU, ROM, RAM, HD,
I/O, or other hardware components. For simplicity, each computer is illustrated as having one of each of the hardware components, even if more than one is used. Note that FIG. 1 is a simplification of an exemplary hardware configuration. Many other alternative hardware configurations are possible and known to skilled artisans. Each of computers 120, 140, and 160 is an example of a data processing system.
ROM 124, 144, and 164; RAM 126, 146, and 166; HD 128, 148, and 168; and databases 14 and 16 can include media that can be read by CPU 122, 142, or 162. Therefore, each of these types of memories includes a data processing system readable medium. These memories may be internal or external to computers 120, 140, or 160. Portions of the methods described herein may be implemented in suitable software code that may reside within ROM 124, 144, or 164, RAM 126, 146, or 166, or HD 128, 148, or 168. The instructions in an embodiment may be contained on a data storage device, such as HD 148. FIG. 2 illustrates a combination of software code elements 204, 206, and 208 that are embodied within a data processing system readable medium 202, on HD 148. Alternatively, the instructions may be stored as software code elements on a DASD array, magnetic tape, floppy diskette, optical storage device, or other appropriate data processing system readable medium or storage device.
In an illustrative embodiment, the computer-executable instructions may be lines of compiled assembly, C, C++, Java, or other language code. Other architectures may be used. For example, the functions of any one of the computers may be performed by a different computer shown in FIG. 1. Additionally, a computer program or its software components with such code may be embodied in more than one data processing system readable medium in more than one computer.
In the hardware configuration above, the various software components may reside on a single computer or on any combination of separate computers. In alternative embodiments, some or all of the software components may reside on the same computer. For example, one or more the software component(s) of the proxy computer 140 could reside on the client computer 120, the server computer 160, or both. In still another embodiment, the proxy computer 140 and database 14 may not be required if the functions performed by the proxy computer 140 are merged into client computer 120 or server computer 160. In such an embodiment, the client computer 120 and server computer 160 may be bi-directionally coupled to the same network (not shown in FIG. 1).
Communications between any of the computers in FIG. 1 can be accomplished using electronic, optical, radio-frequency, or other signals. For example, when a user is at client computer 120, client computer 120 may convert the signals to a human understandable form when sending a communication to the user and may convert input from a human to appropriate electronic, optical, radio-frequency, or other signals to be used by, computers 140 or 160. Similarly, when an operator is at server computer 160, server computer 160 may convert the signals to a human understandable form when sending a communication to the operator and may convert input from a human to appropriate electronic, optical, radio-frequency, or other signals to be used by computers 120, 140, or 160.
Attention is now directed to the methodology of developing a software architecture for the software in accordance with one embodiment. The method can comprise breaking down a transaction into a set of discrete actions. The actual definitions used for separating the transaction into the discrete actions is variable and may be selected by skilled artisans in manners that best suit their particular transactions, hardware requirements, and software requirements. The method can also include determining which functions within the set of discrete actions are common to more than one application. As more are identified, the number of shared resources can increase and the amount of application-specific code can be decreased. Therefore, skilled artisans are encouraged to examine the software from many different levels of abstraction to discover potential shared resources that may otherwise be missed.
The method can further comprise generating software components for the discrete actions. A set of software plug-in modules can correspond to the different discrete actions for the transaction. Each application may have its own set of software plug-in modules. The amount of code within each software plug-in module should be kept relatively low if the identification of shared resources was performed properly. To the extent code for any shared resources does not currently exist, code for the shared resources should be generated to maximize its ability to be used by as many different plug-in modules as possible.
At least two of the software plug-in modules for different applications, whether they use the same or different protocols, can make function calls to any one or more of the shared resources. For different applications using the same protocol, only a request manipulatbn plug-in module, a content manipulation plug-in module, or both may be the only modules changed. Therefore, creating new application for the same protocol may be simplified because other plug-in modules used for the application may be copied from another application using the same protocol. These other plug-in modules may be substantially the same between the applications. By replacing or editing the request manipulatbn plug-in module, content manipulation plug-in module, or both, new applications may be developed very quickly.
Regarding applications in different protocols, each protocol may have a module that performs substantially the same action as any or all of the similar module(s) for the other protocol(s) though reducing this duplicative code by combining the common functionality is preferable.
Attention is now directed to the software architecture of the software in accordance with one embodiment. The software architecture is illustrated in FIGs. 3 and 4 and is directed towards an electronic transaction that can be performed over a network. A basic idea behind the architecture is to allow programming code for shared resources to be commonly used by as many different network applications as possible. Note that all of the resources may or may not be shared by all the applications. The programming code for each application-specific plug-in module may include code to connect the incoming communication in any supported application to the shared resources. By limiting the code within the plug-in modules, a user of the software architecture can reduce development time, increase the likelihood that more applications in the same or different protocols will be properly supported (especially proprietary protocols that may be used by only a limited number of computers or users), and reduce the burden on hardware and software resources for different applications because only relatively small plug-in modules may be used. In FIG. 3, each row of boxes 3200, 3400, and 3600 represents different applications in the same or different protocols. For example, row 3200 may represent a first application using HTTP, row 3400 may represent a different application using HTTP, and row 3600 may represent yet another application in a different protocol, such as POP, SNMP, WAP, and the like. Note that the series of dots between rows 3400 and 3600 indicate that many other applications in the same or different protocols may be present. Additionally, the architecture may be configured to allow the addition of future applications. The software architecture easily supports at least three different and potentially many more protocols.
Referring to row 3200, each of the boxes 3202 through 3214 represents different stages (actions) that may occur during an electronic transaction. For example, box 3202 may represent a request reception plug-in module, box 3204 may represent an authorization plug-in module, box 3206 may represent a request manipulation plug-in module, box 3208 may represent a content retrieval plug-in module, box 3210 may represents a content manipulation plug-in module, box 3212 may represent a content delivery plug-in module, and box 3214 may represent a post-response communication plug-in module (e.g., acknowledgement, billing, etc.). Each module may correspond to one or more of the discrete actions. Details about the individual plug-in modules are described later in this specification. Note that the other rows 3400 and 3600 include corresponding boxes for substantially the same types of actions except that they are designed for different applications. More specifically, box 3402 represents an incoming message reception plug-in module for a different application using the same protocol as box 3202, and box 3602 represents an incoming message reception plug-in module for yet another application using a different protocol compared to box 3202.
New applications that make use of already-supported protocols can be developed with a minimum of effort. This is achieved by creating a new row, which makes use of protocol specific plug-ins used in another row and combines them with other plug-ins developed for the specific application at hand. Some plug-in modules may be substantially the same for many different applications in the same protocol. In different protocols, the plug-in modules for at least some of the different applications may provide substantially the same functionality, although the code within those plug-in modules may be different compared to similar modules for the other protocols.
Within the software architecture, shared resources are illustrated as planes 3102, 3104, and 3106 that lie beneath each of the rows 3200, 3400, and 3600. Referring to FIG. 4, interfaces may be made to each of the shared resources for each plug-in module. Specifically referring to box 3214, functional connectivity 4102 links module 3214 and shared resource 3102. Likewise, functional connectivity 4104 links module 3214 and shared resource 3104, and functional connectivity 4106 links module 3214 shared resource 3106. Links 4102, 4104, and 4106 can be achieved by function calls to the shared resources. Examples of the shared resources may include a content cache, a parameter cache, a connection pool, a domain name server cache, a clock, a counter, a database, a global variables space (e.g., a logging database), or the like. A list of potential shared resources is nearly limitless. Note that not all shared resources may be connected to all modules along a row. For example, modules 3202 and 3204 may not need access to the content cache because they may not receive or process content returned for a request. Each connection from a client may be handled independently on its own thread. However in other embodiments, fewer threads or a single thread can be used to operate all connections to a specific row that supports a particular application or protocol. Unless stated to the contrary, the method below is described from the perspective of proxy computer 140. FIG. 5 includes a flow diagram of a method of performing an electronic transaction that corresponds to the software plug-in modules that lie along any of rows 3200, 3400, and 3600. Note that all modules are not required and that functions of some modules may be combined with others (e.g., authorization may be part of processing an initial request). The process flow diagram will be briefly covered followed by a more detailed description of each module.
The method can comprise receiving a request from a client computer using a request reception plug-in module (block 502) and performing authorization using an authorization plug-in module (block 504). The method can also comprise manipulating a request using a request manipulation plug-in module (block 512). The method can further comprise retrieving content using a content retrieval plug-in module (block 522). The method can yet further comprise manipulating returned content using a content manipulatbn plug-in module (block 532) and sending the modified content to the client computer using a content delivery plug-in module (block 534). The method can still further comprise processing post-response communications using a post-response plug- in module (block 542).
Note that not all of the activities described in the process flow diagram are required, that a limitation within a specific activity may not be required, and that further activities may be performed in addition to those illustrated. Also, some of the activities may be performed substantially simultaneously during with other activities. After reading this specification, skilled artisans will be capable of determining what activities can be used for their specific needs.
Attention is now directed to the protocol-specific plug-in modules along the rows 3200, 3400, and 3600 and how they are related to the activities illustrated in FIG. 5. Although the discussion is directed to row 3200, the corresponding modules along other rows can provide similar functionality. Also, in the example below, client computer 120 is sending a request for content to proxy computer 140, and server computer 160 is providing content in response to the request. The flow of information could be in the opposite direction (server computer 160 seeking information from client computer 120). The method can comprise receiving a request from client computer 120 using request reception plug-in module 3202 (block 502 in FIG. 5). Request reception plug-in module 3202 can be used when a request from client computer 120 is received or accessed by proxy computer 140. Module 3202 can initially generate an associative array from portions of the header of the request. Part or all of the associative array may be used by the other modules along the same row. The associative array may provide information that can be part of function calls to the shared resources. Any or all the data (including the associative array) may be passed from any prior plug-in module (e.g., module 3202) to any or all the subsequent plug-in modules along the same row (e.g., 3204, 3206, 3208, 3210, 3212, or 3214).
The method can also comprise performing authorization using authorization plug- in module 3204 (block 504). The authorization plug-in module 3204 is optional and can be used for determining whether a user at client computer 120 has proper authorization. The authorization modules may be based on an Internet Protocol ("IP") address or a name and a password. Module 3204 may send the IP address or name and password to a shared resource to determine if the user is allowed access. The method can further comprise manipulating the request using request manipulatbn plug-in module 3206 (block 512). Request manipulatbn plug-in module 3206 may be used to modify, replace, or otherwise manipulate the request. For example, proxy computer 140 may have code to redirect a URL within a request to a different URL. More specifically, proxy computer 140 may make a function call to that shared resource using the requested URL. The shared resource may pass the different URL back to module 3206. Module 3206 may have the logic to put the different URL in the correct protocol, so that it will be understood by a computer that may receive the redirected request.
The method can yet further comprise retrieving content using content retrieval plug-in module 3208 (block 522). Content retrieval plug-in module 3208 may be used to send the request and receive or access content in response to the original request or manipulated request. More specifically, a request originating from client computer 120 may have been processed by proxy computer 140 before being received by server computer 160. Content from server computer 160, in response to the processed request from proxy computer 140, would be processed using module 3208. Similarto module 3202, the code may parse the content from server computer 160 into a header portion and a content portion and append that information onto a previously generated associative array.
The method can still further comprise manipulating returned content from the server computer using content manipulation plug-in module 3210 (block 532). Content manipulatbn plug-in module 3210 may be used to add or modify content before sending it to client computer 120. More specifically, proxy computer 140 may add advertisements or supplementary information from third parties to the content provided by server computer 160. In an alternative embodiment, part or all of the content originating from server computer 160 may be deleted or replaced with other content.
The method can comprise sending the modified content to the client computer using content delivery plug-in module 3212 (block 534). Content delivery plug-in module 3212 may be used to route the content, after manipulation, if any, to client computer 120. Some of the information in the associative array generated when the original request from client computer 120 was processed may be used by module 3212 when sending the outgoing content to client computer 120.
The method can also comprise processing post-response communications using post-response plug-in module 3214 (block 542). Post-response communication plug-in module 3214 may be used for acknowledgement, billing, or other purposes. For example, after content is successfully sent to client computer 120 from module 3212, module 3124 could then charge the user's credit card for that transaction. Alternatively, module 3214 may look for a signal that service to or from client computer 120 or server computer 160 is being terminated for the current transaction. Such post-response processing may be helpful in avoiding invoices or other bills sent to a user at client computer 120 if a product or service was either incomplete or defective or to properly reflect the connect time for a transaction.
Along similar lines, one of the planes as illustrated in FIG. 3 may include global space variables that may need to be used by other shared resources, proxy computer 140, or the plug-in modules. System statistics are examples of information that may be within a global variable space. This information may be useful to proxy computer 140 or another computer, such as client computer 120 or server computer 160, in monitoring activity. The statistics may include how many computers are connected to proxy computer 140, the amount of time each of those computers are connected to proxy computer 140, the amount of or time lapsed during transactions being processed through proxy computer 140, orthe like.
These global variables may be used in conjunction with a module, such as authorization module 3204. If too many users are currently logged into proxy computer 140, authorization may be denied even if the computer attempting a connection to proxy computer 140 has proper security clearance. After another transaction by another client computer is terminated, a signal from module 3214 can be sent to the logging system within the shared resources. A new client computer may now gain access to the services provided by proxy computer 140 after the connection from the other transaction is terminated.
Attention is now directed to more specific activities that may be performed by a specific module, and how that specific module may interact with other modules for the same transaction using a specific application. The process flow diagram illustrated in FIGs. 6-8 is used to describe some of the specific activities. Again, unless stated to the contrary, the method is primarily described from the perspective of proxy computer 140. To aid in understanding the method in FIGs. 6-8, a specific example is used and occasionally referenced. In the example, an incoming communication may be a request from client computer 120 sent to proxy computer 140 forwww.yahoo.com. The client computer 120 is communicating using HTTP, using a Netscape™ browser (of AOL Time Warner, Inc. of New York, New York), and has a MacOS X™ operating system (of Apple Computer, Inc. of Cupertino, California). Referring to FIG. 6, the method can comprise receiving an incoming communication for a specific application (block 602). The communication can comprise a request, a message, or other form of communication. The communication can be sent by client computer 120 and received or accessed by proxy computer 140 via network 11. Proxy computer 140 can access or read at least a portion of the incoming communication and determine the specific application for the communication. In the example, the incoming communication is a request from client computer 120 sent to proxy computer 140 forwww.yahoo.com. The incoming communication will also contain other information within the header of the request. In the example, the other information can include the browser and operating system of client computer 120. After determining the application for the communication, proxy computer 140 can determine which row 3200, 3400, 3600, or other or row of plug-in modules will be used for the transaction. At this point in the method, proxy computer 140 may activate any or all of the plug-in modules for the row corresponding to the specific application. In one embodiment, plug-in modules within each row may be activated only as they are first used. Referring to the example, the request is for an application corresponding to row 3200. Therefore, plug-in module 3202 may be activated. If the communication is for another application, plug-in module 3402 or 3602 may be activated for the particular application.
The method can further comprise routing the incoming communication to a first software plug-in module for the specific application (block 604). Proxy computer 140 can route the request to request reception software plug-in module 3202 because the incoming request uses the application corresponding to row 3200.
The method can comprise parsing the incoming communication into a header portion and a content portion (block 622). The parsing can be performed by module 3202 to obtain information from the request.
The method can also comprise generating an associative array using information contained within the header portion (block 624). The associative array can include nearly any finite number of rows. Each row can comprise a key and a value. The key can comprise a parameter within the header portion, and the value can comprise a value for that parameter. In general, the header portbn may include one or more lines of a command followed by a command argument. The command may be a key, and the command argument may be the corresponding value for the key. The associative array may be searched by the key or the value.
By knowing conventions used by each of the protocols for incoming communications and the characteristics of headers used for those protocols, formation of the associative array can be performed without complicated coding requirements. The associative array is flexible regarding tie number of rows and allows different sizes of associative arrays to be used for different protocols.
For HTTP, one of the lines within the header may include a line with "User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.1 ) Gecko." The key will be "User- Agent," and the value will be " Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:1.1 ) Gecko." For POP, a line may include "RETR 27," where 27 is an object identifier for is a particular item to be retrieved. The key will be "COMMAND," and the value will be "RETR." A second entry will be made with a key of "ARGUMENT" and a value of "27." For SNMP, a line may include "get 47.12.112.38," where 47.12.112.38 corresponds to an object identifier. The key will be "COMMAND", and the value will be "GET," and a second entry will have the key "ARGUMENT" and the value "47.12.112.38."
The content may or may not be part of the associative array. If it is, the associative array can include a key of "CONTENT" and the entire content data block as the value. For an image, the content may be a very large amount of data. Alternatively, the associative array may be paired with a data pointer that points to the data block, rather than incorporating it directly into the associative array.
Turning to the example, the associative array may include information as shown in Table 1 below. Descriptive names are used instead of actual names to aid in understanding the associative array. Also, the associative array may include many more rows. Because the associative array may be searched by key or value, the order of the rows is unimportant.
Table 1. Exemplary associative array
Figure imgf000028_0001
The method can also comprise generating a function call to at least one of the shared resources using data within the associative array (block 702 in FIG. 7). In the example, proxy computer 140 can make a function call to a shared resource, more specifically to a clock (shared resource) and a logging system (another shared resource) to get the time and log the beginning of the transaction. The logging information may include the time and a transaction identifier. Note that some of the information within the associative array could be sent with the function call to the shared resource.
The method can further comprise receiving data from the function call (block 704). In the example, the transaction identifier may be passed back to module 3202. The method can still further comprise processing data from the function call with other code within the first software module (block 706). Module 3202 may be more focused on processing the incoming message rather than processing data coming back from the function call. Other modules, such as the content deliver plug-in module 3212, may perform such data processing. Note that the application-specific processing may occur before, during, or after function call(s), if any, are made to the shared resource(s).
A determination may be made whether the first software plug-in module is the last software plug-in module (diamond 722). If so, the method may end. Otherwise, the method may continue with passing any or all of the data (including the associative array) from a prior software plug-in module to the next software plug-in module (block 802 in FIG. 8). In the example, the next software plug-in module is authorization module 3204. Authorization module 3204 may use some of the information that was collected or generated by module 3202. Passing the information reduces the load on hardware by not sending a communication from proxy computer 140 to another computer (e.g., client computer 120) or making the same or similar function call to a shared resource for the same information.
The method can also comprise generating a function call to at least one of the shared resources using data within the associative array (block 822). Authorization module 3204 may make a function call to the parameter system to determine if the user has proper authorization, whether access can be granted (whether number of users currently connected to proxy computer has exceeded its limits), priority of connection (level or speed of service to be provided), etc. Module 3204 may pass user name and password when making the function call to the logging system. Module 3204 may also make a function call to the shared clock to obtain a time for the action.
The method can also comprise receiving data from the function call (block 824). The data may include information regarding whether user at client computer 120 has proper security clearance, whether the connection could be made, priority of the connection, and the like. The method can further comprise processing data from the function call with other code within the current software plug-in module (block 826). An example may include sending a communication from proxy computer 140 to client computer 120 informing the user whether the connection was made. Alternatively, no further processing may occur with module 3204.
A determination may be made whether the current software plug-in module is the last software plug-in module (diamond 842). If so, the method may end. Otherwise, the method may continue with block 802 in FIG. 8 and proceed in an iterative manner until the last software plug-in module is reached.
The remaining modules along row 3200 will be addressed to complete the example transaction to give a better understanding of actions within the modules and some function calls that those modules may make. More or fewer modules may be used. Also, more, fewer, or different function calls may be made by the modules.
Data can be passed to request manipulation software plug-in module 3206. A function call can be made to a shared resource to determine if the request should be changed. The function call may pass information that a request for www.yahoo.com has been received or accessed. The shared resource may include logic to replace the original client request with www.***.com. The associative array may be changed to replace www.yahoo.com with www.***.com or be appended to note that the manipulated request is www.***.com.
Module 3208 may perform the content retrieval. A function call can be made to a content cache (shared resource) at proxy computer 140 to determine if the content cache includes a network page for www.***.com specifically formatted for a computer having a Netscape™ browser and a MacOS X™ operating system. Note that the browser and operating system information can be obtained from the associative array. If the content cache has the network page, it can be passed to module 3208. Otherwise, module 3208 may formulate an HTTP request to server computer 160 requesting the network page for the specific browser and operating system of client computer 120. After proxy computer 140 obtains the proper network page from server computer 160, module 3208 may send a function call to the content cache at proxy computer 140 to cache the network page. The proper network page and other information previously collected may be sent to module 3210.
Content manipulation module 3210 may delete, add, or replace some or all of the content within the proper network page returned. For example, when the proper Google network page is received or accessed, module 3210 may add advertisement(s) around the border(s) of the page. A function call can be made to a shared resource to determine which advertisement(s) should be added. The logging system may keep track of which advertisement is being added, whose advertisement it is, and how many times the advertisement has been added during the current billing cycle. The logging system, which is a shared resource, may access the counter (another shared resource) by itself. In other works, some or all of the shared resources may interact with each other without requiring an application-specific software plug-in module to intervene. The manipulated content and other information may be passed to module 3212.
Content delivery software plug-in module 3212 may take the Google network page formatted for a Netscape™ browser and MacOS X™ operating system and the advertisement(s) from module 3210 and prepare a communication using HTTP. The communication can be sent from proxy computer 140 to client computer 120. Function calls can be made to the logging system to note the actual content sent to client computer 120 and time sent. Any or all information collected or generated by modules 3202-3212 may be passed to module 3214.
Post-response communications module 3214 may be used to track usage or billing information. At the end of a transaction, module 3214 may make a function call to the clock to determine the current time, and make another function call to the logging system to determine how much time lapsed during the transaction and record any billing information. The billing information may be within a shared resource managed by an accounting department. Billing information for the user at client computer 120 may be passed from one of the shared resources to module 3214, which may return some of the information for the user at client computer 120. Proxy computer 140 may send a message to client computer 120 similar to "You were connected for 2.1 minutes and were charged $1.27. Thank you for using our service." Alternatively, no message may be sent and the method may end. Note that not all of the activities described in the process flow diagram in FIGs. 6-
8 are required, that a limitation within a specific activity may not be required, and that further activities may be performed in addition to those illustrated. Also, some of the activities may be performed substantially simultaneously during with other activities. After reading this specification, skilled artisans will be capable of determining what activities can be used for their specific needs.
The power of creating new applications for the same protocol may be better understood with the flow diagram in FIG. 9 and an example. In one embodiment, different applications may be generated for different priorities of users for a network site. The communication protocol may use HTTP. The method can comprise developing a first set of plug-in modules for a first application (block 902). The set may correspond to row 3200 and be directed to premium users of a network site.
A new application may need to be developed for regular users of the network site. The communication protocol may also use HTTP. The method can comprise copying the first set of plug-in modules to form a second set of plug-in modules (block 922). For the new application, only the request manipulation plug-in module, the content manipulatbn plug-in module, or both may be replaced. The remainder of the plug-in modules may be unchanged and be substantially the same as the remainder of the plug- in modules for the first application.
The method may comprise replacing a first request manipulation plug-in module with a second request manipulation plug-in module for a second application (block 924). For example, the premium user may have access to some network pages that the regular user may not. If the regular user requests a premium page, the second request manipulatbn module may direct the regular user to another network page for which the regular user has proper access. The method may also comprise replacing a first content manipulation plug-in module with a second content manipulation plug-in module for the second application (block 926). The premium user may have only 10 percent of his or her window occupied by advertisements, whereas the regular user may have 50 percent of his or her window occupied by advertisements. The second content manipulation module may reformat the retrieved content to allow for more advertising space. The second content manipulation module may also access the shared resources to obtain the advertisements and keep track of which advertisements were used. Device dependentoptimization of network pages (desktop computer vs. cellular phone, etc.) can be achieved by plugging in a module which transcodes content using settings developed for the particular device that made the initial request.
After one or both of the request manipulatbn and content manipulation modules are replaced, the method can still further comprise executing the second application using the second set of plug-in modules (block 942).
Note that while the example focused more on replacing specific modules, in other embodiments, those modules may be generated by editing code within the corresponding modules within the first set for the first application.
After reading this specification, skilled artisans will appreciate that entirely different applications, using the same nel ork protocol, can be developed by simply inserting new plug-in module(s) at the request manipulatbn location, the content request location, or both locations.
In other embodiments, the method and system may be used for nearly any other network communications. As an example, client computer 120 may make a request for information within a database located at server computer 160. The request may be handled in a manner similar to a request for a network page. If the user does not have proper authorization to all information within a request, the request manipulation module may request only that information for which the user has property access or the content manipulatbn module may add information stating that the user does not have proper access to some or all the information.
In another embodiment, the multiple-protocol software architecture and plug-in modules may be installed in client computer 120 or server computer 160. Not all modules in proxy computer 140 may be needed by client computer 120 or server computer 160. Authorization modules 3204, 3404, and 3604 may not be used or can be coded to allow authorization (always authorized) at client computer 120. The content manipulation modules 3210, 3410, and 3610 may not be used by the server computer 160. After reading this specification, skilled artisans are capable of determine which modules are needed and which ones can be eliminated or bypassed (module exists but passes information through without performing any other significant activity).
The software components can be designed to maximize their ability use shared resources while minimizing the amount of code used for application-specific operations. Therefore, relatively smaller plug-in modules (compared to the shared resources) may be used to access the shared resources illustrated in the planes below the modules. In this manner, less code needs to be written for a new protocol compared to the prior-art method of writing or copying and modifying an entire program for a specific protocol. For applications in the same protocol, the specific coding requirements may be much less. Furthermore, protocols are more likely to be supported because the coding requirements are less, and therefore, may be generated for protocols that have relatively fewer users compared to other protocols. The method and system are significantly more efficient in both time and cost compared to existing prior-art methods dealing with the problem of many different applications in the same or different protocols. Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature or element of any or all the claims.

Claims

WHAT IS CLAIMED IS:
1. A system comprising: a logging subsystem; wherein the logging subsystem is configured to monitor event s; wherein the logging subsystem is configured to filter the event s according to criteria corresponding to one or more clients; and wherein the logging subsystem is configured to forward logging information for the filtered events to the one or more clients according to the criteria.
2. The system of claim 1 , further comprising: a network proxy server, wherein the logging subsystem comprises a component of the network proxy server and is configured to monitor event s of the network proxy server; a network coupled to the network proxy server; and the one or more clients, wherein the clients are coupled to the network, wherein each of the clients has an associated set of the criteria and wherein for each of the clients the logging ss is configured to filter the event s according to the set of the criteria associated with client and to forward the resulting logging information to the client.
3. The system of claim 1 , wherein the logging ss is configured to provide logging information to clients in response to corresponding requests for logging information.
4. The system of claim 1 , wherein the logging ss is configured to push logging information to clients.
5. The system of claim 1 , further comprising a network proxy server, wherein the logging system is coupled to the network proxy server to monitor event s in the network proxy server.
6. The system of claim 5, wherein the logging ss comprises a component of the network proxy server.
7. The system of claim 1 , wherein the logging subsystem is configured to forward logging information to each client at a level of detail specified by the client.
8. The system of claim 1 , wherein the logging subsystem is configured to filter logging information for each client using criteria specified in the by the client.
9. The system of claim 8, wherein two or more of the clients specify identical criteria for filtering the logging information and the logging ss delivers redundant subsets of logging information to the two clients based on the identical criteria.
10. The system of claim 8, wherein two or more of the clients specify criteria for filtering the logging information wherein the criteria define exclusive subsets of the logging information and the logging ss delivers load-balancing subsets of logging information to the two clients based on the criteria.
11. The system of claim 8, wherein at least one of the clients specifies criteria for filtering the logging information wherein the criteria specify a subset of available types of event s.
12. The system of claim 8, wherein at least one of the clients specifies criteria for filtering the logging information wherein the criteria specify every nth event.
13. The system of claim 1 , wherein the logging server is configured to dynamically implement subscription requests and unsubscribe requests from clients.
14. The system of claim 1 , further comprising the clients and a network coupled between the logging subsystem and the clients, wherein the logging subsystem is configured to receive subscription requests from the clients via network connections and to forward logging information to the one or more clients via network connections.
15. A method for logging event s comprising: for each of one or more clients, providing a corresponding set of criteria; and for each of a plurality of event s, detecting the event and for each of the one or more clients, determining whether the event meets the set of criteria corresponding to the client, and if the event meets the set of criteria, forwarding log information for the event to the client.
16. The method of claim 15, wherein the method is implemented in a network proxy server coupled to the one or more clients via a network and wherein forwarding log information comprises transmitting the log information to the corresponding client via the network.
17. The method of claim 15, wherein forwarding log information comprises pushing the log information to the corresponding client.
18. The method of claim 15, wherein forwarding log information comprises transmitting the log information to the corresponding client in response to a request for the log information.
19. The method of claim 15, wherein the set of criteria corresponding to a first client identifies a level of detail of the log information to be forwarded to the first client.
20. The method of claim 15, wherein the one or more clients comprise a first plurality of clients, wherein the sets of criteria corresponding to the first plurality of clients are identical, and wherein the log information transmitted to each of the first plurality of clients is redundant
21. The method of claim 15, wherein the one or more clients comprise a first plurality of clients, and wherein the sets of criteria corresponding to the first plurality of clients define exclusive subsets of the logging information, and wherein the log information transmitted to each of the first plurality of clients is a load-balancing subset of the log information.
22. The method of claim 15, wherein the set of criteria corresponding to a first client identifies a selected type of event s.
23. The method of claim 15, wherein the set of criteria corresponding to a first client identifies every nth event.
24. The method of claim 15, wherein providing the sets of criteria corresponding to the one or more clients comprises accepting subscription requests from the clients, wherein the subscription requests contain the sets of criteria corresponding to the clients.
25. The method of claim 24, further comprising dynamically subscribing or unsubscribing one or more clients to the logging ss.
26. The method of claim 15, wherein the set of criteria corresponding to a first client identifies all of the events.
27. The method of claim 15, wherein the set of criteria corresponding to a first client identifies a subset of the events comprising less than all of the events.
28. A software product comprising a plurality of instructions embodied in a medium readable by a data processor, wherein the instructions are configured to cause the data processor to perform the method comprising: for each of one or more clients, maintaining a corresponding set of criteria; and for each of a plurality of event s, detecting the event and for each of the one or more clients, determining whether the event meets the set of criteria corresponding to the client, and if the event meets the set of criteria, forwarding log information for the event to the client.
29. The software product of claim 28, wherein the instructions are configured to cause the data processor to monitor event s in a network proxy server, to receive the sets of criteria from the clients via a network connection, and to transmit the log information to the corresponding clients via the network connection.
PCT/US2003/001300 2002-01-18 2003-01-16 A highly redundant, high-reliability and high-performance platform logging/billing generation and collection subsystem WO2003062993A2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
AU2003236530A AU2003236530A1 (en) 2002-01-18 2003-01-16 A highly redundant, high-reliability and high-performance platform logging/billing generation and collection subsystem
EP03731937A EP1474762A2 (en) 2002-01-18 2003-01-16 A highly redundant, high-reliability and high-performance platform logging/billing generation and collection subsystem

Applications Claiming Priority (8)

Application Number Priority Date Filing Date Title
US34934402P 2002-01-18 2002-01-18
US34942002P 2002-01-18 2002-01-18
US34942402P 2002-01-18 2002-01-18
US60/349,424 2002-01-18
US60/349,344 2002-01-18
US60/349,420 2002-01-18
US10/342,113 2003-01-14
US10/342,113 US7073178B2 (en) 2002-01-18 2003-01-14 Method and system of performing transactions using shared resources and different applications

Publications (2)

Publication Number Publication Date
WO2003062993A2 true WO2003062993A2 (en) 2003-07-31
WO2003062993A3 WO2003062993A3 (en) 2004-03-04

Family

ID=27617815

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2003/001300 WO2003062993A2 (en) 2002-01-18 2003-01-16 A highly redundant, high-reliability and high-performance platform logging/billing generation and collection subsystem

Country Status (4)

Country Link
EP (1) EP1474762A2 (en)
CN (1) CN1639714A (en)
AU (1) AU2003236530A1 (en)
WO (1) WO2003062993A2 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100353709C (en) * 2004-01-16 2007-12-05 英业达股份有限公司 Platform event filtering system and method
EP2611109A1 (en) * 2011-12-29 2013-07-03 Amadeus System for high reliability and high performance application message delivery
CN110119411A (en) * 2019-05-14 2019-08-13 张良 A kind of technique transfers platform intelligent search method and system
WO2022250890A1 (en) * 2021-05-28 2022-12-01 Microsoft Technology Licensing, Llc Transaction log validation in a database transaction log service
US11720550B2 (en) 2021-05-28 2023-08-08 Microsoft Technology Licensing, Llc Transaction log validation in a database transaction log service

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106060063B (en) * 2016-06-24 2019-04-23 武汉斗鱼网络科技有限公司 A kind of filter method and device for internet site front end logic entrance

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
MALOWIDZKI M: "Advanced Event Filtering Approach For CORBA-Based Management Systems" IEEE SYMPOSIUM RECORD ON NETWORK OPERATIONS AND MANAGEMENT SYMPOSIUM 2000 IEEE, 10 April 2000 (2000-04-10), pages 45-58, XP010376674 Piscataway, NJ, USA *
OBJECT MANAGEMENT GROUP: "Event Service Specification" INTERNET CITATION, [Online] 9 February 1997 (1997-02-09), XP002148647 Retrieved from the Internet: <URL:ftp://anonymous:anonymousftp.omg.org/ pub/docs/formal/97-02-09.ps> [retrieved on 2000-09-21] *
OBJECT MANAGEMENT GROUP: "Notification Service Joint Revised Submission (OMG TC Document telecom/98-11-01)" INTERNET CITATION, [Online] 3 November 1998 (1998-11-03), XP002265995 Retrieved from the Internet: <URL:www.omg.org/cgi-bin/doc?telecom/98-11 -01.pdf> [retrieved on 2003-12-23] *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100353709C (en) * 2004-01-16 2007-12-05 英业达股份有限公司 Platform event filtering system and method
EP2611109A1 (en) * 2011-12-29 2013-07-03 Amadeus System for high reliability and high performance application message delivery
WO2013098316A1 (en) * 2011-12-29 2013-07-04 Amadeus A system for high reliability and high performance application message delivery
JP2015513708A (en) * 2011-12-29 2015-05-14 アマデウス エス.エイ.エス A system for delivering highly reliable and high performance application messages
AU2012327228B2 (en) * 2011-12-29 2016-02-18 Amadeus S.A.S. A system for high reliability and high performance application message delivery
CN110119411A (en) * 2019-05-14 2019-08-13 张良 A kind of technique transfers platform intelligent search method and system
WO2022250890A1 (en) * 2021-05-28 2022-12-01 Microsoft Technology Licensing, Llc Transaction log validation in a database transaction log service
US11720550B2 (en) 2021-05-28 2023-08-08 Microsoft Technology Licensing, Llc Transaction log validation in a database transaction log service

Also Published As

Publication number Publication date
WO2003062993A3 (en) 2004-03-04
EP1474762A2 (en) 2004-11-10
AU2003236530A1 (en) 2003-09-02
CN1639714A (en) 2005-07-13

Similar Documents

Publication Publication Date Title
US7073178B2 (en) Method and system of performing transactions using shared resources and different applications
US7200665B2 (en) Allowing requests of a session to be serviced by different servers in a multi-server data service system
US9124592B2 (en) Method and system for application level load balancing in a publish/subscribe message architecture
US8671212B2 (en) Method and system for processing raw financial data streams to produce and distribute structured and validated product offering objects
US8639848B2 (en) Data communication efficiency
US20020120697A1 (en) Multi-channel messaging system and method
US20030189946A1 (en) Method and apparatus for implementing persistent and reliable message delivery
KR100985237B1 (en) Packet routing via payload inspection for alert services, for digital content delivery and for quality of service management and caching with selective multicasting in a publish-subscribe network
US7206848B1 (en) Intelligently classifying and handling user requests in a data service system
KR100971506B1 (en) Method and apparatus for reliable and efficient content-based routing and query and response in a publish-subscribe network
US20110238814A1 (en) Method for creating global distributed namespace
JP2005539298A (en) Method and system for remotely and dynamically configuring a server
KR20100076953A (en) Aggregating and delivering information
CN1761194A (en) System and method for subscription propagation in a content-based publish/subscribe system
US9762405B2 (en) Hierarchical publish/subscribe system
US20130024527A1 (en) Hierarchical publish/subscribe system
US20130024529A1 (en) Hierarchical publish/subscribe system
JP4958951B2 (en) Content collection
WO2003062993A2 (en) A highly redundant, high-reliability and high-performance platform logging/billing generation and collection subsystem
US20030167326A1 (en) Highly redundant, high-reliability and high-performance platform logging/billing generation and collection subsystem
EP1323087A1 (en) System for processing raw financial data to produce validated product offering information to subscribers
US20030177284A1 (en) Plug-in API for protocol and payload transformation
US7206855B1 (en) System and method for exchanging information across a computer network at variable transmission rates
JP4305364B2 (en) Web service request relay system, Web service request relay method, relay server, and program thereof
CN114402577A (en) Caching capabilities for single-page applications

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ OM PH PL PT RO RU SC SD SE SG SK SL TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2003731937

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 20038055473

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 2003731937

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 2003731937

Country of ref document: EP

NENP Non-entry into the national phase in:

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP