EP1969480A1 - Method and apparatus for reliable and efficient content-based routing and query and response in a publish-subscribe network - Google Patents
Method and apparatus for reliable and efficient content-based routing and query and response in a publish-subscribe networkInfo
- Publication number
- EP1969480A1 EP1969480A1 EP03714457A EP03714457A EP1969480A1 EP 1969480 A1 EP1969480 A1 EP 1969480A1 EP 03714457 A EP03714457 A EP 03714457A EP 03714457 A EP03714457 A EP 03714457A EP 1969480 A1 EP1969480 A1 EP 1969480A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- filters
- network
- filter
- notification
- publish
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 439
- 230000004044 response Effects 0.000 title claims abstract description 64
- 230000003993 interaction Effects 0.000 claims abstract description 38
- 230000002085 persistent effect Effects 0.000 claims abstract description 36
- 230000001902 propagating effect Effects 0.000 claims abstract description 32
- 239000003795 chemical substances by application Substances 0.000 claims description 74
- 238000012360 testing method Methods 0.000 claims description 70
- 230000014509 gene expression Effects 0.000 claims description 67
- 238000012545 processing Methods 0.000 claims description 48
- 230000008569 process Effects 0.000 claims description 39
- 238000013507 mapping Methods 0.000 claims description 37
- 238000011144 upstream manufacturing Methods 0.000 claims description 37
- 230000009471 action Effects 0.000 claims description 35
- 230000009467 reduction Effects 0.000 claims description 32
- 230000000644 propagated effect Effects 0.000 claims description 30
- 230000005540 biological transmission Effects 0.000 claims description 26
- 238000003860 storage Methods 0.000 claims description 26
- 230000015654 memory Effects 0.000 claims description 21
- 230000009466 transformation Effects 0.000 claims description 16
- 238000009826 distribution Methods 0.000 claims description 15
- 230000006399 behavior Effects 0.000 claims description 13
- 230000003044 adaptive effect Effects 0.000 claims description 12
- 230000000737 periodic effect Effects 0.000 claims description 9
- 230000001131 transforming effect Effects 0.000 claims description 8
- 230000007246 mechanism Effects 0.000 claims description 7
- 230000008439 repair process Effects 0.000 claims description 5
- 238000013519 translation Methods 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000000844 transformation Methods 0.000 claims description 4
- 230000001629 suppression Effects 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims description 2
- 238000012423 maintenance Methods 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 claims description 2
- 241000288140 Gruiformes Species 0.000 claims 1
- 208000037656 Respiratory Sounds Diseases 0.000 claims 1
- 230000003139 buffering effect Effects 0.000 claims 1
- 206010037833 rales Diseases 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 50
- 230000006870 function Effects 0.000 description 34
- 238000011084 recovery Methods 0.000 description 27
- 230000002688 persistence Effects 0.000 description 24
- 238000001914 filtration Methods 0.000 description 20
- 238000012384 transportation and delivery Methods 0.000 description 20
- 239000008186 active pharmaceutical agent Substances 0.000 description 13
- 239000000243 solution Substances 0.000 description 12
- 230000008901 benefit Effects 0.000 description 11
- 238000013139 quantization Methods 0.000 description 10
- 230000003542 behavioural effect Effects 0.000 description 9
- 238000009825 accumulation Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 8
- 238000007726 management method Methods 0.000 description 8
- 239000011800 void material Substances 0.000 description 6
- 238000013459 approach Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000001965 increasing effect Effects 0.000 description 4
- 238000007689 inspection Methods 0.000 description 4
- 238000007619 statistical method Methods 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 3
- 230000001976 improved effect Effects 0.000 description 3
- 230000007774 longterm Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 230000003466 anti-cipated effect Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000013475 authorization Methods 0.000 description 2
- 239000000470 constituent Substances 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000004141 dimensional analysis Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 230000006855 networking Effects 0.000 description 2
- 230000001105 regulatory effect Effects 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 230000002441 reversible effect Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- SPNQRCTZKIBOAX-UHFFFAOYSA-N Butralin Chemical compound CCC(C)NC1=C([N+]([O-])=O)C=C(C(C)(C)C)C=C1[N+]([O-])=O SPNQRCTZKIBOAX-UHFFFAOYSA-N 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000002155 anti-virotic effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- JLQUFIHWVLZVTJ-UHFFFAOYSA-N carbosulfan Chemical compound CCCCN(CCCC)SN(C)C(=O)OC1=CC=CC2=C1OC(C)(C)C2 JLQUFIHWVLZVTJ-UHFFFAOYSA-N 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 229940090411 ifex Drugs 0.000 description 1
- HOMGKSMUEGBAAB-UHFFFAOYSA-N ifosfamide Chemical compound ClCCNP1(=O)OCCCN1CCCl HOMGKSMUEGBAAB-UHFFFAOYSA-N 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000000059 patterning Methods 0.000 description 1
- 238000012913 prioritisation Methods 0.000 description 1
- 238000005086 pumping Methods 0.000 description 1
- 238000010926 purge Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000013068 supply chain management Methods 0.000 description 1
- 230000008093 supporting effect Effects 0.000 description 1
- 238000012065 two one-sided test Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/30—Profiles
- H04L67/306—User profiles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
- H04L67/5651—Reducing the amount or size of exchanged application data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/566—Grouping or aggregating service requests, e.g. for unified processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/63—Routing a service request depending on the request content or context
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/561—Adding application-functional data or data for application control, e.g. adding metadata
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
- H04L69/32—Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
- H04L69/322—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
- H04L69/329—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]
Definitions
- 60/368,870 entitled “Implementing Query-Response Interactions On a Publish- Subscribe Infrastructure By Mapping Data Advertisements as Subscriptions and Queries as Notifications," filed March 28, 2002
- 60/369,832 entitled “Method and Apparatus for Implementing Persistent and Reliable Message Delivery,” filed April 3, 2002
- Network bandwidth is increasing exponentially.
- the network infrastructure including routers, servers, daemons, protocols, etc.
- Internet applications and network routers cannot keep up with the speed of the bandwidth increase.
- more and more devices and applications are becoming network enabled.
- the load that these devices and applications put on the network nodes have increased tremendously.
- the increase of network load and number of applications also makes the complexity of implementing and maintaining network applications much higher.
- the increase of network bandwidth and the ubiquitous use of network devices and applications can cause problems for routing and transmission of data in the old network infrastructure, particularly when publishing content to subscribers.
- a model for having networks push information from servers to clients is the publish-subscribe style.
- the server becomes a simplified publisher of its information, without regard to which clients may be interested in that information or where they are located in the network.
- the clients become subscribers for information, with information delivered as it becomes available, potentially without regard to details about where in the network it was published.
- the network is then responsible for efficiently routing published information to subscribers, for matching information to active subscriptions, and for doing all of this in a way that is transparent to the publishers and subscribers.
- a heavyweight server and a lightweight client can begin to disappear, or rather to merge into the notion of a peer that can be either publisher, or subscriber, or both.
- Numerous kinds of applications have a natural affinity for publish- subscribe-style interaction between peers.
- a common theme underlying many of these applications is that the information being published and subscribed for is in the form of events. For example, an investor buys or sells a stock, causing the price of the stock to change.
- a traffic incident occurs on a freeway, causing traffic on the freeway to back up.
- a security hole in a software system is discovered, causing a patch to be developed for the users of the software.
- All of these exemplary phenomena are events that are potentially of interest to large numbers of subscribers and can be propagated over a network to notify those subscribers that the events happened.
- An event is thus simply a self-contained, succinct piece of information about something potentially interesting that happened at some point in time at some place on the network.
- Another example involves a scheduled broadcast, which has differing characteristics from applications involving only asynchronous events where the time of events is unpredictable and random. First, the event is scheduled to take place at a known time. Secondly, an event does not need to be a succinct piece of information. Instead, it could be a massive amount of data. Directing this massive load of data to the parts of the network where interested subscribers are found requires substantial server processing.
- the server or publisher performs the routing decisions for the network in order to instruct the network on where to send published content in the publish- subscribe model.
- the publisher stores the subscriptions for content that it publishes.
- the publisher compares the content with each of the subscriptions to identify any matches. If the content (event) satisfies any subscriptions, the publisher pushes the content to the corresponding subscriber via the network.
- This conventional publish-subscribe model places a tremendous burden on the publishers, particular as more devices become network-enabled and as the number of subscriptions increases.
- a complementary approach can be just as odious — a subscriber evaluates its own subscriptions on all published events.
- a method and apparatus provide for propagating filters in a publish-subscribe network.
- a plurality of filters relating to subscriptions to content in the network are received.
- a number of the filters is reduced based upon particular criteria, and the reduced number of filters are propagated for use in satisfying the subscriptions.
- a method and apparatus provide for implementing query-response interactions on a publish-subscribe network.
- An advertisement relating to a data set and a query representing a logical expression are received.
- the advertisement is mapped to a corresponding subscription.
- the query is mapped to a corresponding notification.
- the subscription and the notification are used for implementing of the advertisement and the query in the network.
- a method and apparatus provide for providing persistence of messages transmitted via a network. Messages are received via the network. The messages are stored for later retrieval. Persistent and reliable delivery of the messages is provided in response to failures relating to the network.
- FIG. 1 is a diagram illustrating intelligent routing in a network core.
- FIG. 2 is a network diagram illustrating intelligent routers for publishers and subscribers.
- FIG. 3 is a diagram illustrating a network infrastructure for intelligent routers and backbone routers.
- FIG. 4 is a diagram of hardware components of an intelligent router.
- FIG. 5 is a diagram of publisher and subscriber machines.
- FIG. 6 is a diagram of channel managers for intelligent routers.
- FIG. 7 is a diagram of software components in a user machine for interfacing the machine with intelligent routers
- FIG. 8 is a diagram of software components for an intelligent router.
- FIG. 9 is a diagram of a packet structure for a message.
- FIG. 10 is a flow chart of a publisher method.
- FIG. 11 is a flow chart of a subscriber method.
- FIG. 12 is a diagram of channel and subscriber screens.
- FIG. 13 is a flow chart of a content-based routing method.
- FIG. 14 is a flow chart of a caching method.
- FIG. 15 is a diagram illustrating a cache index.
- FIG. 16 is a flow chart of an agent method for an outgoing message.
- FIG. 17 is a flow chart of an agent method for an incoming message.
- FIG. 18 is a diagram illustrating an example of encoding of a message.
- FIG. 19 is a diagram of a database structure for storing subscriptions.
- FIG. 20 is a flow chart of a wildcard method.
- FIG. 21 is a flowchart of a method for reliable transmission of notifications in a publish-subscribe network.
- FIG. 22A is a flow chart of a filter propagation method.
- FIG. 22B is a flow chart of a filter reduction method.
- FIG. 22C is a flow chart of another filter propagation method.
- FIG. 22D is a diagram of a publish-subscribe network topology.
- FIG. 23 A-23B are flow charts of a content-based routing method using pre- computation.
- FIG. 24A is a flowchart of a query-response method using publish-subscribe.
- FIG. 24B is a sequence diagram further illustrating the method of FIG. 24 A.
- FIG. 24C is a flowchart of another query-response method using publish- subscribe.
- FIG. 24D is a sequence diagram further illustrating the method of FIG. 24C.
- FIG. 24E is a flowchart of another query-response method using publish- subscribe.
- FIG. 24F is a sequence diagram further illustrating the method of FIG. 24E.
- FIGs. 24G and 24H are flowcharts of another query-response method using publish-subscribe.
- FIG. 241 is a sequence diagram further illustrating the method of FIG. 24G and 24H.
- FIG. 25 A is a block diagram illustrating components of a persistent publish- subscribe network.
- FIG. 25B is a diagram illustrating embodiments of cache manager routines.
- FIG. 25C is a flowchart of a persistent caching method.
- FIG. 25D is a flowchart of a persistent message retrieval method.
- An Internet-scale, or other distributed network-scale, event notification system provides applications with a powerful and flexible realization of publish-subscribe networking.
- an application program uses event notification application program interfaces (APIs) to publish notifications and/or to subscribe for and receive notifications about events occurring inside the network.
- APIs application program interfaces
- a notification in the system is given a subject, which is a string or other structure that classifies the kind of information the notification encapsulates.
- a notification is completed with a set of attributes containing information specific to the notification.
- an application might publish notifications about transactions on the New York Stock Exchange using the subject quotes.nyse and attributes symbol and price.
- the application might publish an individual notification having specific attribute values, for example with symbol equal to SNE (the stock ticker symbol for Sony Corporation) and price equal to 85.25.
- SNE stock ticker symbol for Sony Corporation
- price 85.25.
- Most if not all of the attributes in a notification are predefined, in the sense that they are found in all notifications for the same family of subjects. However, publishers can add discretionary attributes on a per-notification or other basis in order to provide additional event-specific information. Therefore, not all or even any attributes need be predefined.
- subscribers are not restricted to subscribing only for subjects or whole channels.
- Channels are further explained and defined below. They can include an hierarchical structure specifying, for example, a subject field and one or more levels of related sub-fields (sub-subjects).
- subscribers can provide much more finely-tuned expressions of interest by specifying content-based filters over the attributes of notifications. For example, a subscriber might subscribe for all notifications for the subject quotes.nyse having symbol equal to SNE and price greater than 90.00 (indicating perhaps a sell opportunity for a block of shares owned by the subscriber). All notifications matching the subscription can be delivered to the subscriber via a callback or other type of function that the subscriber provides at the time it registers its subscription or at other times.
- One subscription can be broken down into many filters.
- the callback can perform many computations, including something as simple as writing a message to a terminal or sending an e-mail, to something more complex such as initiating the sale of a block of shares, and to something even more complex that initiates new publish-subscribe activity (for example, replacing the existing subscription with a new subscription for a buy opportunity at a price of 75.00, or publishing a new notification that the subscriber's portfolio has been modified).
- Applications are aided in their publishing and subscribing activities by agents, for example.
- the agents can possibly make use of or be implemented with proxies.
- the agents when used, provide network connectivity for outgoing notifications and subscriptions and delivery of incoming matching notifications to subscribers. Once a notification enters the network, the system's network of routers propagate the notifications to all subscribers whose subscriptions match the notification. One way of accomplishing this would be to broadcast the notification to all points of the network and then let the application agents decide whether the notification is relevant to their subscribers. However, this is not necessarily a scalable approach — the network would usually be quickly overwhelmed by the load of message traffic, especially in the presence of large numbers of active and verbose publishers.
- FIG. 1 is a diagram conceptually illustrating this intelligent routing in a network core.
- a publisher 14 transmits content in messages via an edge router 16 to a network core 10, used in a publish-subscribe network.
- a publish-subscribe network includes any type of network for routing data or content from publishers to subscribers. The content is transmitted via one or more channels 18 representing logical connections between routers or other devices.
- An intelligent router 12 in network core 10 determines whether to route or forward the message.
- intelligent router 12 can determine if the message includes content as subscribed to by a subscriber 24.
- Each subscription encapsulates a subject filter and an attribute filter.
- Routers can possibly expand a subject filter to the set of matching subjects and merge attribute filters on a per-subject basis.
- An intelligent router evaluates the subject filter against the subject of notifications, and evaluates the attribute filter against the attribute values in notifications.
- the syntax for subject filters can possibly use wildcards, and the syntax for attribute filters can use Boolean expressions, both of which are further explained below.
- the term "filter" is used to describe a set of events that a subscriber is interested in receiving from publishers.
- Routing rules are generated from the filters and are used by intelligent routers to make routing decisions. Therefore, if the entire filter set is not satisfied by a message 26, for example, intelligent router 12 drops (discards) message 26, meaning that the message is not forwarded. If any filter of the entire set is satisfied by a message 20 according to the evaluations of subject and attribute filters, for example, intelligent router 12 routes (forwards) message 20 via edge router 22 and possibly other devices to a subscriber 24, or performs other functions internal to router 12 with message 20, according to all the routing and/or action rules prescribed for the matching filter. The search will continue until either the entire set of filters has been exhausted, or decisions about all the rules have been obtained, whichever comes first.
- This type of intelligent content-based routing in a network core provides for real- time data delivery of, for example, alerts and updates.
- real-time data delivery for alerts include, but are not limited to, the following: stock quotes, traffic, news, travel, weather, fraud detection, security, telematics, factory automation, supply chain management, and network management.
- real-time data delivery for updates include, but are not limited to, the following: software updates, anti-virus updates, movie and music delivery, workflow, storage management, and cache consistency. Many other applications are possible for delivery of information for subscriptions.
- Table 1 illustrates storing of subscriptions with subjects and predicates for the filtering. They can be stored in any type of data structure, as desired or necessary, anywhere in the network. As explained below, the predicates are components of subscriptions. The subscriptions can be expressed in any way, examples of which are provided below.
- Table 2 provides an example of a publication and subscription for a quote server. This example is provided for illustrative purposes only, and subscriptions can include any number and types of parameters for any type of data or content.
- the predicates provide the Boolean expressions for the subscription and the subjects provide an indication of a channel for the subscription.
- Subscriptions can be expressed in many different ways. Use of Boolean expressions is one such example and provides an ability to easily convert the subscription into a subject filter and an attribute filter for content-based routing. Subscriptions can alternatively be expressed without reference to a subject; however, use of a subject or channel (further explained below) provides a context for interpreting and applying filters to attributes.
- FIG. 1 illustrates one publisher, one subscriber, and one intelligent router for illustrative purposes only; implementations can include many publishers, subscribers, and intelligent routers.
- intelligent router refers to a router or other entity having the ability to make routing decisions by inspecting the payload of a packet or message in a network core or other locations.
- FIG. 2 is a network diagram illustrating intelligent routers for publishers and subscribers.
- a routing entity 30 providing channel services is, for example, effectively layered on a network infrastructure, as explained below, for routing messages among intelligent routers.
- a publisher 32 conceptually includes, for example, an application 34 to receive an indication of published content, such as a pointer for retrieving the content, and an agent 36 to encode the content for network transmission via channel services 30.
- a collection of logically interconnected intelligent routers 38, 40, 42, 44, 46, and 48 route the content from the publisher using routing rules generated from subject filters and attribute filters for subscriptions.
- a plurality of links 39, 41, 43, and 45 provide the logical connections between intelligent routers 38, 40, 42, 44, 46, and 48.
- Other links 37 and 47 provide, respectively, logical connections between publisher 32 and intelligent router 38, and between a subscriber 54 and intelligent router 46.
- Subscriber 54 includes an agent 50 to detect and receive the subscribed content, and an application 52 to present the content.
- a channel can include, for example, a related set of logical multicast connections implemented in a distributed manner.
- a channel in this exemplary embodiment is a logically related collection of network resources used to serve a community of publishers and subscribers exchanging content. The content is classified according to the channel subject namespace, and the resources are managed, controlled, and provisioned via channel services provided by channel managers. Multiple channels may share the same resources.
- Channels can provide a highly scalable directory service such as, but not limited to, the following examples: publisher and subscriber information, authentication and authorization information, message types, management information, and accounting and billing information.
- Channels can also provide, for example, persistence through caching, a fast data delivery mechanism, security, and user and network management. Channels can be used for any other purpose as well.
- the filtering by the intelligent routers can occur in a network core to distribute routing decisions.
- intelligent routers can also function as edge routers connecting a user device, such as a publisher or subscriber, with the network core.
- the same device connected to the network can function as both a publisher to push content to subscribers via routing decisions in the network and as a subscriber to received pushed content.
- the intelligent routers and channels can be connected in any configuration, as necessary or desired for particular implementations, and the configuration shown in FIG. 2 is provided for illustrative purposes only.
- FIG. 3 is a diagram of an exemplary network infrastructure for intelligent routers and conventional backbone routers, also illustrating logical connections for channels.
- the intelligent routers in this example use existing backbone routers in the network, such as the Internet or other distributed network, and the intelligent routers are thus effectively layered on the backbone routers.
- Internet Service Provider (ISP) networks 58, 59, and 60 each include several backbone routers for conventional routing of messages or packets.
- a plurality of intelligent routers 61-70 are connected with one or more backbone routers in ISP networks 58, 59, and 60.
- Intelligent routers 61-70 are also interconnected by a plurality of links 73-85, representing examples of links, and can be connected to end user devices by the links as well.
- Intelligent routers 61-70 can be controlled by one or more administrator machines such as an entity 71, and one or more virtual private network (VPN) controllers such as an entity 72.
- the ISP networks 58, 59, and 60 would also be connected to publisher and subscriber machines (not shown in FIG. 3).
- the backbone routers in and among ISPs 58, 59, and 60 are interconnected in any conventional way within the existing network infrastructure.
- the intelligent routers 61-70 and links 73-85 can be implemented using existing network infrastructure, and they provide for content-based routing in the network core.
- the links 73-85 represent logical connections between intelligent routers 61-70 and can be implemented using, for example, existing network infrastructure or other devices.
- a link for example, can be implemented using a logical connection called the tunnel.
- a tunnel includes the hardware, and possibly software, network infrastructure for implementing a link, and one tunnel can be a component of multiple channels.
- the channels facilitate content-based routing in the intelligent routers by providing logical configurations for particular types of content and thus providing a context for attributes transmitted over the channels.
- intelligent routers can perform routing decisions without channels, the channels enhance the efficiency of content-based routing by the intelligent routers in the network core.
- a link is a connection between two routers-albeit intelligent routers.
- a channel is a network entity encompassing a (typically large) collection of routers, configured statically or dynamically by the interconnecting links to achieve one-to-many or many-to-many logical connections.
- a channel is a top-level logical entity describing the essential characteristics of the channel. Under one channel, there could be many subjects. Each subject will form a sub-network (such as a multicast tree) involving a collection of interconnected routers. These subject-based sub-networks can be allocated, oriented, and configured in different manners.
- the channel being a collection of all the sub-networks formed for the subjects under it, may resemble a mesh of networks, for example.
- FIG. 4 is a diagram of exemplary hardware components of an intelligent router 92, which can correspond with any of the other referenced intelligent routers.
- a network node 90 can include intelligent router 92 connected with a conventional backbone router 95.
- Intelligent router 92 includes a processor 93 connected to a memory 94 and a secondary storage 97 (possibly implemented with a detached machine, for example), either of which can store data, as well as cache data, and store applications for execution by processor 93.
- Secondary storage 97 provides non- volatile storage of data.
- processor 93 provides instructions to backbone router 95 for it to route (forward) or not route (discard) messages or packets based upon routing rules generated from subject filters and attribute filters for subscriptions.
- intelligent router 92 can alternatively be implemented in an application specific integrated circuit (ASIC) within backbone router 95 to provide the intelligent routing functions in hardware possibly with embedded software.
- ASIC application specific integrated circuit
- the intelligent routing functions can also be alternatively implemented in a combination of software and hardware in one or multiple routing devices.
- FIG. 5 is a diagram of exemplary publisher and subscriber machines.
- a publisher machine 100 or 118 can include the following components: a memory 102 storing one or more publisher applications 104 and an agent application 105; a secondary storage device 112 providing non- volatile storage of data; an input device 108 for entering information or commands; a processor 114 for executing applications stored in memory 102 or received from other storage devices; an output device 110 for outputting information; and a display device 116 for providing a visual display of information.
- a subscriber machine 122 or 140 can include the following components: a memory 124 storing one or more applications 126 and an agent application 128; a secondary storage device 130 providing non- volatile storage of data; an input device 132 for entering information or commands; a processor 134 for executing applications stored in memory 124 or received from other storage devices; an output device 136 for outputting information; and a display device 138 for providing a visual display of information.
- Publisher and subscriber machines can alternatively include more or fewer components, or different components, in any configuration.
- Network 120 includes intelligent routers for providing distributed routing of data or content in the network core via packets or messages. Although only two publisher and subscriber machines are shown, network 120 can be scaled to include more publisher and subscriber machines.
- the publisher and subscriber machines can be implemented with any processor-controlled device such as, but not limited to, the following examples: a server; a personal computer; a notebook computer; a personal digital assistant; a telephone; a cellular telephone; a pager; or other devices.
- Network 120 with intelligent routers can include any wireline or wireless distributed network, connecting wired devices, wireless devices, or both. Network 120 can also potentially use existing or conventional network infrastructure.
- FIG. 6 is a diagram illustrating channel managers 150 for intelligent routers.
- channel managers 150 are implemented with multiple servers 152, 154, and 156. Each server includes its own local storage 158, 160, and 162.
- Intelligent routers 164, 166, and 168 contact channel managers for information about particular channels.
- the channel managers can also provide for data persistence, fail over functions, or other functions.
- the channel managers thus provide the channel services, which include a database or set of databases anywhere in the network specifying, for example, channel-related information, properties for data persistence, user information for publishers and subscribers, and infrastructure information.
- the infrastructure information can include, for example, an identification of intelligent routers and corresponding tunnels connecting them, subjects for the channels, and attributes for the channels (a name and type for each attribute).
- Packets or messages can also carry channel-related information including identification of fixed attributes and variable attributes.
- a user when on-line can download channel information. For example, a user can register by using a user name and password. Upon authenticating the user's log-on, the user can open (invoke) a channel and retrieve information about the channel from the channel managers. Publishers can use that information in publishing content, and subscribers can use that information for entering and registering subscriptions.
- Each channel manager 152, 154, and 156 acts as a primary for each intelligent router.
- each intelligent router is provided two Internet Protocol (IP) addresses in this example, one for a primary channel manager and another for a back-up channel manager. The intelligent routers use those IP addresses to contact a channel manager and retrieve channel information.
- IP Internet Protocol
- an intelligent router can contact a back-up channel manager.
- the channel managers 152, 154, and 156 thus share data, as indicated by the lines connecting them, concerning channel properties and other information.
- Each channel manager also has a designated back-up so that if the channel manager fails, another one can take over processing for it.
- Devices in the network can use commands, for example, to retrieve channel information, examples of which are provided in Table 3.
- Intelligent routers can alternatively only have a primary channel manager or more than two channel managers.
- FIG. 7 is a diagram of exemplary software components in a stack 180 in a user machine or device for connecting it with a network having intelligent routers.
- the user machine can be used as a publisher, subscriber, or both, and it can include the exemplary devices identified above.
- Stack 180 can include one or more user applications 182, which can provide for receiving subscriptions from a user, receiving channel information from a publisher, or receiving content or data to be published.
- User application 182 can also include any other type of application for execution by a user machine or device.
- the stack 180 can also include, for example, an agent 184, an event library 186, a cache library 188, a channel library 190, a messaging library 192, and a dispatcher library 194.
- Agent 184 provides for establishing network connections or other functions, and Table 3 provides examples of commands implemented by agent 184, which can use proxy commands or other types of commands.
- Event library 186 logs events concerning a user machine or other events or information.
- Cache library 188 provides for local caching of data.
- Channel library 190 stores identifications of channels and information for them.
- Dispatcher library 194 provides connections with a control path 196, a channel manager 198, and one or more intelligent routers 200, and it can include the exemplary functions identified in Table 4.
- Messaging library 192 provides a connection with a data path 204.
- Tables 5-9 provide examples of messaging APIs in the C programming language.
- Tables 5 and 6 provide examples of APIs to send and retrieve messages.
- Tables 7 and 8 provide examples of APIs to send and retrieve notifications.
- Table 9 provides examples of APIs to send and retrieve control messages.
- PC_msg_SessionHandle *sess PC_Status PC_msg_cleanup(PC_msg_SessionHandle sess); PC_Status PC_msg_closeTransport(PC_msg_SessionHandle sess); PC_Status PC_msg_create(PC_msg_SessionHandle s, PC_msg_DataType dType,
- PC_Status PC_msg_setSubj ect PC_msg_MsgHandle msg, PC_CHAR *subj ect
- PC_Status PC_msg_setSubjectint PC_msg_MsgHandle msg
- PCJNT value // for each type PCjStatus PC_msg_send(PC_msg_MsgHandle msg);
- PC_Status PC msg init(ChannelHandle ch, PC HINT chid, PC UINT userid,
- PC_Status PC_msg_recv(PC_msg_SessionHandle sh, PC_msg_MsgHandle *msg);
- PCjStatus PC_msg_ctrlRecv(PC_msg_SessionHandle sh, PC_msg_MsgHandle
- PCJJINT attributePos PC_msg_Attribute **attr
- PC_msg_MsgHandle msg PC_msg_SessionHandle sh; PC_msg_TypeInfo Types[2]; PC_msg_AttributeArray *attrArray; PC_CHAR *com ⁇ any; PC_INT value;
- PC_msg_getAttrValueByPosString (msg, 0, &company); PC_msg_getAttrValueByNameInt(msg, "stockvalue", &value); PC_msg_getDynamic Attributes(msg, &attrArray) ; PC_msg_freeAttributeArray(attr Array) ; PC_msg_delete(msg);
- FIG. 8 is a diagram of exemplary software components 210 for an intelligent router such as those identified above and intelligent router 92 shown in FIG. 4.
- Software components 210 can be stored in, for example, memory 94 for execution by processor 93 in intelligent router 92.
- Components 210 include, for example, a filtering daemon 212, a dispatcher 214, a routing daemon 216, and a cache manager 218.
- Filtering daemon 212 provides filtering for content-based routing to process content for subscriptions according to routing rules, as explained below.
- Dispatcher 214 provides for communication of control messages such as those required for propagating filters via path 220, and the dispatcher can also provide for a single point of entry for users and one secure socket with channel managers, enhancing security of the network.
- Dispatcher 214 uses control messages to obtain attributes (name-value pairs) from a channel manager.
- Routing daemon 216 provides for communication with a data path 222, which can occur via a conventional backbone router as illustrated in FIG. 4 or other routing device.
- Cache manager 218 provides for local caching of data at the network node including the corresponding intelligent router. The operation of cache manager 218 is further explained below, and it provides for distributed caching of data throughout the network core.
- Content-based routing can be implemented at the kernel level, as an alternative to the application level.
- Memory accessible by the kernel is separate from that in the application layer.
- To have content-based routing running in the application requires, for example, that message data be copied from the kernel memory area to the application area, and switching the context of the application from that of the kernel to that of the routing application. Both can induce substantial overhead. If instead the kernel is modified to support content-based routing, the routing could take place much faster being rid of the overhead described above.
- the routing daemon 216 may or may not directly send or receive data via the data path 222, depending on the implementation.
- the daemon is a process running in the application layer, pre- computing the content-based routing table to be injected into the kernel. Once injected, however, the routing table can be used by the kernel to make routing decisions.
- the filtering daemon pre-computes the filtering table and injects it into the kernel.
- the routing daemon nor the filtering daemon would directly interact with the data path.
- FIG. 9 is a diagram of an example of a packet structure 230 for a message possibly including content for subscriptions.
- a packet or message for use in content- based routing includes, for example, a header section and a payload section.
- the header section specifies routing or other information.
- the payload section specifies data or content, or an indication of the data or content.
- Packet structure 230 includes an IP header 232, a User Datagram Protocol (UDP) Transmission Control Protocol (TCP) header 234, a length value 238, one or more subject fields 240, and one or more attributes 242.
- Packet structure 230 illustrates a basic structure for a length value and the subjects and attributes.
- a packet used in content-based routing can also include other or different elements, such as those illustrated in the example of FIG.
- the attributes can include discretionary attributes appended to the end of a message, for example. These discretionary attributes are ad-hoc information, for example, added by the publisher (or even routers) that cannot necessarily be conveyed using the message format prescribed for the channel.
- FIG. 10 is a flow chart of an exemplary publisher method 250 for use by a publisher to set-up a channel and publish content.
- Method 250 can be implemented, for example, in software modules including agent 106 for execution by processor 114 in publisher machine 100.
- agent 106 in the publisher machine receives a publisher creation of a proxy for a channel (step 252).
- the proxy provides for communication with the network.
- Agent 106 determines a message format for the channel through an interface (step 253), and the format information can be obtained from, for example, the channel managers or other entities in the network.
- Agent 106 sets up the proxy for the channel using the received channel information (step 254), which includes receiving attributes for the channel (step 256) and creating a notification on the channel (step 258).
- the notification provides content for devices "listening" for content on the channel.
- the attributes define parameters and characteristics for the notification.
- Agent 106 transmits an identifier (ID) of the channel and content information to intelligent routers in the network core or elsewhere for use in processing subscriptions (step 260).
- ID identifier
- the publisher populates the notification attributes with appropriate values (step 261), and the publisher can then publish content on notification in accordance with the channel attributes (step 262).
- Steps 260-262 in this example accomplish publishing the notification, which can alternatively involve different or additional steps depending upon a particular implementation. Therefore, the information associated with a notification in this example is partitioned into an ordered sequence of attributes, each of which has a name, a position within the notification (starting at 1), a type, and a value.
- attributes can have different characteristics depending upon a particular implementation. Attributes can include, for example, predefined attributes, discretionary attributes, or both.
- the intelligent routers can use the channel ID in a packet to obtain the attributes for the corresponding channel, which determines the structure or format for packets transmitted via the channel.
- each packet can contain, for example, a tag associated with a channel ID and other header information such as a publisher ID and subjects.
- the tags can be used to map subjects to numbers in the message format, an example of which is shown in FIG. 18. Small integer values, for example sixteen bit values, can be used for the numbers. Alternatively, any other type of numbers or information can be used to map the subjects. Mapping subjects to numbers can provide particular advantages; for example, it can save space in the message format and provide a uniform or standard way to specify indications of the subjects in the message so that they can be quickly located and identified.
- Intelligent routers can locally store the mapping or, alternatively, use the numbers to remotely obtain the corresponding subject through a command.
- Table 10 illustrates a structure for mapping numbers to subjects, in this example using integer values.
- the subject tree parameter in the table indicates that a subject can include one or more subject fields in an hierarchical relationship; for example, a subject tree can include a string of subject fields demarcated by particular symbols. Examples of subject trees are provided in Table 2.
- a subject tree quotes.nyse includes a subject "quotes" and a sub-field "nyse" with those two terms demarcates by a ".” as found in URLs or other network addresses. Aside from using periods and specifying URL-type strings, subject trees can be specified in any way using any characters and symbols for demarcation.
- the intelligent routers can quickly locate subjects and attributes, or other information, in the packet for content-based routing. For example, a channel can specify byte positions of subjects and attributes transmitted over the channel, making them easy to locate by counting bytes in the packet. Alternatively, intelligent routers can parse packets to locate subjects and attributes, or other information.
- Table 11 provides an example of a publisher program in the C++ programming language.
- Table 12 provides an example of an API to create a channel.
- Table 13 provides an example of a channel configuration file maintained by a channel manager (see FIG. 6) and providing channel-related information, as illustrated.
- the system can alternatively have a global channel manager providing IP addresses of geographically dispersed servers functioning as local channel managers in order to distribute the processing load.
- Table 11 provides an example of a publisher program in the C++ programming language.
- Table 12 provides an example of an API to create a channel.
- Table 13 provides an example of a channel configuration file maintained by a channel manager (see FIG. 6) and providing channel-related information, as illustrated.
- the system can alternatively have a global channel manager providing IP addresses of geographically dispersed servers functioning as local channel managers in order to distribute the processing load.
- Table 11 provides an example of a publisher program in the C++ programming language.
- Table 12 provides an example of an API to create a channel.
- Table 13 provides an example of
- n2 (p, "quotes.nyse”); n2.SetPredefinedAttr(l, "SNE”); // attribute symbol is in position 1 n2.SetPredefinedAttr(2, 80.18); // attribute price is in position 2 p.Publish(n2);
- PCjStatus re; re PC_chn_create(Provider nfo, authinfo, ConfigurationFile, &hChannel);
- the type information is propagated to all edge routers.
- # type of information e.g.
- the MulticastIP is
- FIG. 11 is a flow chart of a subscriber method 264 for use in receiving and processing subscriptions.
- Method 266 can be implemented, for example, in software modules including agent 128 for execution by processor 134 in subscriber machine 122.
- GUI graphical user interface
- the information identifying the channels can be received from, for example, the channel managers providing channel-related information.
- Any type of application 126 can be used for presenting identifications of channels in any particular way or format.
- the application receives a user's selection of a channel (step 268) and calls an API or other program for the selected channel (step 270).
- the API presents subscription options to the user for the channel corresponding with the selected option (step 272).
- the API receives values for the subscription from the user (step 274) and sends the subscription to agent 128 for processing, as explained below (step 276).
- the parameters for the subscription can include, for example, the predicates as illustrated in Table 1.
- Each channel can use its own API, for example, in order to process subscriptions according to the particular requirements or parameters for the corresponding channel.
- These APIs can include, for example, web-based or Java-based APIs for receiving subscriptions and can use any type of user interface and processing to receive information for a subscription and pass it along to the agent application.
- FIG. 12 is a diagram conceptually illustrating channel and subscriber screens or GUIs 278 and 284, which can be used in conjunction with method 264 for receiving a subscription.
- Screen 278 includes a plurality of sections 282 identifying available channels for selection by a user.
- screen 284 can be displayed for receiving a user's values for the subscription in a section 286.
- a user can select a section 288 to submit the subscription or select a section 290 to cancel the subscription.
- Screens 278 and 284 can be formatted as, for example, HyperText Markup Language (HTML) web pages or in any other format.
- HTML HyperText Markup Language
- the screens can include any configuration of sections and content, possibly including, for example, text, graphics, pictures, various colors, or multi-media information in order to provide, as desired, a user-friendly and visually appealing interface for subscribers.
- the screens can also include a toolbar 280 providing, for example, conventional browser functions.
- Table 14 provides an example of a subscriber program in the C++ programming language.
- SubscriberApp a a.run(); Content-Based Routing Via Payload Inspection and Channels
- FIG. 13 is a flow chart of a content-based routing via payload inspection method 300.
- Method 300 can be implemented, for example, in software modules for execution by processor 93 in intelligent router 92, as represented by filtering daemon 212. Alternatively, it can be implemented in an ASIC or a combination of hardware and software.
- the content-based routing as illustrated in method 300 can be performed in intelligent routers anywhere in the network, such as in the network core or in edge routers.
- the content-based routing involves inspecting a payload section of a packet in order to determine how to process the packet.
- This content-based routing methodology can include, for example, processing a list of subscriptions (using filters, for example) in any order, comparing a message subject-by-subject and attribute- by-attribute with routing rules to determine a routing for the message, and performing the processing in a network core.
- the rules can include rules governing in-router processing or any rules associated with a filter. These routing decisions can thus be distributed throughout a network core.
- the use of subjects as represented by channels determines a message format, thus providing an intelligent router with a way of quickly locating attributes within the message, for example by knowing their byte positions in the message or packet for a particular channel.
- intelligent router 92 receives a packet for a message (step 302). It determines from the packet a channel ID for the corresponding message (step 304) and retrieves attributes for the channel using the channel ID (step 306). In this example, the type of channel (determined from the channel ID) determines locations and data types of attributes in the packet. The attributes for the channel can be locally stored or retrieved remotely such as via a channel manager. Intelligent router 92 retrieves a filter, which corresponds with a subscription (step 308). The filter includes one or more attribute tests, usually a group of attribute tests for subscriptions. Intelligent router 92 applies attributes in the packet to the corresponding attribute test(s) in the filter description (step 310). If all the attribute test(s) in the filter description produce a positive result (step
- the intelligent router executes a set of functions prescribed by the rules associated with the filter (step 314). These functions can include, for example, routing the packet to the next link, and/or performing some action or computation with the content of the packet at the local router as prescribed by the rule(s).
- the action or next link can be identified, for example, in a data structure specifying the corresponding subscription.
- the rule is a link, it typically identifies the next network node to receive the packet, which can include an intelligent router, backbone router, a network-connected device, or other entity.
- the next links can be specified or associated with the subscriptions in other ways.
- step 312 If all the attribute test(s) in the filter description did not produce a positive result (step 312), meaning the attributes do not satisfy all the attribute test(s), the filter is declared a mismatch (step 315).
- the intelligent router recursively follows the above procedure until all the attribute tests in the filter description are exhausted or a first negative result is encountered, whichever comes first.
- the intelligent router determines if more filters exist (step 316) and, if so, it returns to step 308 to retrieve the attribute test(s) for the next filter to process the attributes for it.
- the matching procedure (steps 308, 310, 312, 314, 315, and 316) continues until either the complete set of filters is exhausted, or results for all the action or routing rules can be determined, whichever comes first. If the packet does not satisfy any filter, it will be dropped (discarded) and not forwarded.
- Intelligent router 92 can sequence through the filters in any particular order. For example, as illustrated in Table 15, intelligent router can store the filters for subscriptions in a file or routing table and linearly sequence through them to apply the attributes to filters (attribute tests).
- the routing table can include links or pointers to the filters.
- the content-based routing can optionally use more than one method at the same time, depending on the applications and performance-enhancing heuristics such as the switching of algorithms based on traffic conditions, for example.
- the filters for the processing can optionally be encrypted, decrypted, transformed, and merged at a router in the network for use in performing inspecting of a payload section for the content- based routing. For example, a subscription such as price > $3.54122 may be truncated to price > $3.54 because the publications in the application are known not to contain currency attributes beyond the second decimal points. Also, foreign currency may be translated into U.S. currencies as well when a publication sent from overseas reaches the first router located in the U.S., for example.
- intelligent router 92 can select filters for processing in other orders or according to various algorithms that can possibly enhance the speed and efficiency of processing.
- Table 16 provides examples of subscriptions and corresponding links for them; in these examples, the subjects relate to a particular channel and the subscriptions for the subjects can be represented by routing rules for the filters.
- the subjects can include, for example, network addresses such as Uniform Resource Locators (URLs) identifying a source of content.
- URLs Uniform Resource Locators
- FIG. 14 is a flow chart of a caching method 320.
- Method 320 can be implemented, for example, in software modules for execution by processor 93 in intelligent router 92, as represented by cache manager 218. Alternatively, it can be implemented in an ASIC or a combination of hardware and software, either in the same or different physical device as the corresponding intelligent router, h method 320, intelligent router 92 receives a message having data or content, a channel ID, and subjects (step 322). intelligent router 92 time marks the data (step 324) and locally caches it such as in memory 94 or secondary storage 97 (step 326). It indexes the cached data by, for example, channel ID, subjects, and time stamps (step 328).
- intelligent router 92 receives a request for data (step 330), it retrieves cached data, using the index, according to the request (step 332). Intelligent router 92 transfers the cached data to backbone router 95 or other routing entity for eventual transmission to the requestor or others.
- Method 320 can be repeatedly executed in order to continually cache data and retrieve cache data in response to requests.
- FIG. 15 is a diagram illustrating a cache index (336) for use with method 320.
- Cache index (336) receives data (338) and stores it with time stamps (340). As data is gathered, it is marked upon every duration of delta t, where delta t represents the time between marks, for example t 2 - ti. Other types of indexes for time marking in any way can alternatively be used.
- Table 17 conceptually illustrates indexing of cached data.
- Table 18 conceptually illustrates a data structure for storing a connection history for caching.
- Table 19 provides examples of data structures for use in locally caching data in network nodes having intelligent routers.
- the time marking can occur at any fixed or variable interval. For example, data can be cached and indexed every five minutes.
- channel manager 218 Upon receiving a command to retrieve cached data (such as #.getCache) specifying a time and subject, channel manager 218 uses the cache index to determine if it can retrieve cached data corresponding with the request for step 332.
- Each subject or channel can include, for example, its own IP address in a multicast tree and a set of intelligent routers. Therefore, Table 18 represents a connection history among such routers that can be locally stored a user machine; if an edge router fails, the machine can access the connection history to determine how to reconnect with upstream routers for the channel when the edge router comes back online. It can also execute a get cache command for the duration of the time that it was disconnected in order to obtain any pending content for subscriptions, for example.
- PC CHAR pLastTGrainData; /* could be a list */
- a subject node contains a subject identifier, subject level, pointer to parent channel or subject node, file descriptor for its own directory, pointer to hash table containing its next level subject nodes, and pointer to a data node.
- a data node contains a pointer to its subject parent node, file descriptor for the data directory, circular buffer containing the data structures for the data stored on each storage device, head and tail of the buffer, and lock for locking the data node during retrieval and storage.
- the stored time grain node is the node representing the actual data file, and the last time grain node represents the last buffer that has not yet been stored to the storage device but is maintained in memory.
- the caching and data storage threads in this example use the mutex of the last time grain node for preventing concurrent access to the last time grain node.
- FIG. 16 is a flow chart of an agent method 350 for an outgoing subscription message.
- Method 350 can be implemented, for example, in software modules as represented by agent 128 for execution by processor 134 in user (subscriber) machine 122.
- agent 128 receives a subscription such as via the method described above in FIGS. 11 and 12 (step 352).
- Agent 128 creates a string specifying a Boolean expression for the subscription (step 354) and parses the string to detect any e ⁇ ors in the subscription (step 356). If an error exists, agent 128 can present an error message to the user (step 360) in order for the user to correct the error and re-enter the subscription.
- agent 128 stores the expression in a data structure, an example of which is provided below (step 362).
- Agent 128 translates constituent not-equal expressions in the data structure to positive form (step 364) and translates the data structure to a co ⁇ esponding disjunctive normal form (DNF) structure (step 366).
- Agent 128 also simplifies AND expressions of the DNF structure to contain only range filters and membership tests (step 368).
- the DNF is a well-known canonical form in which a Boolean expression is represented as an OR of one or more sub-expressions called disjuncts, each sub- expression being an AND of one or more attribute tests.
- This transformation is performed prior to creation of the DNF, and it is needed because the routers in this example require formulae to be in positive form.
- the transformation in step 368 is performed after the DNF is created and involves an extra simplification of the resulting AND expressions, and it is also performed to simplify the work of the routers in this example.
- an AND of multiple attribute tests for the same attribute can be simplified into a canonical "range filter" having either one lower bound, one upper bound, both a lower and upper bound, or a single value in the case of an equality test.
- range filter is then encoded according to Table 22.
- the expression simplifies to false, since no value for "price" can satisfy the expression.
- the agent need not create a message at all, thereby relieving the router of unnecessary work.
- agent 128 can optionally convert them as explained below (step 370). Otherwise, any wildcards can be converted in the network, rather than on the user machine or other device.
- the syntax for subject filters is the only syntax that uses wildcards
- the syntax for attribute filters is the only syntax that uses Boolean expressions.
- implementations can use different or varying types of syntax for subject filters and attribute filters.
- Agent 128 encodes the resulting DNF expression into a message (step 372) and transfers the message to an intelligent router (step 374).
- the encoding can involve converting the subscription to a flat message format, meaning that it constitutes a string of data.
- This transferring can involve propagating routing rules generated from subject filters and attribute filters for the subscription to one or more intelligent routers or other routing entities in the network.
- the subscription expression can be mapped into a conventional packet structure, for example.
- the encoding for step 372 involves marshalling subscriptions for a channel into a messaging format of the messaging API for propagation throughout a channel.
- a subscription is internally messaged, for example, as a notification with subject
- #.SUBSCRIPTION Because there are both a variable number of subject filter fields and a variable number of attribute tests, one pair of bytes is used to store the number of subject filter fields, and another pair of bytes is used to store the number of attribute tests in this example.
- the individual fields of the subject filter are marshaled sequentially, for example, in the order in which they were specified in the original subscription and are each marshaled into a two-byte portion of the message. Wildcard fields can be marshaled as described below.
- the operands of the tests are marshaled at the end of the message in a manner similar to the marshaling of attribute values of notifications.
- they Prior to marshaling the attribute tests and operands, they are sorted by attribute order within each disjunct of the DNF with tests on predefined attributes in position order, followed by tests on discretionary attributes in name order.
- the set of relational tests on scalar valued attributes within each disjunct are simplified to a canonical form as range filters having either one limit (for left- or right-open ranges or equality tests) or two limits (for closed ranges between distinct limits).
- the remaining information about the tests is encoded into, for example, two-byte pairs in the same order as the operands; this sequence of two-byte pairs is placed in the message immediately following the sequence of two-byte encoding of subject filter fields.
- the two-byte pairs can constitute one form of a sequence of bit-string encodings of attribute tests, which can also be used to represent other types of encodings aside from two-byte pairs. Examples of attribute tests are provided below.
- Table 20 The schema for the encoding of the attribute tests is depicted in Table 20.
- Table 21 illustrates encoding for the two-byte pairs
- Table 22 illustrates encoding of the Operator ID in the two-byte pairs.
- the Subscription ID field of the message is a value generated by the agent for uniquely identifying the subscription to the agent's edge router in subsequent requests to modify or unsubscribe the subscription, hi particular, a dynamic modification to the attribute filter of a subscription is propagated using the message format shown in the example of FIG. 18, except that the subject is #.RESUBSCRIPTION and the Subscription ID is that of the previously registered subscription being modified. And an unsubscription is propagated using, for example, the message format of FIG. 18 up through the Subscription ID field, with the subject being #.UNSUBSCRIPTION and the Subscription ID being that of the previously registered subscription being unsubscribed.
- FIG. 19 presents a Unified Modeling Language (UML) diagram 390 depicting the objects used by the agent in step 362 to store the expression.
- UML Unified Modeling Language
- This diagram illustrates an hierarchical relationship for specifying the subscription, which can include variables, constant values, or both.
- the objects in the diagram can be instances of filter classes depending upon a particular implementation.
- Each SimpleFilter object depicts the values of attributes used to store information about a co ⁇ esponding attribute test of the filter expression, hi the expression of FIG. 19, an OR filter 396 connects two AND filters 392 and 400.
- the AND filter 392 contains a simple filter 394 with attributes for the subscription.
- the OR filter 396 contains a simple filter 398
- the AND filter 400 contains simple filters 402 and 404.
- attributes price, symbol, and volume are assumed to be predefined attributes of the associated channel and are assumed to be defined in positions 0, 1 and 2, respectively.
- the types of the attributes are assumed to be unsigned integer (typecode 6), character array (typecode 12), and unsigned integer (typecode 6), respectively.
- FIG. 18 presents the marshaling of the subscription into a message.
- the schematic 386 on the left side of FIG. 18 shows the actual message contents, while the schematic 388 on the right provides a legend for the different parts of the message.
- the width of each schematic in this example is four bytes.
- the sixteen-bit attribute test encodings are shown as bit sequences, with gaps showing the separation into the different parts. Note that the two tests on price in this example cannot be combined since they are in separate disjuncts, and thus they are marshaled separately as ranges that have no right bound ("right-open ranges"). On the other hand, the two tests on volume can be combined since they are in the same disjunct, and thus they are marshaled together as a single "closed-range” test.
- FIG. 17 is a flow chart of an agent method 376 for an incoming message.
- Agent 128 receives a message from an intelligent router co ⁇ esponding with a subscription (step 378).
- Agent 128 determines a channel co ⁇ esponding with the subscription (step 380), for example by the channel ID in the message, and calls an API for the channel (step 382).
- the API present the data for the subscription in a GUI or other format at the user machine (step 384).
- the processing of incoming messages can use a process of decoding the data in the reverse of the encoding process described above, and this decoding (reverse encoding) can be performed in a router or in other network entities. Wildcard Processing
- FIG. 20 is a flow chart of a wildcard method 410.
- This method illustrates an example of using a set of routing rules for a filter to convert wildcards in expressions for subscriptions.
- Method 410 can be implemented, for example, in software modules as represented by agent 128 for execution by processor 134 in user machine 122.
- wildcards can be processed in the network by processor 93 under software control in intelligent router 92 or in the co ⁇ esponding functions contained in ASIC 91. Wildcards include open fields or variable length fields, examples of which are provided in Table 21.
- agent 128 or other entity receives a subscription having a wildcard (step 412).
- the subject length for subscriptions can be specified by a publisher when publishing content, and the subject can be pre-processed on the publisher machine, for example, to count the fields of the subject and thus obtain a field count (length) for it.
- Agent 128 retrieves a sub-field for the subscription (step 418) and determines if the filter operand sub-field O[i] is a wildcard (step 420).
- the parameter "i" represents a field where i is an integer representing the field number in this example.
- agent 128 determines if the last filter operand sub-field is a ">" (step 426) and, if so, it changes the length constraint to field length > N-l (step 428).
- Wildcard processing can use any type of symbol, and a ">" is only one such example, hi this example, a "a " can mean a.b, a.c, a.d, etc. and all their sub- subjects at all levels (for example, a.b.x, a.c.x, a.b.x.y, etc.). Other symbols can be used for other implementations of wildcards.
- agent 128 propagates the transformed rule to intelligent routers or other entities in the network (step 430). Accordingly, the method iterates through the sub-fields in order to process them for conversion of the wildcards to non- wildcard rules, meaning rules that do not contain wildcards.
- the conversion of wildcards can occur anywhere in the network, for example on the subscriber machine or in an intelligent router. The conversion can thus occur in one entity with the transformed rule propagated to other entities or it can occur dynamically.
- Table 23 provides a summary, along with examples, of these exemplary routing rules for processing wildcards. These routing rules can be generated in the intelligent routers, for example, or generated in other network entities and propagated to the intelligent routers. In addition, the routing rules in Table 23 are provided for illustrative purposes only and other routing rules are possible for converting wildcards.
- FIG. 21 illustrates a method 520 for providing reliability in publish-subscribe networks. In this method 520, a reliable transmission protocol is used.
- Method 520 can be executed by a processor 93 under software control in an intelligent router 92 or in the co ⁇ esponding functions contained in ASIC 91. Method 520 can also be implemented, for example, in software modules, e.g., as represented by an agent 105 for execution by a processor 114 in a user machine 100.
- TCP Transmission Control Protocol
- Another technique which may be used in a publish subscribe system is forward-e ⁇ or co ⁇ ection. Forward-e ⁇ or co ⁇ ection sends multiple notification packets to the receiver in the hope that one of them will get through. This increases the probability that a packet will be received, at the cost of increased bandwidth use, but does not guarantee delivery.
- Reliable multicast is a third technique, it ensures that a set of multicast receivers receive the "same" set of packets from the multicast sender. This inherently does not work well with a CBR publish-subscribe system since different receivers will necessarily receive different sets of notifications depending on the different subscriptions.
- a node in a publish-subscribe network receives, via the network, a subscription to content (step 522).
- Content received from a publisher is published via the network, using CBR based upon the subscription, by receiving a notification concerning the content at a node in the network (step 524), determining if the notification is to be forwarded to a neighboring node (step 526), and selectively forwarding the notification to the neighboring node, based upon the determination, using a reliable transmission protocol (step 528).
- the reliable transmission protocol used in step 528 preferably is TCP.
- TCP is a communications protocol on the transport layer of the OSI model.
- the forwarding step 528 is done on a single node-to-single node basis, and if there is a transmission failure, the forwarding step 528 includes receiving an indication of a failure in the transmission of the notification (step 5281) and retransmitting the notification in response to the indication (step 5282).
- the request for such an indication of failure is preferably encapsulated in the packets of the notification.
- the notification packets are preferably kept in a buffer in case they need to be retransmitted.
- the packets may be dropped on a FIFO basis, or the packets may be stored in secondary storage.
- the packets may be selectively dropped from the buffer based on QoS (e.g. , those packets not governed by a QoS will be dropped first).
- the method may be repeated for next neighboring node if further routing is needed at the neighboring node (step 530).
- step 530 This illustrates that the method 520 achieves reliable notification delivery from publisher to subscriber in a hop- by-hop manner.
- CBR Content-based routing
- a publish- subscribe network enhanced with CBR functionality such as the publish-subscribe networks described above (e.g., see FIGS. 1-3)
- a subscriber's interest in publications of specific content i.e., a subscription
- the filters are preferably used by routers in the network to make routing decisions.
- the present CBR scheme capable of operating in the network core involves the propagation of these filters.
- CBR Cost-to-Value-to-Value
- the filter propagation methods described herein cover aspects of propagating content filters in any name and any form into, out of, and throughout the core of the network.
- the propagation of filters has many aspects and involves many issues.
- One important issue addressed by the filter propagation methods described is avoiding the buildup of large numbers of filters as the filters move "up" the propagation stream from a huge number of filter sources (e.g., subscriber) to a few filter destinations (e.g., core router, edge router or publisher). This issue is otherwise known as the "accumulation" problem, and it happens for example in networks composed of cascaded nodes connected in a tree-like topology.
- the solutions designed to address this and other issues may include one or more sets of rules, procedures, policies, processes, mechanisms, protocols, message formats, data structures, and/or algorithms.
- filter propagation can be made: fast (e.g., through simplification, merging, efficient marshalling, etc.); scalable (e.g., through merging, approximation, etc.); dynamic (e.g., new subscription, un- subscription, repair, etc.); secure (e.g., authentication); private (e.g., encryption); reliable (e.g., TCP connection, reliable tunnels, encoding, recovery protocol, etc.); flexible (e.g., lazy propagation (see below), various multicast topologies or protocols); and, adaptive to the need of users (e.g. , publisher/subscriber behavior-encoding).
- the filter propagation methods described herein utilize these solutions to achieve these benefits and other benefits.
- a publish-subscribe system can be implemented with a multicast, broadcast or mobile publish-subscribe strategy.
- Filter propagation may be embedded in the network topology or executed "on top" of or in addition to the network topology imposed by the strategy chosen for linking publishers and subscribers.
- the filter propagation strategy may also be the very strategy for linking publishers and subscribers, such as in a publish-subscribe scheme involving advertisements of publications where subscriptions embedded with filters can be delivered directly towards the advertising destinations.
- the filter propagation may use a protocol consistent with that used for multicast (e.g., protocol independent multicast - sparse mode (PEVISM) and - dense mode (PIMDM), Core-Based Tree (CBT), distance vector multicast routing protocol (DVMRP), multicast open shortest path first (MOSPF), etc.).
- a protocol consistent with that used for multicast e.g., protocol independent multicast - sparse mode (PEVISM) and - dense mode (PIMDM), Core-Based Tree (CBT), distance vector multicast routing protocol (DVMRP), multicast open shortest path first (MOSPF), etc.
- PVISM protocol independent multicast - sparse mode
- PIMDM - dense mode
- CBT Core-Based Tree
- DVMRP distance vector multicast routing protocol
- MOSPF multicast open shortest path first
- Strategies include one or more the following examples: 1) all filters are propagated to all possible upstream links when they are received (the eager propagation scheme); 2) filter propagation is initiated in response to incoming publishers' packets (the lazy propagation scheme); 3) propagation takes place in response to broadcast advertisements by publishers; 4) propagation takes place after computation of network topology (e.g., MOSPF); 5) filter propagation is periodic (for example, to satisfy the refreshing requirement in a system that supports filter time-outs); 6) filter re-propagation is required during the repair of a fault in the network; 7) filter re-propagation is required during the re-configuration of the network; and/or any combination of the above.
- the strategies are preferably subject to the filter propagating methods and rules described herein.
- the manner in which the source and the destination involved in a filter propagation process are connected may vary.
- the source and destination involved in a filter propagation process may be connected: 1) hop-by-hop (e.g. , router to router); 2) end-to-end (e.g., user machine to cache in an off-line data retrieval); 3) using tunnels; 4) using transmission control protocol (TCP); 5) using user datagram protocol (UDP) (e.g., supplanted with reliable data recovery mechanisms); 6) using a security protocol; 7) using other protocols; 8) taking place only within the kernel at routers and/or user machines; 9) involving only user-level processes at routers and/or user machines; 10) involving both the kernel and user-level processes at routers and/or user machines in all possible combinations (e.g., kernel-to-application, application-to-kernel-to- application, etc.); and/or, 11) any combination of the above.
- TCP transmission control protocol
- UDP user datagram protocol
- Destination of Filter Propagation Filters can be injected anywhere in the publish-subscribe network, as given in the following examples: 1) all routers and user machines forming end-to-end connections between publishers and subscribers; 2) every router in the network including the edge routers; 3) core routers only; 4) edge routers only; 5) subscriber machines; 6) publisher machines; 7) administrator machines; 8) cache; 9) backup routers; 10) any routers where packets may be diverted; 11) new routers involved in a network repair or fault recovery; and/or, 12) any combination of the above.
- the filters are preferably modified through approximation, merging and other processes. For each of the destination categories above, the form of filters being propagated may be different.
- a filter propagation source may be anywhere in the publish-subscribe network.
- the filters are preferably modified, through approximation, merging and other processes, during propagation, and so they often take on different forms from different sources.
- Filter propagation may also vary depending on the behavior and needs of potential recipients. For example: 1) publishers located at a permanent or semipermanent location want filters to be propagated towards them; 2) mobile publishers may not want filters to be propagated toward their location because they are moving away soon; 3) small-volume publishers who are only publishing a few publications before logging off do not need filters; 4) large- volume publishers may want filters to be propagated before sending the first sets of publications to save network bandwidth; 5) publishers with restricted access authorization may be denied filter propagation to protect the privacy and confidentiality of the subscribers in that application; 6) core routers implementing a filter time-out strategy need periodic updates from downstream routers; and/or, 7) edge routers that are not in constant connection with the subscriber machines may request periodic updates of subscription or filter interest. All these behavioral properties and others can be differentiated, for example, by using publisher advertisements, encoding publisher behavior in its first publication, periodic fransmission of filter updates, and other techniques to convey the need of the filter recipient.
- filter propagation may also vary depending on the behavior and needs of filter senders. For example: 1) a subscriber may only want to submit temporary filters with a (system-defined or user-defined) time-out; for example, if he/she is interested in sampling publications; 2) subscribers are mobile, and filters should expire after the subscribers have moved; 3) subscribers not in constant connection with their edge routers may be requested to send periodic filter updates in a CBR system supporting automatic filter time-outs; 4) some policies may dictate that subscribers need to be authenticated before propagating their filters; 5) routers in a CBR system that supports automatic filter time-outs would need to send periodic filter updates; 6) specific queries for only a fixed set of data may involve or may be implemented by filters (e.g., data from cache responding to a query or subscription, persistence backup data, data being recovered during a fault recovery, etc.); and/or, 7) routers that have modified filters for propagation may indicate to upstream routers how the propagated filters are modified based
- the processing of filters may involve transformations that modify the content or representation of the filters and or management functions that merely organize the filters.
- the original filter expressions in a subscription may undergo different sets of processing procedures in different parts of the data delivery path, which includes the following path sections: 1) from the subscriber machine to the edge router; 2) from the edge router to a core router; 3) between two core routers; 4) from a core router to the edge router on the publisher end; and, 5) from the edge router to the publisher machine.
- Some examples of these transformation processes are described above (e.g. , see Agent Processing and Wildcard Processing). These processes may be executed throughout the publish- subscribe network, e.g., on subscriber machines, publisher machines, edge routers or core routers.
- Components such as agents (on subscriber and publisher machines) and daemons (on routers) may execute these processes.
- DNF Disjunctive Normal Form
- 10) simplification to remove redundant predicate expressions e.g., $1 > 5 && $2 > 3 && $1 > 3
- 11) subsumption of two or more filters e.g., $1 > 5 && $2 > 3 && $1 > 3
- 11) subsumption of two or more filters e.g., 1 > 5 && $2 > 3 && $1 > 3
- 11 subsumption of two or more filters
- 12) approximation of a filter before and/or after merging 13
- merging of two or more distinct filters e.g., based on approximation
- 14) grouping of two or more distinct filters e.g., for propagation to different uplinks
- 15) un-subscription or removal of filters e.g., for propagation to different uplinks
- 16) expiration of filters e.g., 17) delta updating of filter (e.g., bookkeeping an old filter set in any representation and
- a message may be used to send or propagate a filter.
- the content and format of the message containing the filter being propagated may be affected by the transformation processes the filter has undergone and, during its propagation, the circumstances triggering the filter propagation.
- policies and procedure may be instituted to govern or some all aspects of filter propagation. These aspects may include, for example: 1) administering (e.g., global shutdown and restart); 2) maintenance (e.g., fault recovery); 3) filter persistency (e.g., expiration time-outs, one-time filtering of cached data); 4) performance enhancing policies (e.g., load balancing, edge selection, eager versus lazy propagation); 5) monitoring (e.g., fault reporting); 6) accounting (e.g., statistical bookkeeping); 7) billing; 8) regulating (e.g., authority to propagate filters, type of filters allowed in the network, etc.); 9) user authentication; 10) user assistance (e.g., directory service for subject filtering); 10) subscriber privacy (e.g., filters should be hidden from others); 11) synchronization (e.g., acknowledgment to subscribers when filters are received by the edge router, acknowledgment to subscribers when the filter propagation is completed,
- synchronization e.g., acknowledgment to subscribers when
- the method 450 illustrates an example of using a set of the solutions described in sections A-I above for propagating filters in a publish-subscribe network.
- the method 450 can be implemented, for example, in software modules as represented by an agent 128 for execution by a processor 134 in a subscriber machine 122.
- the method 450 can be executed by a processor 93 under software control in an intelligent router 92 or in the co ⁇ esponding functions contained in ASIC 91. Further, the method 450 can be executed by a combination of these.
- a plurality of filters is received (step 452).
- the filters may be injected anywhere in the publish-subscribe network.
- the received filters are processed to reduce the number of filters (step 454).
- the reducing step reduces the number of filters being propagated in order to avoid or reduce the filter accumulation problem. If too many filters are accumulated at one node in this publish-subscribe network, as in cases where there are a lot of immediate downlink nodes to the cu ⁇ ent node, or where the cu ⁇ ent node is at a very late stage of the filter propagation path, publishers and routers can be overwhelmed and network efficiency will suffer because of the huge computational load required to process and match all the filters.
- the following further illustrates the effect of the filter-reducing step 454.
- this reducing factor ensures that the number of filters at each destination of the filter propagation will never grow much faster than the number of stages in the propagation path, thus effectively solving the accumulation problem.
- FIG. 22B illustrates prefe ⁇ ed sub-steps of the reducing step.
- These sub-steps preferably include simplifying filters to remove redundant predicate expressions (step 4542), grouping filters coming only from downlinks relevant to a subject (step 4544), and within this group of filters, approximating filters (before or after merging) (step 4546), clustering filters according to their relative distances (step 4548), subsuming filters (step 4550), and merging of two or more filters (step 4552).
- These steps may be performed in a different order than shown, and some of these steps may be omitted. Simplification is described above in Agent Processing, with reference to FIG. 16.
- the filter grouping step 4544 is important for sending the co ⁇ ect filter information upstream. Some examples of its importance are given below.
- a router may have many downlinks (e.g., links to downstream routers). Not all of the downlinks have subscribers sending subscriptions or filters to them.
- each link to a router can serve as both an uplink and a downlink because publishers and subscribers are allowed to join the network from any arbitrary direction.
- this link When a filter request is received from a link, this link should be regarded as the cu ⁇ ent uplink, and only filters injected from those downlinks belonging to the same subject and apart from the cu ⁇ ent uplink should be grouped for subsequent approximation, clustering, subsumption and merging operations.
- Grouping 4544 may also be needed for other purposes, such as accounting and network traffic confrol. It may even be omitted altogether in some situations, for example, when the global map of filters coming from all the links is required for any reason.
- Approximation modifies filters to capture more, rather than fewer, publications. For example, approximation may reduce the number of predicates in the representation of the filter being propagated, reduce the resolution of the filter coordinates, or truncate a long string to the first 12 characters. Approximation has the effect of aligning the formats of the filters so that they may become better candidates for subsumption and merging (steps 4550, 4552). The approximation step also reduces the size of the message propagating the filter.
- step 4550 eliminates filters that are a sub-set of another filter. For example, if a first filter is for all condos with an area greater than or equal to 1000 ft 2 and less than or equal to 1300 ft 2 and a price range of greater than or equal $200K and less than or equal to $300K, and a second filter is for all condos with an area greater than or equal to 1100 ft 2 and less than or equal to 1200 ft 2 and a price range of greater than or equal $225K and less than or equal to $275K, the second filter is a subset of the first filter and, therefore, is eliminated by subsumption.
- Steps 4542, 4544, 4546, 4548, 4550, and 4552 may be performed in any order, depending on the specific filter reduction scheme and/or algorithm chosen. For example, that steps 4546 and 4550 maybe interchanged. Approximation may take place after subsumption. Further, steps 4550 and 4552 may be integrated and performed as a single step. For example, merging could include subsumption, as shown in the graphs below.
- merging is similar to subsumption in that it combines two or more filters. Merging, however, may use approximation to combine two or more filters that are partially or completely distinct. Viewing various filters geometrically illustrates this process. For example, see Graphs 1-4 below:
- Graph 1 geometrically illustrates two filters, si and s2, that may be reduced through subsumption, since filter s2 is a sub-set of filter si, as described above. In graph 2, however, neither filter s3 or filter s4 is a sub-set of the other. Rather, filters s3 and s4 partially overlap.
- the merging step 4552 merges s3 and s4 by approximating the minimum filter sA that encompasses both s3 and s4. Since s3 and s4 are both sub-sets of sA, s3 and s4 are subsumed into sA.
- Graph 3 illustrates another example of merging; in graph 3, filters s5 and s6 are completely distinct.
- the merging step 4552 approximates a filter sA that includes s5, s6 and the area between these filters. Filters s5 and s6 are then subsumed into sA.
- Graph 4 illusfrates yet another example of merging in which the filters s7 and s8 are only partially distinct for one attribute. Since s7 and s8 overlap for one attribute and have the same value for the other attribute, the merging step 4552 simply merges the two filers together into sA, subsuming the overlapping portion.
- subsumption (step 4550) and merging (4552) depend on locating filters close to one another.
- clustering algorithm may be used as an optional step (step 4548) to collect filters with boundaries close to or overlapping one another.
- the clustering algorithm can be performed incrementally in real time as filters are added or removed.
- Fixed-size or variable-size grids of any granularity over the attribute space of filter attributes can also be used to group filters based on their respective grid coordinates.
- the portions of sA that are not covered by either s3 or s4 or s5 and s6 are refe ⁇ ed to as the leakage area (or relaxation area, redundant area, etc.), in that these portions will capture publications that are not requested by s3 or s4 or by s5 or s6.
- the merging step 4552 therefore, sacrifices or trades leakage for filter reduction. However, as greater approximations are made, the leakage area may reach a point at which it is too large to be offset by the benefits of filter reduction. Therefore, the publish-subscribe network in which the method 450 is being executed preferably has a defined leakage area threshold that is used in the merging step 4552 to determine whether the leakage area for each approximation is too large.
- the merging step 4552 preferably performs a statistical analysis on the leakage area to determine what anticipated percentage of additional packets will be received due to the approximation (i.e., packets falling into the leakage area). This statistical analysis may be based on packet distribution profiles built using a history of incoming packets or that of outgoing packets (e.g. , at the router performing the method 450). If the anticipated percentage exceeds the leakage area threshold, the approximation is not used.
- the leakage threshold is typically a percentage figure, and more than one threshold can be used as criteria for merging.
- leakage threshold #1 is specified as 10% of the number of packets that would be accepted by the actual filters s5 and s6 involved in the merging step 4552, and if it is estimated that the leakage area might contain 20% of the total packets falling in filters s5 and s6, the merging operation may be disallowed.
- leakage threshold #2 is 2% of the total number of packets passing the router, and if it is estimated that the gap between filters s5 and s6 might constitute 1% of the total number of packets passing the router, the merging operation may be permitted even though the first leakage threshold (#1) is exceeded, because the overall leakage is still a small percentage of the packets that would consume the bandwidth downstream if they are routed without filtering.
- a large number and variety of interplays between various leakage thresholds can be designed and specified.
- the leakage area threshold may be dependent on the number of filters merged, the area of the filter coverage, or any characteristics of the filter distribution. For example, when the total number of filters is small, a tighter constraint may be used for merging filters, as the accumulation problem is not imminent.
- a merged filter sA that is composed of two or more distinct filters, may have zero leakage area in certain situations and is therefore exact. For example, assuming two-dimensional filters (i.e., s7 and s8 in Graph 4), when the two rectangles representing the filters are aligned with and contiguous to each other, leaving no gap between them, the merging step 4552 resulting in filter sA will be exact. In general, in multiple dimensions, this situation could happen if two or more filters have overlapping or intersecting predicated ranges for one attribute and the same predicated range for any other attribute.
- the method 450 may alternatively include additional filter processing (in addition to that in step 454), such as the filter processing described in section G above. As shown, the method 450 determines whether to propagate the filters based on recipient behavioral rules (step 456), examples of which are given in section E above. The determining step 456 preferably checks whether filters (and/or what filters) are to be propagated to the potential recipient.
- the potential recipient may be a mobile publisher that has sent a request not to propagate filters to the node (e.g., edge router) to which it is attached.
- the potential recipient may be a new publisher that has sent a request to propagate all filters to the node to which it is attached before it will start publishing a huge amount of content.
- Other potential recipient behavioral rules such as those listed in section E, may be used.
- the order of steps 458 and 456 is interchangeable. Additionally, some steps such as step 454 (specifically, the merging substep 4552) and step 456 are optional, for example, for a limited number of subscriptions (i.e., filters) or a network composing only a few stages of propagation because filter accumulation is not yet a serious problem.
- step 458 examples of which are given in section F above, may also be made.
- the filter sender may have included a filter time-out.
- the determining step 458 checks the filter to see if the filter has expired.
- the filter sender may need to be authenticated for the filter to be propagated to the next node.
- the determining step 458 may reject the filter if its origin cannot be authenticated.
- Other filter sender behavior rules such as those listed in section F, may be used.
- this filter propagation procedure 450 may be initiated from anywhere in the publish-subscribe network, including the network administrator.
- the full set or the reduced number of filters is propagated to the next node (step 460).
- the next node (the filter recipient) may be anywhere in the publish-subscribe network.
- the number of filters may be reduced by each of steps 454 to 458. Indeed, the number of filters could be reduced to zero by steps 456 or 458 (e.g., based on a potential recipient rule that says do not propagate any filters).
- the method 450 may also propagate 460 based on one of the filter propagation strategies for the publish-subscribe network, as described in section A above.
- the propagation step may include determining filter propagation sfrategy and propagating filters based on satisfaction of a requirement(s) of the determined filter propagation strategy.
- propagation 460 may take place upon receipt of filters or may only take place in response to broadcast advertisements by publishers.
- the method 450 may be executed according to any of the policies and procedures listed in section I above.
- the method 450 may also format the filter propagation message according to section H above. The manner in which the source and the destination involved in a propagation step 460 are connected may vary, as described in section B above.
- the method 450 may determine if there is another potential recipient (another node) (step 462). If there is another potential recipient, the method 450 determines whether there is any need to repeat the procedure for this potential recipient.
- the filter set e.g., filter addition, filter removal, etc.
- filter propagation becomes unnecessary for that link.
- the method 450 may determine whether to propagate based on the behavioral rules, steps 456 and 458. The method 450 may repeat until there are no further potential recipients of reduced filters.
- FIG. 22C illustrates another filter propagation method 470.
- This method illustrates another example of using a set of the solutions described in sections A-I above for propagating filters in a publish-subscribe network.
- the method 470 can be executed by a processor 93 under software control in an intelligent router 92 or in the co ⁇ esponding functions contained in ASIC 91.
- the method 470 can be implemented, for example, in software modules as represented by an agent 128 or 105 for execution by a processor 134 or 114 in a subscriber or publisher machine 122 or 100.
- the network topology is a bi-directional topology with a series of publishers P attached from any arbitrary direction and via any arbifrary path to a network of intelligent routers C, as seen in FIG. 22D.
- a network may be implemented for example by a Core-Based Tree multicast scheme, hi the example, a new publisher, P N , and a new router C N , are attached to the network at a router C.
- the new publisher P N in its effort to connect to a particular publish-subscribe network may need to connect to the core through a series of new routers that will form an extension of that network and or may connect through routers that are already part of that network. For illustrative purposes, only 1 new router C N is shown in Figure 22D.
- any link can be an uplink or downlink at the same time.
- the uplink (only one) is defined as the link from which the cu ⁇ ent filter request is received and a downlink (among many) the link to an immediately downstream recipient of the filter request away from the uplink.
- Upstream and downstream are directions defined in the same sense as uplink and downlink.
- the new publisher P N broadcasts or multicasts to the downstream router C (or all downstream nodes) a notification requesting filters (step 472).
- the router C updates its bookkeeping to ensure that the same set of filters will not be sent twice to the same uplink, preferably by means of reference counting (step 476).
- the router C determines whether filter request from this uplink has been serviced before, preferably by means of examining the reference counter for this uplink (step 478).
- the router C will relay the request to (step 480) and await a response from the router's downlinks (i.e., downstream routers) (step 482). Those downstream routers in turn transmit the request to their downstream routers and await their response. This process is repeated until the receiving node could make a decision to respond. This completes the first stage.
- the second stage begins when any node that receives a filter request has collected the response from all of that node's downstream routers or when that node is the final node in the network and no response is expected from downsfream (step 484). If there are no filters to send, a negative acknowledgment (or a null filter set in any representation) will be sent upstream. If all the new filters have been received from all the node's downstream routers, the entire or partial set of filters (e.g., delta changes) will be sent upstream (step 486) after optional filter reduction (step 454). Only filters collected from the downlinks with respect to the cu ⁇ ent uplink will be grouped, merged and reduced, according to method 450.
- This collect-and-then-propagate process is then repeated upstream until the publisher or the initial router sending the filter request receives the filters or the negative acknowledgments from all that publisher's or initial router's downstream routers.
- the actual propagation of filters or a null set (negative acknowledgment) as described above constitutes the second stage.
- the two stages differ in their propagation direction as well as the propagated content, and they can overlap in time as the filter request could go further down some branches of the network than others.
- this filter propagation scheme only initiates the filter propagation process upon reception of a filter request from an uplink, rather than eagerly pumping filters to every uplink, it may be known as a lazy propagation scheme as it pumps filters to only the upstream router requesting filters and only when it requests them. It is noted here that the filter request may not need to be a special-formatted notification.
- the first notification from any publisher can be regarded as an advertisement of its presence, and the filter recipient behavioral rules encoded and embedded in the data notification can be used to determine if filters are being requested, according to Section E.
- Each publisher P's filters may be stored at the publisher user machine 100 or at the router C to which the publisher P is connected.
- the method 470 may also use additional solutions such as those described in sections A-I above.
- CBR The embodiment of CBR described in this section involves a method for routing data packets in the core of a publish/subscribe network, based on deep packet inspection and filter matching.
- this CBR method forms a map from a finite set of attribute spatial coordinates to routing instructions off-line and then utilizes the map in real time to make routing decisions.
- This CBR method is very fast, achieving a data-forwarding rate of the same order of magnitude as that of traditional Internet Protocol (IP) routing. Consequently, this CBR method is a suitable candidate for pragmatic commercial deployment of content-based routing in the core of a network.
- IP Internet Protocol
- subscribers refine the selectivity of packets transmitted from one or many publishers in a publish-subscribe network by specifying predicates over a range of attribute values.
- Each predicate is an attribute test (attribute tests may include a subject test as well) that relates an attribute in a certain position in the packet to 1 or 2 operands (an operand may be a complex object such as a regular expression) in a relational or membership test.
- 10 ⁇ $l ⁇ 20 is a test to examine whether the data value in the first attribute position denoted by $1 is greater than 10 and less than 20 (the definition of a predicate can be simplified by limiting the test to involve only 1 operand, e.g., a 2-sided range test can be broken into a conjunction of two one-sided tests).
- the subscriber submits a collection of these tests in one or a plurality of subscriptions, each of which may include the subject field of the subscriber's interest.
- filters are objects derived from subscriptions and differ from subscriptions only in their representations.
- filters are objects derived from subscriptions and differ from subscriptions only in their representations.
- One example of filter is the conjunctive filter described below.
- a subscription is the original filter, one that is specified, used for filtering packets and/or sent to the network directly by the subscribers.
- subscriptions may be converted into objects of other formats called filters.
- each subject is implemented using a multicast network. Filters that are of the same subject and injected to each node of a multicast network for a particular subject will filter packets only with the same subject field, in effect performing subject field filtering implicitly. Filters can therefore be stored under their subject headings at every node in this network without their subject fields. Filters without subject fields are often known as attribute filters.
- the terms subscription, filter and attribute filter may be used interchangeably and its meaning made clear only by their context.
- each subscription can be processed and broken down by the proxy into its equivalent Disjunctive Normal Form (DNF), as described above, before being sent to the edge router.
- DNF Disjunctive Normal Form
- a DNF is a disjunction of one or more sub-filters called conjunctive filters.
- attribute filters are all conjunctive filters because of the proxy action.
- conjunctive filters have the advantageous property of a uniform format, which facilitates the propagation of these filters in the core of the network, the single occu ⁇ ence of each attribute in the filter expression also permits the interpretation of these filters as multi-dimensional geometrical objects, which is useful for describing the embodiment of CBR herein.
- conjunctive filters are used as examples.
- the benefits of the downstream bandwidth gain obtained by using CBR may be easily out-weighed by the huge amount of computation required if a naive method is used to match a large collection of filters one by one and predicate by predicate.
- This huge amount of computation in the critical data-forwarding path can be a severe drag on publish-subscribe network efficiency.
- the embodiment of CBR described herein seeks to reduce this huge amount of computation and to move the maj ority of computations out of the critical data-forwarding path.
- the embodiment described herein provides a very efficient method for CBR by approximating filter expressions into more compact forms and pre-computing the filter coverage information off-line.
- the filter coverage over every single point in the attribute space is predetermined off-line when filters are inserted or removed.
- the co ⁇ ect action on the packets with attribute values falling on that point will be the union of the actions predicated by all the filters covering that point.
- These actions may involve the packet, including for example, packet forwarding to a physical or an encapsulated interface, packet diversion to a user-level process, or packet piping to a parallel machine (e.g., cache, backup router, etc.).
- Some actions may not involve the packet at all, including for example, triggering a general application running on the router to respond to the arrival of a specially designated packet, prolonging the lifetimes of some filters because there are still active publications in the relevant area of the filters, automatically injecting new filters based on the content of the publications, etc.
- the actions predicated by each filter can be embodied by a data structure, such as an action bit mask (ABM) with each bit position representing a distinct (forwarding) action, or a list of distinct actions (Distinct Action List - DAL) when the number of actions becomes too unwieldy to be placed in a bit mask (e.g. , an edge router connecting a huge number of subscribers).
- ABS action bit mask
- DAL Distinct Action List - DAL
- the pre-computation thus generates a CBR map from a number of coordinates in the attribute space to the same number of unions or lists of actions.
- this simple map is similar to the Multicast Routing Table used in subject group multicast, it can be known as the CBR Table.
- This CBR method also preferably includes approximating the filter expressions to more compact forms, for example by means of spatial quantization and predicate reduction. Because there are a huge number of coordinates in each attribute dimension, the attribute space must be quantized, forming a grid. Only the quantized regions instead of individual coordinates are used to construct the CBR table. Because spatial quantization has the effect of grouping individual points in the real attribute space into a single point in the quantized attribute space, every point in the original real attribute space is still indirectly accounted for in the CBR table, sharing with its neighboring points with the same quantized coordinates the list of actions predicated for those quantized coordinates.
- the method of CBR described herein preferably reduces the area of the attribute space that is quantized. Specifically, preferably only that area of the attribute space for which matching data packets are expected to be received is quantized. As described below, this area for which matching data will possibly be received is determined based on information received from publishers and/or on a statistical analysis of past (i.e., previously received) packet and/or filter distribution. For example, if a publisher provides information that matching data packets for above a certain value of an attribute will never be published, the attribute space above that certain value preferably is not quantized. Likewise, if statistical analysis shows that filters for below a certain value of an attribute have never been received, the attribute space below that certain value preferably is not quantized.
- FIG. 23 A illusfrates a method 480 of CBR by approximating filter expressions into more compact forms and pre-computing the filter coverage information off-line, according to the embodiment described herein.
- This method of CBR minimizes real time predicate-by-predicate filter matching.
- Method 480 can be executed by processor 93 under software control in intelligent router 92 or in the co ⁇ esponding functions contained in ASIC 91.
- the pre-computation step which involves filter management functions, is computationally intensive, it is preferably done in software.
- filter representations such as the radix representation, it is also possible to maintain the filters in hardware.
- Method 480 can be implemented, for example, in software modules, e.g., as represented by agent 106 for execution by processor 114 in user machine 100.
- the method 480 first determines whether it is time to perform spatial (re-)quantization (step 482), which can be triggered by a periodic or aperiodic time-out, or by a pre-defined event such as the observation that the packet statistics has become overly uneven across the cu ⁇ ent quantized attribute space.
- Spatial quantization is performed off-line if needed and when the router is relatively idle (step 484). Spatial quantizing involves the computation of a new grid that will preferably have the property of rendering either the packet distribution or the filter distribution (or some co ⁇ elation of the two) uniform across the resulting quantized attribute space. Other methods and criteria for spatial quantization can be used.
- the grid area may be constructed to only cover a reduced area of the attribute space in which matching packets are expected to be received. For example, if the first attribute $1 represents the number of stocks in a certain industrial sector in the US, the spatial quantization needs only take place in the range from 0 to a number comparable to the total number of stocks in all the US stock exchanges combined. After the statistical distribution of this attribute can be formed from the packet statistics collected, adaptive quantization, as discussed below, can be used to dynamically adjust this range, initially to a much smaller number that is the maximum of the number of stocks for all sectors, and in the future to higher numbers when the maximum grows.
- the router determines if any filters have been inserted at or removed from (step 486) the network location (e.g., a core or edge intelligent router 92, or a publisher machine 100) at which the method 480 is being executed.
- a reference-count method can keep track of whether a newly injected filter has the identical information (e.g., attributes, matching attribute values, and filter actions) as another filter previously injected (step 488, step 494). Only distinct filters need to be processed.
- the location is assumed to be an intelligent router 92. For example, a new filter may have been propagated upstream to the intelligent router 92 or an old filter may have timed-out due to an expiration time-out set for the old filter.
- the filter is first compared to the other filters already received and stored by the router using its real coordinates, for the purpose of (1) updating the reference count for the co ⁇ ect filter in the real attribute space (step 488), and (2) determining if the filter to be inserted/removed is a distinct filter or a filter already received before (step 490). If the filter is not a distinct filter, the execution flow will be passed back to the packet matching routine.
- the attribute values of the packet are mapped to the quantized space to obtain the quantized coordinates of the attribute values (step 492).
- a similar filter identification procedure is performed, this time using the quantized coordinates, for (1) updating the reference count for the co ⁇ ect filter in the quantized attribute space (step 494), and (2) determining if this filter is a distinct filter in the filter's quantized coordinates (step 496).
- two maps of filter coordinates to filter references may preferably be used.
- the filter can then be approximated and reduced (step 498). For example, only a sub-set of the attributes of a filter with too many predicates may be selected for quantization and mapping. Likewise, attributes with predicates of a more complicated nature (e.g. , predicates requiring string matching, regular expression, in-string, in-a ⁇ ay, range operators, etc.) or requiring a higher resolution may be handled separately.
- the compacting of filters may involve both the mapping of filters to a quantized attribute space (step 492) and the approximation and reduction of the filter expressions (e.g., step 498).
- the compacting step includes spatially quantizing the attribute space (step 484). This step forms a grid in the attribute space, as described above and shown below:
- the CBR table can now be updated (step 500).
- Each filter is associated with a rule indicating one or more (routing) actions for publications matching that filter.
- the (routing) actions predicated by the filter are inserted into the grid cells in this CBR table in the following manner.
- step 500 Using the grid formed in the attribute space of the filter coverage information, step 500 first identifies all the grid cells (i.e., quantized regions) that the quantized coordinates of the filter cover or overlap. Then step 500 inserts into these grid cells the predicated (routing) actions. For example, assuming five possible destination nodes, two filters requiring routing to user machines connected to nodes 2 and 3, respectively may cover the grid cell or quantized region X (see above).
- step 500 inserts or affirms the rules respectively requiring routing to nodes 2 and 3 to grid cell or quantized region X.
- This process is iterated for each of the grid cells identified to be covered or overlapped by the quantized coordintes of the filter.
- step 500 represents the mapping by generating or updating an ABM for each grid cell or quantized region.
- the ABM for grid cell or quantized region X assuming the nodes are numbered 0-4, is:
- mapping step 500 may include a bookkeeping step as described herein.
- a CBR table is constructed/updated (step 500) from the ABMs generated for each grid cell or quantized region. This preferably involves, directly or indirectly, both spatial requantization and CBR table updating, and all the supplementary steps 482-500.
- Steps 482-500 are preferably executed off-line (i.e., when the intelligent router 92 is not receiving published packets) or in the background of the real-time routing process as filters are injected, suspended, paused, or removed. Accordingly, if no published packets are received, or if steps 482-500 are being executed in the background, method 480 again determines first if it is time to re-quantize the attribute space (step 482) and subsequently if any filters have been inserted or removed (step 486). The CBR table is then used in the real-time routing process in the data-forwarding path, as shown in FIG 23B.
- the size of the CBR table can be massive for a large number of attributes or a large number of grid divisions for each attribute. In principle, only those grid cells in the CBR table that contain action rules (i.e., those which are covered by filters), need to be stored.
- the CBR table can be condensed using any technique. Whereas FIG. 23 A illustrates the CBR management aspects of method 480, FIG. 23B illustrates the CBR data forwarding aspects of method 480. If method 480 determines that published packets are received (step 502), the intelligent router 92 receives the published packets (step 504), as seen in FIG. 23B.
- the received packets can be injected into the network using a point-to-point connection (e.g., wireless connection to an edge router).
- Step 506 determines the quantized coordinates of the packet so that the co ⁇ ect grid cell of the CBR table can be looked up and the action rules contained in that grid cell can be extracted and used to handle the packet (508). These actions may include determining which nodes or other locations in the network to route the packet, or dropping the packet.
- the CBR method 480 presented so far may incur e ⁇ or in the form of packet leakage as a result of filter approximation (see step 492). Because the boundaries of the grid cells, instead of the original filter boundaries, are used for matching packets, the actual regions (e.g., area in 2 dimensions and volume in 3 dimensions) used for matching are enlarged, and unwanted packets whose attribute values fall between the actual filter boundary and the grid cell boundary, herein known as the relaxation region, will e ⁇ oneously be accepted. As described above, the reduced filter resolution is deemed an acceptable sacrifice when speed performance and scalability are acquired or improved in the process and when the leakage e ⁇ or is small in comparison with the total number of incoming packets.
- the leakage e ⁇ or in some cases may not be small.
- the filter resembles a point.
- the expanse (e.g., area, volume, hyper-volume) of the relaxation region can be much larger than that of the actual filter itself.
- Such an undesirable property of the CBR method 480 can be mitigated by various techniques that may be performed in the leakage reduction step 516, one of which is described below as an example.
- ID exact matching using any ID search technique is preferably conducted for all the attributes.
- An exemplary leakage reduction step 516 according to ID exact matching is described below: a.
- Step 508 Perform CBR matching step 508 without leakage reduction, as described above (if there is no match, drop the packet); b. For packets that match (see step 508 above), select one attribute dimension and match that attribute value of the packet with the projection of all the filter boundaries on that attribute axis, using any ID exact matching technique and its co ⁇ esponding data structure. A list of all the forwarding nodes for which a matched filter is found is obtained; c. If there is no ID match, the packet can be dropped because it is guaranteed that no filter will match the packet on all its attribute tests; d.
- step b If there is a match, pick another attribute dimension and repeat step b until (i) a mismatch is found in any dimension; (ii) the lists of forwarding nodes produced by all the previously performed ID searches do not contain common interfaces (i.e., the ID matches are generated from filters from different interfaces); or, (iii) the exact matching test is done for all the attribute dimensions, which may include those disabled for CBR; e. If any one of the dropping condition described in step d does not happen, and all the attributes are tested without a mismatch, a decision not to drop the packet can be made. f. In cases where more than one interface is involved, the interfaces for which the packet is not dropped must be compared with the list of interfaces obtained from step a.
- sub-step a may be performed after the ID leakage reduction step (sub-steps b-e) is completed. Some sub-steps may be integrated.
- the data-forwarding path traffic through the intelligent router 92, at which method 480 is performed, is checked to determine if it is above a set threshold (step 510) (i.e., the intelligent router 92 is "busy," e.g., being occupied with the critical task of forwarding packets). If the intelligent router 92 is busy, the packets that map to covered grid cells are routed (step 508) based on the routing actions determined by step 508 (step 512), and the co ⁇ esponding packet statistics is updated (step 514). Consequently, if the intelligent router 92 is busy, "leakage" resulting from routing packets that map to a partially covered grid cell, but that fall outside the actual filter coverage in that grid cell, is accepted as a sacrifice to improve routing speed.
- step 510 determines that the intelligent router 92 is not busy (i.e., the traffic is not above the threshold)
- leakage reduction is preferably performed (step 503).
- the leakage reduction step 516 reduces the leakage described in the preceding paragraph.
- the leakage reduction may be performed, for example, according to the below-described methods.
- Attribute tests requiring a higher resolution than provided by the CBR map can be processed after the filter bucket is retrieved from the first grid-matching step of CBR (step 508).
- the attributes can be separated into two groups, one group entered into the CBR map and another group in the filter bucket.
- the filter predicates for the attributes described in 1-3 may be reduced by step 498 above. Additionally, if the intelligent router 92 is not busy (see step 520), a predicate- by-predicate or other exact filter matching is used to match such packets to the filter bucket determined by the partially covered grid cells, as ascertained from the CBR table. Since the filter bucket for that grid cell or quantized region is aheady reduced (from the total number of filters) by the pre-computation (in steps 482-500), this process is still more efficient that simply doing straight predicate-by-predicate CBR.
- the method determines if there are any reduced filter predicates or packets mapped to partially covered grid cells (step 522) for the filter bucket determined by the mapped-to grid cell. If there are such reduced filter predicates (e.g., from examples 1-3 above) or such mapped packets, predicate-by-predicate or other exact filter matching is performed (step 524). The matched packets are then routed based on the routing actions determined from the CBR table and the exact matching, for example, by intersecting the interface lists obtained by the two methods (step 512) and the packet statistics is updated (514).
- reduced filter predicates e.g., from examples 1-3 above
- predicate-by-predicate or other exact filter matching is performed (step 524).
- the matched packets are then routed based on the routing actions determined from the CBR table and the exact matching, for example, by intersecting the interface lists obtained by the two methods (step 512) and the packet statistics is updated (514).
- the fraffic threshold checking step 510 may precede only some of the computationally intensive steps, such as steps 518 and 524, as determined by a balancing of waste versus computational intensity. Further, the method 480 may have separate paths for packets that map to completely covered grid cells and for packets that map to partially covered grid cells. The traffic threshold step 510 may then be only performed in the partially covered grid cell path.
- the leakage reduction step 516 in method 480 may be replaced with a more advanced leakage reduction procedure, described herein.
- the advanced leakage reduction procedure further reduces leakage e ⁇ or.
- exact multi-dimensional matching can be performed within the grid cell by cross-producting after the grid table search.
- Cross-producting involves the following procedure: a. Perform ID exact-matching as described above, using any ID exact-matching technique.
- the main difference in this alternative step is that a list of filter labels instead of a list of forwarding nodes is produced; b. If the list is empty (i.e., no matches), the packet is dropped; c. If there is at least one match, the search proceeds to another attribute.
- Step ac are repeated until any one of the short-circuiting conditions below is met: (i) a mismatch is found in any dimension; (ii) there is no common filter label in all the filter label lists produced by the previous ID tests; or, (iii) all the dimensions have been tested; and d. If any one of the dropping condition described in step c does not happen, and all the attributes are tested without a mismatch, a decision to accept the packet can be made.
- a filter with a long list of predicates will be covered by at least one such grid table; or
- the cross-producting solution is exact, hi the case of range filters, however, there are traditionally two problems with cross-producting as a list of filter labels need to be stored between any pair of neighboring filter boundaries.
- the storage is of order N 2 , where N is the number of filters, when the system is injected with range filters that highly overlap each other.
- the list-intersection step is of order N, which can be a very slow process when N is large.
- the hybridized approach combining the grid search and the cross-producting method reduces both problems to a reasonable size because only those filter boundaries that fall into the same grid cells need to be stored and matched for packets falling into the boundaries of that grid cell.
- the grid cells have taken over the role of storing filter label lists from the intervals between neighboring filter boundaries, and since there are fewer grid cells, filter label lists need to be stored.
- the cross-producting method as described above works as long as the number of grid cells remain practically a small number.
- the method of dividing attributes into various permutation sets and assigning one CBR tables for each set limits the number of grid cells in each table to a reasonable number. For example, if E is the number of divisions in each attribute axis, the total number of grid cells for k number of attributes would be E .
- the size is now reduced to M L 4 , where Mis smaller than the total number of permutations k!/(k-4)!4l. It is noted that some permutations highly overlap one another and only one can be chosen as a representative, (e.g., ⁇ $1, $2, $3 ⁇ , ⁇ $1, $2, $4 ⁇ , ⁇ $1, $2, $5 ⁇ ).
- a CBR method according to the present embodiment (e.g., method 480) (minus the exact predicate-by-predicate filter matching steps 516 and 524) has at least the following properties: 1) the packet forwarding speed is very fast and comparable to IP routing, requiring only Ofk log(L)] steps, where E is the number of divisions in any attribute axis and k the number of attributes being quantized and mapped. With 4 attributes and 32 divisions per attribute, about 22 steps are required; 2) the packet forwarding speed is a constant in real time. It is not dependent on the number of filters (N). This property makes it possible to estimate the maximum bandwidth capacity sustainable by this CBR method; 3) it is fast for dynamic updating as well.
- insertion of a filter may take 0(L k ) steps, depending on the implementation; 4) the storage requirement is small and limited, being approximately k (L-l) S A + N r L k / 8, where S A is the size of the attribute data type in bytes and N r the number of forwarding interface at the router.
- the first term is the total size of all k number of the indexing components of the data structure and the latter the actual forwarding component (i.e., the CBR table) being indexed.
- the amount of storage is only 1 MB; 5) the indexing data structure, being separated from the much larger forwarding component, is very amenable to caching.
- the CBR method 480 is fast, predictable, dynamic, scalable, not memory demanding, flexible, and hence practical for deployment in the core of a publish-subscribe network.
- the grid is formed by spatially quantizing the attribute space.
- the attribute space is uniformly quantized.
- Adaptive quantizing determines region(s) of the attribute space for which the most matching packets are likely to be received. For example, adaptive quantizing may determine the region(s) of the attribute space in which the past packet distribution is most concentrated. The determined region or regions are then quantized to a greater degree than the other quantized regions of the attribute space. In other words, in the determined regions, the number of grid cells will be greater and the size of the grid cells will be smaller than in the other quatized regions of the attribute space.
- the 2 diagrams below of an adaptive grid illustrate this:
- the grid spacing is changed on either attribute axis so that more grid cells will be finer and more concentrated in the area encompassed for example by quantized coordinates (3, 2), (4, 2), (3, 3) and (4, 3).
- the region Z has been adaptively quantized, hi the region Z, where there would be four (4) grid cells without adaptive quantizing, there are now sixty-four (64) grid cells.
- the filter coverage information sS previously would have partially covered three grid cells, with the adaptive grid there is only one (1) partially covered grid cell Y. Consequently, the amount of potential leakage is significantly reduced. Further, if one or more of the sixty-four grid cells in the region Z were to be partially covered, the amount of potential leakage in each of these grid cells would be significantly less than in the partially covered grid cell Y.
- the second method has the advantage of not increasing the storage demand as the first method does because the same number of grid cells is generated. However, it also provides less accuracy as the former method.
- Described herein is a family of methods for using publish-subscribe networking to support query-response style interaction.
- the methods involve different mappings from the primitive absfractions of the publish-subscribe style, namely publications (also called notifications) and subscriptions, to those of the query-response style, namely advertisements (also called offers), queries and responses. Additional variations distinguish between the use of singleton advertisements and queries (which characterize a single content item) and set advertisements and queries (which characterize a plurality of content items) .
- publishers create and distribute content items called publications or notifications.
- this information is structured as a tuple of typed attributes, and in some systems a distinguished attribute called a subject is used as a primary classifier for the notification.
- Subscribers create and distribute subscriptions, which characterize the notifications in which they are interested.
- a notification is typically a singleton content item, while a subscription typically uses a complex expression to characterize a set of notifications of interest.
- the same entity can act as both publisher and subscriber.
- Interaction takes place by having publishers push content items to subscribers. This push delivery respects a certain temporal order among subscriptions and notifications; in particular, a notification is matched only against subscriptions that are active at the time of the notification but not against future subscriptions.
- the embodiments described herein are a family of methods for using the push- style interactions of publish-subscribe to support the pull-style interactions of query- response. The following describes five exemplary methods to achieve this.
- FIG. 24A is a flowchart illustrating a first query-response method 600 using a publish-subscribe system.
- Method 600 may be executed using a variety of publish-subscribe systems.
- method 600 may be executed by processor 93 under software control in intelligent router 92 (or in the co ⁇ esponding functions contained in ASIC 91) and implemented in software modules, e.g., as represented by agent 128 and 105, for execution by processor 134 and 114 in user machine 122 and 100.
- a publish-subscribe network is used to match advertisements to queries, whereby an advertiser uses a subscription to register a singleton or set advertisement (step 602), and a requester uses a publication/notification to distribute a singleton query (step 604).
- a singleton or set advertisement is mapped to a subscription and a singleton query is mapped to a notification.
- Any matching notification is pushed to the advertiser by the publish-subscribe system performing CBR using the subscription (the advertisement) as a filter for the notification (the query) (step 606).
- the CBR may be performed according to any of the above-described methods using the above-described infrastructure.
- the advertiser receives the notification and returns responses to the requester (step 608) via some non-publish-subscribe means (e.g., a unicast connection to the requester) or via an agreed-upon convention using the publish-subscribe network (e.g. , requiring the requester to register a separate subscription (co ⁇ esponding to the requester's notification) in order to receive the responses, which are delivered as notifications).
- some non-publish-subscribe means e.g., a unicast connection to the requester
- an agreed-upon convention using the publish-subscribe network e.g. , requiring the requester to register a separate subscription (co ⁇ esponding to the requester's notification) in order to receive the responses, which are delivered as notifications).
- FIG. 24B is a sequence diagram further illustrating the method 600.
- FIG. 24B depicts the interaction between the advertiser (e.g., using publisher machine 100), the CBR infrastructure (e.g., network of intelligent routers 92), and the requester (e.g., using subscriber machine 122).
- the sequence diagram illustrates that multiple processors (e.g., processor 93, processor 114, and processor 132) preferably execute method 600.
- FIG. 24C is a flowchart illustrating a second query-response method 610 using a publish-subscribe system. Method 610 may be executed using a variety of publish-subscribe systems.
- method 610 may be executed by processor 93 under software control in intelligent router 92 (or in the co ⁇ esponding functions contained in ASIC 91) and implemented in software modules, e.g., as represented by agent 128 and 105, for execution by processor 134 and 114 in user machine 122 and 100.
- producers distribute content items in duplicate using related pairs of subjects.
- a producer publishes notifications using one subject of a pair, called the initial subject, for the initial advertisement of a singleton content item (step 616).
- the producer publishes a notification using the other subject of the pair, called the past subject, to periodically republish advertisements that had been first published with an initial subject (step 618).
- a consumer submits one subscription on the initial subject to receive new advertisements (step 612), and it submits another subscription on the past subject to receive previously published advertisements (step 614). Alternatively, the consumer may only subscribe to the previously published advertisements.
- the matching previously published advertisements (and, if subscribed, the new advertisements) are pushed to the consumer using CBR (step 620). At some point the consumer decides to un-register the subscription for the past subject (step 622).
- FIG. 24D is a sequence diagram further illustrating the method 610.
- FIG. 24D depicts the interaction between the advertiser (e.g., using publisher machine 100), the CBR infrastructure (e.g., network of intelligent routers 92), and the consumer/requester (e.g., using subscriber machine 122).
- the sequence diagram illustrates that multiple processors (e.g., processor 93, processor 114, and processor 132) preferably execute method 610.
- FIG. 24E is a flowchart illustrating a third query-response method 630 using a publish-subscribe system. Method 630 may be executed using a variety of publish-subscribe systems.
- method 630 may be executed by processor 93 under software control in intelligent router 92 (or in the co ⁇ esponding functions contained in ASIC 91) and implemented in software modules, e.g., as represented by agent 128 and 105 for execution by processor 134 and 114 in user machine 122 and 100.
- Publish-subscribe networks are augmented with a caching feature, such as the one described above, whereby notifications published in the past are cached (step 632) and made available to future subscribers.
- Producers publish notifications to advertise singleton content items (step 634). Consumers submit a single subscription both to query previously advertised content items of interest (which are delivered by replaying the relevant notifications from the cache) and to obtain future advertisements of content items of interest (step 636). As in traditional publish-subscribe networks, the subscription characterizes a set of content items of interest. The matching previously published notifications are replayed from the cache (step 638) and pushed to the consumer using CBR (step 640).
- FIG. 24F is a sequence diagram further illustrating the method 630.
- FIG. 24F depicts the interaction between the advertiser (e.g. , using publisher machine 100), the CBR infrastructure (e.g., network of intelligent routers 92), and the consumer/requester (e.g., using subscriber machine 122).
- the sequence diagram illustrates that multiple processors (e.g., processor 93, processor 114, and processor 132) preferably execute method 630. 4.
- FIGs. 24G and 24H are flowcharts illustrating a fourth query-response method 650 using a publish-subscribe system. Method 650 may be executed using a variety of publish-subscribe systems.
- method 650 may be executed by processor 93 under software control in intelligent router 92, or in the co ⁇ esponding functions contained in ASIC 91, and implemented in software modules, e.g., as represented by agent 128 and 105, for execution by processor 134 and 114 in user machine 122 and 100.
- FIG. 24G illusfrates a data advertisement phase of method 650.
- a data advertisement is received (step 652).
- an ontology transformation is performed whereby the data advertisement is transformed to an equivalent subscription (step 654).
- the subscription (transformed data advertisement) is registered with the CBR network (e.g., the subscription is stored in intelligent routers 92) (step 656).
- FIG. 24H illustrates a query-response phase of method 650.
- a query is received
- the query is transformed to an equivalent notification (step 660) using ontology transformations.
- the transformation from the old space (e.g., data advertisement) to the new space (e.g., equivalent subscription) in steps 654 and 660 first involves defining attributes in the new space that represent combinations of attributes and expression operators from the old space.
- the transformation then involves creating notifications and subscriptions in the new space that are equivalent to the queries and advertisements of the old space, respectively.
- the transformation must satisfy the requirement that the notification matches the subscription if and only if the untransformed data advertisement matches the untransformed query.
- the notification is pushed to the advertiser by the publish- subscribe system performing CBR using the subscription (transformed data advertisement) as a filter for the notification (step 662).
- the CBR may be performed according to any of the above-described methods using the above-described infrastructure.
- the advertiser receives the transformed notification and returns responsive data to the requester (step 664) via some non-publish-subscribe means or via an agreed-upon convention using the publish-subscribe network.
- FIG. 241 is a sequence diagram further illustrating the method 650.
- FIG. 241 depicts the interaction between the advertiser (e.g., using publisher machine 100), the CBR infrastructure (e.g., network of intelligent routers 92), and the requester (e.g., using subscriber machine 122).
- the sequence diagram illustrates that multiple processors (e.g., processor 93, processor 114, and processor 132) may execute method 650.
- processors e.g., processor 93, processor 114, and processor 132
- Ae A V ⁇ xV 2 x ...
- a mapping between A and S, and between Q and N is defined as a pair of functions (MP AS , MP QN ), MP AS :A ⁇ S, MPQ N -Q ⁇ N, such that for any A e A and Q
- the queries are:
- the query matching (i.e., the response) is:
- the reciprocal notification matching is:
- Example 2 We again consider a simple stock quote information system.
- the data set is:
- the query matching i.e., the response
- the reciprocal notification matching is:
- the publish-subscribe network is augmented with additional interface points and/or message formats that allow producers to distinguish between notifications, advertisements and responses, and that allow consumers to distinguish between subscriptions and queries.
- the network honors the separation between the various primitives while using a uniform set of data structures, algorithms and protocols to simultaneously implement both styles of interaction.
- the embodiment described herein provides apparatus and methods for persistent storage of messages and recovery, restart and/or continuation of message flows in the face of network, link and/or node failures in a publish-subscribe network.
- Message persistence is the ability to store messages and retrieve them at a later time.
- a large number of specific applications, e.g. email generally require lengthy message persistence for messages flowing through the network, hi ideal conditions, with no failures in the network an always-connected subscriber should not need any persistence beyond that required for these specific applications.
- messages can get "lost" while traversing through the network due to various reasons - e.g., (1) failures or buffer overflows occurring either inside the network or at the user end or (2) users doing an explicit disconnect from the network and connecting back again after a time period.
- the persistence model of an event notification platform of the present embodiment is divided into two levels: short-term persistence and long-term persistence.
- Short-term persistence is designed for recovering from packet loss due to network congestion or short-term link failure.
- Long-term persistence is designed for recovering from other failures including, e.g., loss of user connections, failure of user machines, failure of applications, and/or longer-term network failure.
- a channel can either be persistent or real-time.
- a real- time chaimel transmits data that is generally only useful in real-time and does not have any application-specific persistence requirements.
- a persistent channel stores data traversing through network for a persistence time frame T. In other words, persistence for a persistent channel is guaranteed for a time frame T.
- This persistence of data is achieved through the following, for example: caching data at each edge node for the persistent duration of a channel; retrieving data from the cache transparent to the users under failure conditions; allowing the user to explicitly retrieve data from the cache; making the flow of data through the network persistent by guarding against router failures and setting up reliable tunnels between routers; and, protecting the channel components against failure through replication.
- Timed Persistence is the ability to retrieve the last time frame T of data from the publish-subscribe network. For example, if a subscriber leaves the network, any data on a persistent channel that is received during the subscriber's absence is held in the network for a time frame T (from the data's receipt). If the user returns within the time frame T, the user does not lose any data.
- FIG. 25 A is a block diagram illustrating certain components of a publish- subscribe network that provide persistence through caching.
- the network includes core routing nodes and an edge routing node.
- Each routing node preferably includes an intelligent router 92 (shown with the edge routing node) and a conventional backbone router (not shown), as described above in FIG. 4.
- Each intelligent router 92 that needs to perform caching for persistent channels preferably has a cache manager 218 co-located with it, as illustrated by FIG. 25 A.
- the cache manager 218 is described above with references to FIG. 8.
- the intelligent router 92 is preferably responsible for short-term persistence for retrieving lost data or recovering from router failures.
- the cache manager 218 is responsible for caching data to provide long-term persistence for a channel.
- the cache manager 218 preferably caches this data in the cache 540.
- the cache 540 preferably includes a memory and a disk (not shown).
- the agent 128 is responsible for communicating with the cache manager 218 to retrieve data from the cache 540, receiving the retrieved data and for organizing the retrieved data.
- each of the first level of core routing nodes upstream from the edge routing node preferably includes a cache manager 218. Upsfream is the direction moving away from the agent 128 (i.e., away from the subscriber machine 122).
- the first level of upsfream core routing nodes refers to the routing nodes immediately upstream from the edge routing node.
- FIG. 25 A only depicts one first level upstream core routing node, core routing node 548.
- a cache manager 218 provides for local caching of data at a network node at which it is located. Therefore, the operation of cache managers 218 located at various core routing nodes, including, e.g., core routing node 548, provides for distributed caching of data throughout the network core. This distributed caching provides a backup for the caching at edge routing node.
- Cache manager 218 preferably includes routines or sub-routines responsible for managing the cache 546.
- FIG. 25C is a flow chart illustrating an example method of persistent caching 550 for data contained in a single message. As shown, the method 550 shows an exemplary execution of steps performed by each routine shown in FIG. 25B. In operation, these routines preferably operate independent of one another. Method 550 may be performed at an intelligent router 92, at an edge routing node or core routing nodes, that provides caching.
- the method 550 can be implemented, for example, in software modules (e.g., cache manager 218) for execution by a processor 93 in intelligent router 92. Alternatively, it can be implemented in an ASIC or a combination of hardware and software, either in the same or different physical device as a co ⁇ esponding intelligent router 92.
- the intelligent router 92 forwards messages that need to be cached to the cache manager 218.
- the cache manager 218 receives the messages containing data (step 552), time marks the data in the messages (step 554), and caches the data in the cache 546 (step 556) (analogous to steps 322-326 in FIG. 14 (see above)).
- the cache manager 218 indexes the cached data (step 558) in a number of ways, e.g., channel identifier, subjects, publisher identifier, timestamp etc, as described above.
- the indexing step 558 may take place prior to, simultaneously with, or after the caching step 556.
- the cached data may be indexed and stored in a hierarchical directory structure (e.g., see FIG. 15). See also Tables 17-19.
- the data is cached in memory (the cache 546) and periodically moved to disk (step 560).
- a persistent time frame "T" for a particular channel is divided into N time grains each of size G.
- the caching in memory (step 556) is only for the duration of G.
- the cache manager determines that time interval G has passed (step 559), the data is moved to the disk (step 560).
- the cache manager 218 stores the data on the disk for the duration of persistent timeout interval T.
- the data co ⁇ esponding to a time interval G is deleted from the disk (step 564) once the time becomes greater than the Persistent timeout (T) for the channel + the upper limit of the interval (step 562).
- T Persistent timeout
- the cache manager 218 uses a time granularity G of 15 minutes.
- the policy preferably used is that when the last data cached during a time interval G (15 minutes) has been stored for T (2 hours), the entire data cached during that 15-minute interval will be discarded.
- the data cached in the beginning of that 15-minute interval will have been stored for longer than 2 hours before it is deleted, hi this example, the data cached during each 15-minute interval is a block of data. If the persistent time frame T is divided into N intervals, any point in time there will be N+l blocks of data (N on disk and 1 in memory) in the cache 540 for each subject. If the cache manager 218 receives a request for data (step 566), the cache manager 218 retrieves that requested cached data (for which time T has not passed), using the index (step 568) and transfers the retrieved data to the backbone router for routing to the requesting subscriber machine 122 (step 570). As described below, with reference to FIG.
- the cache manager 218 may invoke caches at the first level of upsfream core routing nodes (e.g., at core routing node 548) to retrieve the data. If the cache manager 218 does not receive a request, method 550 loops back to step 552.
- the data is also stored in caches at the first level of upsfream core routing nodes (e.g., core routing node 548).
- 552-564 are preferably performed at the first level of upstream core routing nodes prior to the messages being routed to and method 550 being performed at the edge routing node.
- the locations of the cache 546 (e.g., at the edge routing node) and the caches at the first level of upstream core routing nodes (e.g., at the first upsfream core router 548) are stored locally in the agent 128. Therefore, if the subscriber machine 122 moves (e.g., the subscriber is a mobile subscriber that re-connects to the network at a different edge routing node), the agent 128 will provide the new edge routing node with the location of the caches for the edge routing node and first level of upstream core routing nodes to which the subscriber machine 122 was connected prior to moving. This enables persistence to be maintained even if the subscriber machine 122 moves. As seen in FIG.
- a flowchart of a method of persistent message retrieval 580 a user application 182 at subscriber machine 122 preferably invokes the agent 128 to request data from the cache (step 582).
- the agent 128 invokes the cache 546 at the appropriate edge routing node (e.g., using a getcache command) to get the data (step 584).
- the agent 128 may indicate what time frame of data to retrieve.
- the cache manager 218 may check the connection history of a subscriber machine 122 (see Table 18 above) to determine what cached data to retrieve. If the connection history indicates that the subscriber machine 122 was off-line for X period of time, the cache manager 218 may retrieve all available cached data for X. Additionally, the cache manager 218 may simply retrieve all data cached within persistent time frame T. If the edge routing node cache 546 has the data (step 585), the agent 128 retrieves the data from the cache 546 (step 586).
- the cache manager 218 uses a list of the caches at the first level of upsfream routing nodes (e.g., core routing node 548) provided by the agent 128 to invoke these upstream caches to locate the data (step 588). Once the cache manager 218 locates the data in an upstream cache or caches, the cache manager 218 retrieves the data from the upstream cache(s) (step 590). The cache manager 218 may perform duplicate suppression (i.e., eliminate duplicate data) (step 592) before forwarding the data to the agent 128 (step 594).
- duplicate suppression i.e., eliminate duplicate data
- An infrastructure consistent with the present embodiment has the following components or equivalent components: channel managers; event agent; and, routing nodes and caches. To guard against loss of messages during failure conditions, each of these components needs to be protected.
- Channel Manager Fault Protection Channel managers are described above with reference to FIG. 6.
- the channel managers are protected from failures through replication.
- the channel managers operate in a primary-backup group, where there is one primary module and the other backup modules.
- the primary module is responsible for all updates to the data and also responds to queries.
- the backup can only serve queries but cannot perform updates. It forwards all updates to the primary.
- Components of the publish-subscribe network that need to retrieve channel information from the channel managers are refe ⁇ ed to as channel manager clients and are assigned a channel manager. These clients include intelligent routers 92, and their cache managers 218, and subscriber machines 122, and their agents 128. If the channel manager assigned to the client fails, the client accesses another channel manager and connects to it.
- Level 0 controls the failure effect locally (i.e., within the failed router) and the recovery is also limited to the failed routing node.
- Level 0 recovery depends on hardware redundancy to recover from hardware failures. The hardware resources in a router are replicated. The main advantage of level 0 recovery is the short recovery time and transparency.
- Level 1 controls the failure effect within a LAN. Level 1 recovery may be implemented with IP fail-over using cluster technology. However, tunnels of a failed router to its neighbors have to be reconstructed after recovery. Therefore, level 1 recovery is less transparent and takes longer than level 0 recovery.
- Level 2 recovery is to recover router failure by remote backup router over a WAN, without affecting network topology, which may cause complex reconfiguration and data loss.
- Level 2 may be implementing by grouping routers to form a recovery set (e.g., a primary and backup router).
- One router may be a primary in one group and a backup for another group.
- the primary exchanges routing and network configurations with the backup, and when the backup detects a failure of the primary, it takes over the functions of the primary.
- the backup ensures that the primary router failed, and not the link to the primary router, by ensuring that it can connect to the majority of neighbor primary routers of the failed primary router. If it cannot, level 3 recovery is triggered.
- Level 3 on the other hand may involve a system-wide coordinated recovery.
- Level 3 is used when all else fails.
- the router that detects such a failure sends a restart notice to all downstream routers.
- the routers receiving the notice suspend all other recovery, stop all network services, and discard all multicasting information in the restart message.
- the router restarts all services, including tree-building process for multicasting. Any failure in the network is treated by a lower level recovery procedure. If a lower level recovery cannot be applied, the recovery is escalated to the next higher level. If none of the lower level recoveries work, the level 3 recovery is used. 3. Reliable Tunnels
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Information Transfer Between Computers (AREA)
- Computer And Data Communications (AREA)
Abstract
Description
Claims
Applications Claiming Priority (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US36910502P | 2002-03-28 | 2002-03-28 | |
US36883302P | 2002-03-28 | 2002-03-28 | |
US36887002P | 2002-03-28 | 2002-03-28 | |
US36883102P | 2002-03-28 | 2002-03-28 | |
US36983202P | 2002-04-03 | 2002-04-03 | |
US44778203P | 2003-02-19 | 2003-02-19 | |
PCT/US2003/009624 WO2003083703A1 (en) | 2002-03-28 | 2003-03-28 | Method and apparatus for reliable and efficient content-based routing and query and response in a publish-subscribe network |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1969480A1 true EP1969480A1 (en) | 2008-09-17 |
EP1969480A4 EP1969480A4 (en) | 2008-12-03 |
Family
ID=28679208
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP03714457A Withdrawn EP1969480A4 (en) | 2002-03-28 | 2003-03-28 | Method and apparatus for reliable and efficient content-based routing and query and response in a publish-subscribe network |
Country Status (6)
Country | Link |
---|---|
EP (1) | EP1969480A4 (en) |
JP (2) | JP2005521950A (en) |
KR (1) | KR100971506B1 (en) |
CN (1) | CN100458767C (en) |
AU (1) | AU2003218455A1 (en) |
WO (1) | WO2003083703A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220365979A1 (en) * | 2019-08-06 | 2022-11-17 | Twitter, Inc. | Managing query subscription renewals in a messaging platform |
Families Citing this family (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7574408B2 (en) * | 2006-05-05 | 2009-08-11 | Microsoft Corporation | Publisher unions |
CN101668031B (en) | 2008-09-02 | 2013-10-16 | 阿里巴巴集团控股有限公司 | Message processing method and message processing system |
JPWO2010090027A1 (en) * | 2009-02-05 | 2012-08-09 | 日本電気株式会社 | Broker node and event topic control method in distributed event delivery system |
US8516066B2 (en) | 2009-08-25 | 2013-08-20 | Telefonaktiebolaget L M Ericsson (Publ) | Method and apparatus for handling subscriptions to changes in user data in a telecommunications system |
US9720681B2 (en) * | 2011-07-20 | 2017-08-01 | Data I/O Corporation | Device programming system with data broadcast and method of operation thereof |
US8750309B2 (en) * | 2012-02-29 | 2014-06-10 | Telefonaktiebolaget L M Ericsson (Publ) | Compound masking and entropy for data packet classification using tree-based binary pattern matching |
US9886210B2 (en) * | 2015-06-09 | 2018-02-06 | Ultrata, Llc | Infinite memory fabric hardware implementation with router |
US9602455B2 (en) * | 2015-08-07 | 2017-03-21 | Machine Zone, Inc. | Scalable, real-time messaging system |
JP6547580B2 (en) | 2015-10-20 | 2019-07-24 | 富士通株式会社 | Distributed control method, distributed control system, and distributed control program |
JP6525102B2 (en) * | 2016-02-18 | 2019-06-05 | 日本電気株式会社 | Communication system, edge server, method and program |
US10129360B2 (en) | 2016-03-28 | 2018-11-13 | The Boeing Company | Unified data networking across heterogeneous networks |
JP6665697B2 (en) | 2016-06-09 | 2020-03-13 | 富士通株式会社 | Past information providing program, past information providing method, and past information providing device |
US9608928B1 (en) * | 2016-07-06 | 2017-03-28 | Machine Zone, Inc. | Multiple-speed message channel of messaging system |
KR102031726B1 (en) * | 2017-11-16 | 2019-10-14 | 경북대학교 산학협력단 | OPERATING METHOD FOR INTERNET OF THINGS SYSTEM BASED ON CoAP USING DISTRIBUTED PUBLISH-SUBSCRIBE TECHNIQUE |
KR102033500B1 (en) * | 2017-12-26 | 2019-11-08 | 경희대학교 산학협력단 | Packing Routing method by Edge Cloud in Distributed Cloud System |
JP6750646B2 (en) | 2018-06-07 | 2020-09-02 | トヨタ自動車株式会社 | In-vehicle device, information processing method, and information processing program |
KR102339661B1 (en) * | 2018-09-28 | 2021-12-14 | (주)구름네트웍스 | Gateway Devices For DDS |
KR102279601B1 (en) * | 2018-09-28 | 2021-07-19 | (주)구름네트웍스 | Method Of Gateway For DDS |
US20200364552A1 (en) * | 2019-05-13 | 2020-11-19 | Baidu Usa Llc | Quantization method of improving the model inference accuracy |
US11714694B2 (en) * | 2019-11-08 | 2023-08-01 | Salesforce, Inc. | Error notification mechanism for streaming events |
KR102527601B1 (en) * | 2021-04-02 | 2023-05-02 | 이상규 | Method for chatting messages by topic based on subscription channel reference in server and user device |
CN113783838B (en) * | 2021-08-05 | 2024-02-06 | 山东有人物联网股份有限公司 | Subscription method, subscription control device and computer readable storage medium |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH08153054A (en) * | 1994-11-28 | 1996-06-11 | Mitsubishi Electric Corp | File transfer system |
US6065058A (en) * | 1997-05-09 | 2000-05-16 | International Business Machines Corp. | Dynamic push filtering based on information exchanged among nodes in a proxy hierarchy |
JP3307587B2 (en) * | 1998-05-08 | 2002-07-24 | 日本電気株式会社 | Middleware stored data updating method and server system executing the method |
US20010052015A1 (en) * | 1998-06-24 | 2001-12-13 | Chueng-Hsien Lin | Push-pull sevices for the internet |
US6415027B1 (en) * | 1998-08-12 | 2002-07-02 | Bellsouth Intellectual Property Corporation | Networks, systems and methods for intelligently routing traffic within a telephone network |
AU6255199A (en) * | 1998-09-17 | 2000-04-17 | Tod Mcnamara | System and method for network flow optimization using traffic classes |
FI108195B (en) * | 1998-10-19 | 2001-11-30 | Nokia Networks Oy | Mechanism for network initiated information transfer |
US6252862B1 (en) * | 1999-05-20 | 2001-06-26 | Motorola, Inc. | Method and apparatus for routing packet data in a communications system |
JP4374094B2 (en) * | 1999-06-14 | 2009-12-02 | 株式会社ジャストシステム | Information processing apparatus, information processing method, and computer-readable recording medium recording a program for causing a computer to execute the method |
US6539396B1 (en) * | 1999-08-31 | 2003-03-25 | Accenture Llp | Multi-object identifier system and method for information service pattern environment |
JP3679289B2 (en) * | 1999-12-06 | 2005-08-03 | 株式会社日立製作所 | Data relay method by gateway device |
DE10000223A1 (en) * | 2000-01-05 | 2001-07-12 | Basf Ag | Microcapsules which are useful in, e.g. detergent or skin care compositions, can release a fragrance from a hydrophobic core when the polymer coating of the capsule is broken down |
EP1130845A3 (en) * | 2000-02-18 | 2001-09-12 | Agilent Technologies Inc. a Delaware Corporation | Publish/subscribe system |
GB2363216A (en) * | 2000-03-08 | 2001-12-12 | Ibm | Publish/subscribe data processing with subscriptions based on changing business concepts |
JP2001256098A (en) * | 2000-03-09 | 2001-09-21 | Hitachi Ltd | Method for controlling cache in proxy server |
KR20010107151A (en) * | 2000-05-25 | 2001-12-07 | 반창모 | Multi-purpose multi-media kiosk with computer network and its service method |
US7216179B2 (en) * | 2000-08-16 | 2007-05-08 | Semandex Networks Inc. | High-performance addressing and routing of data packets with semantically descriptive labels in a computer network |
-
2003
- 2003-03-28 CN CNB038121808A patent/CN100458767C/en not_active Expired - Lifetime
- 2003-03-28 KR KR1020047015458A patent/KR100971506B1/en active IP Right Grant
- 2003-03-28 AU AU2003218455A patent/AU2003218455A1/en not_active Abandoned
- 2003-03-28 EP EP03714457A patent/EP1969480A4/en not_active Withdrawn
- 2003-03-28 JP JP2003581058A patent/JP2005521950A/en active Pending
- 2003-03-28 WO PCT/US2003/009624 patent/WO2003083703A1/en active Application Filing
-
2009
- 2009-02-13 JP JP2009031766A patent/JP2009163753A/en active Pending
Non-Patent Citations (3)
Title |
---|
GERO MÜHL: ""Generic Constraints for Content-Based Publish/Subscribe"" PROC. OF THE 6TH INTERNATIONAL CONFERENCE ON COOPERATIVE INFORMATION SYSTEMS (COOPIS'01), [Online] 5 September 2001 (2001-09-05), - 7 September 2001 (2001-09-07) pages 211-225, XP002500569 Retrieved from the Internet: URL:http://www.kbs.cs.tu-berlin.de/publications/fulltext/coopis2001.pdf> [retrieved on 2008-10-20] * |
LUKASZ OPYRCHAL, MARK ASTLEY , JOSHUA AUERBACH, GURUDUTH BANAVAR, ROBERT STROM, DANIEL STURMAN: "Exploiting IP Multicast in Content-Based Publish-Subscribe Systems" PROC. OF THE IFIP/ACM INTL CONFERENCE ON DISTRIBUTED SYSTEMS PLATFORMS AND OPEN DISTRIBUED PROCESSING (MIDDLEWARE), LNCS VOL. 1795, [Online] 4 April 2000 (2000-04-04), - 8 April 2000 (2000-04-08) pages 185-207, XP002500570 Retrieved from the Internet: URL:http://www.cs.unibo.it/~panzieri/SisMid/Opyrchal.pdf> [retrieved on 2008-10-17] * |
See also references of WO03083703A1 * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220365979A1 (en) * | 2019-08-06 | 2022-11-17 | Twitter, Inc. | Managing query subscription renewals in a messaging platform |
Also Published As
Publication number | Publication date |
---|---|
CN100458767C (en) | 2009-02-04 |
CN1656474A (en) | 2005-08-17 |
KR100971506B1 (en) | 2010-07-21 |
WO2003083703A1 (en) | 2003-10-09 |
KR20040102061A (en) | 2004-12-03 |
JP2005521950A (en) | 2005-07-21 |
EP1969480A4 (en) | 2008-12-03 |
AU2003218455A1 (en) | 2003-10-13 |
JP2009163753A (en) | 2009-07-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7551629B2 (en) | Method and apparatus for propagating content filters for a publish-subscribe network | |
US7376092B2 (en) | Method and apparatus for implementing persistent and reliable message delivery | |
US7672275B2 (en) | Caching with selective multicasting in a publish-subscribe network | |
WO2003083703A1 (en) | Method and apparatus for reliable and efficient content-based routing and query and response in a publish-subscribe network | |
US7627603B2 (en) | Method and apparatus for implementing query-response interactions in a publish-subscribe network | |
US7653753B2 (en) | Method and apparatus for content-based packet routing using compact filter storage and off-line pre-computation | |
US7587517B2 (en) | Packet routing via payload inspection for quality of service management | |
US7545805B2 (en) | Method and apparatus for content-based routing and filtering at routers using channels | |
US20030195946A1 (en) | Method and apparatus for reliable publishing and subscribing in an unreliable network | |
US6910033B2 (en) | Method for storing Boolean functions to enable evaluation, modification, reuse, and delivery over a network | |
US20040078450A1 (en) | Packet routing via payload inspection for digital content delivery | |
CN1701304B (en) | System, method and apparatus for inspecting packet routing via payload in a publish-subscribe network | |
US20060146999A1 (en) | Caching engine in a messaging system | |
US20040083305A1 (en) | Packet routing via payload inspection for alert services | |
CA2594082A1 (en) | A caching engine in a messaging system | |
JP2008252907A (en) | Packet routing via payload inspection, and subscription processing in publish-subscribe network | |
US20030165139A1 (en) | Packet routing via payload inspection | |
US7117270B2 (en) | Method for sending and receiving a Boolean function over a network | |
CN101312457A (en) | Method, router, apparatus and the network used in a publish-subscribe network | |
US7411954B2 (en) | Efficient implementation of wildcard matching on variable-sized fields in content-based routing | |
Ott et al. | XML-based semantic multicast routing: An overlay network architecture for future information services | |
KR20040039288A (en) | Packet routing via payload inspection and subscription processing in a publish-subscribe network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20041022 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): DE FR GB |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04L 12/28 20060101ALI20081023BHEP Ipc: G06F 17/00 20060101AFI20031011BHEP Ipc: G06F 17/30 20060101ALN20081023BHEP Ipc: H04M 7/00 20060101ALI20081023BHEP Ipc: H04L 29/08 20060101ALI20081023BHEP |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 20081031 |
|
17Q | First examination report despatched |
Effective date: 20100914 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20141001 |