WO2018149479A1

WO2018149479A1 - Distributed meta messaging computing

Info

Publication number: WO2018149479A1
Application number: PCT/EP2017/053260
Authority: WO
Inventors: Wen GUAN
Original assignee: Guan Wen
Priority date: 2017-02-14
Filing date: 2017-02-14
Publication date: 2018-08-23

Abstract

The invention relates to a new Distributed Meta Messaging Computing architecture for distributing metadata as messages via network which includes metadata and message structure, Distributed Messaging Network, Knowledge Repository and agent repository. Further, the invention relates to a method for broadcasting messages in a distributed independent open network environment, wherein Messaging Stations are connected as a distributed open messaging network to relay messages, wherein the method comprises formatting a message with a header and a body with a publisher interface, publishing messages to a Messaging Station with a publisher interface, collecting messages with passive listener, active subscriber or active fetcher in a Messaging Station, relaying or distributing messages from one Messaging Station to other Messaging Stations, message systems or applications, consuming or retrieving messages from Messaging Stations with a consumer interface, and parsing messages.

Description

Distributed Meta Messaging Computing

DESCRIPTION

TECHNICAL FIELD

This invention relates generally to metadata, messaging, distributed computing, and more particularly to distributing metadata as messages over network through a Distributed Messaging Network.

BACKGROUND Nowadays, internet is one of the most important way to provide or get information or services, such as news, e-commerce, hotel booking, ticket reservation and so on. At the same time, more and more information and service is coming to internet. It's an era of information explosion.

However, how to distribute data or services to users is still a big issue. Thanks to search engines and similar services, users can find a lot of information about data or services. But in search engines, it is driven by search engines to collect information about data or services. The normal provider who provides data or services is passive. As a result, the service providers have no control when and whether their information will be crawled by search engines. In addition, for services which are more than content-based information, such as operation interface, it's not easy for search engines to crawl. Besides, search engines cannot dig out information about data or services which are security blocked by firewalls. So a lot of private information is still not well organized or indexed in organizations. Even the information or service is open to all over the world, search engines may take some time to crawl it. A common way to get real-time information is not available. At the same time, service providers always want to distribute their information or services to as many people as they can, not only in some central locations such as search engines, amazon and so on.

Hence, a common solution to distribute the metadata of data or services as messages, where service providers can drive efficiently in a proactive way to distribute their metadata about data or services, including data or services which are more than content-based information, is attractive. With this way, an organization can have its own knowledge repositories easily to index private data or services. At the same time, the organization can easily publish its public information to attract consumers.

Although message broker or message bus have been used in many situations, they are only used in separate organized applications or software environment. They are not used to relay messages, especially in an open unorganized network environment with routing loops.

Meanwhile, there is a big explosion on smart devices usage in recent years. With the limits of wireless cost, power of batteries, cooling and so on, it's not good to use smart devices as a computing node. But its CPU is powerful enough to handle metadata messages. Currently a lot of different applications are already developed to consume and deploy some well-definite services or information on internet. A general method to consume and deploy diverse types of services or information as messages on internet is still missing.

The Emergence of Distributed Meta Messaging Computing:

Consider the following scenarios:

1. For organizations such as institutes and companies, there are a lot of data or services, which can be private or public. For many cases, the data or services are not indexed. For example, in a group, a web document or service created by a member may be forgotten, which can happen very frequently. Although search engine can crawl a lot of internet information, independent information which is not referred in other internet information can easily be missed by search engine. In addition, to protect private information, organizations normally have firewalls to block crawling from outside.

2. E-commerce has becoming one the most important economic activities. But currently e-commerce is totally controlled by e-commerce web sites. We remember Amazon, Aliexpress and so on, but we cannot remember the brand of the product provider (Currently the brands we remember are mostly from the entity shops, not from the virtual online shops). We remember booking.com, but we don't remember the hotel name. The product or service provider needs to register their information on different web sites. For example, a hotel needs to register itself on booking.com, hostelworld.com and so on. It's not good for product or service provider to build their brand. E- commerce only build the web site's brand. But in production area, it's important to build the product brand.

3. With the development of e-commerce, self-employed service or trade, personal servicer or trade, and similar services have become more and more popular. For example, Uber and ride-sharing are already very popular. But there are still many other similar services which don't have service channel like Uber. If the service providers can broadcast their services to subscribers with filters, it will be very interesting.

4. From Grid to Cloud, we are working on different directions to make use of distributed computing resources. But we are still on the way to be able to easily make use of distributed computing resources.

5. 'BigData' is one of the most important feature of current era. At the same time, the source of the data can from different organizations, groups or teams. How to obtain and organize BigData is a big issue. These examples different in many respects, but they also have much in common: they need to share or distribute their products or services to others. In fact, they only need to distribute an abstract of their product or service to others, where the abstract information about their product or service is metadata. It's also the main point we will discuss in the following, to distribute metadata to others.

Figure 9, for scenario 1, an organization can have a lot of data or services. Every data or service can distribute its metadata to one or more private area managed by the organization. Then the organization can retrieve metadata in the private area to decide which metadata can be distributed to public area. In this Figure, search engines can consume metadata from the private or public area instead to crawl internet. It not only solves the firewall issues, but also provides a better way for organizations to manage their data or services.

Figure 10 : For scenario 2, in this Figure, one product provider or several product providers together can build their brands and setup a product web site. Then they distribute their product metadata to a public area, where different e-commerce web sites can consume metadata from the public area and then build an e-commerce store with these metadata. In this model, e-commerce web sites only manage metadata. Consumers can browser metadata information from the e-commerce web sites. But if consumers want to buy the product or service, they will be redirect to the product or service providers, where brand web sites can be built. In this model, the brand can be a private product brand. It can also be a service standard level.

Figure 11 : For scenario 3, in this Figure, self-employer or people can register their services at one web site and then broadcast the metadata of their services. Users or sites can subscribe to metadata they are interested with filters, to get services they want. One popular application of this model is e-news. Different news providers can have a iot of news on their web sites. The metadata of these news is collected and showed on mobile apps. Users can click different metadata to read full news from different news providers.

Figure 12 : For scenario 4, to make use of distributed computing resources, in fact, there are only two main points: how to find resources and how to use resources. If the computing resources providers can distribute the metadata of their resources to public area, it solves the problem how to find resources. Applications can consume the metadata information to find compatible computing resources. But in a distributed environment, many applications may find the same metadata of the computing resources. To make sure the application can get the computing resource before it starts real computing work, the application can send an agent to the computing resource (If the application sends real computing work to computing resources before it gets the computing resource, this part of computing work may need to wait or fail if the computing resource is occupied by others). The application and running agents can build a virtual cluster. In this model, the application can be a platform application or a user application.

For scenario 5, it similar like Figure 11. BigData can be in a distributed system, wherein the data source distributes the metadata to an area to build the catalogue of BigData.

In all these scenarios above, we distribute metadata to private or public area. In fact, we can combine all these examples together with filters. So hotel booking sites can only consume hotel metadata and computing applications only consume computer resources metadata. The Distributed Meta Messaging Computing architecture will solve all these problems. In the Distributed Meta Messaging Computing, metadata is distributed and relayed as messages through Distributed Messaging Network. Knowledge Repositories consumes messages from the Distributed Messaging Network with different filters to build search engines, e-commerce sites, hotel bookers, or even computing applications. The detail of Distributed Meta Messaging Computing will be discussed in the following.

SUMMARY OF THE INVENTION

A Meta Computing architecture, Distributed Meta Messaging Computing, is a method, framework and system for broadcasting metadata via network by distributing metadata as messages through a Distributed Messaging Network.

According to an embodiment of the invention, a message structure is defined for distributing metadata, wherein the message structure contains a header with information for message distributing and a body including the metadata. The metadata not only comprises meta-information of data or services, but also includes URL (Uniform Resource Locator) references which can be used to locate and access the data or service that the metadata describes.

According to an embodiment of the invention, a Distributed Messaging Network is adapted to distribute meta messages in the Distributed Meta Messaging Computing architecture. The Distributed Messaging Network consists of one or many connected Messaging Stations which can be deployed by the same or different organizations. A Messaging Station can receive messages by actively consuming, fetching or subscribing messages from other places, or passively accepting messages published by others. It's also open to publish received messages to other Messaging Stations and passively accept subscribes, fetchers or consumers from others. The Messaging Station caches messages just like a news kiosk. An important part is that the Messaging Station discards duplicated messages, which avoids messaging explosion in connected Messaging Stations with routing loops. With this approach, a Distributed Messaging Network can distribute messages to many Messaging Stations over the network.

Users can subscribe or consume messages to get metadata information from one or more message stations. Users can also get or search metadata information from knowledge repositories which consume messages from the Distributed Messaging Network and assemble metadata information from messages into knowledge.

The Distributed Messing Network of the invention presents a way to broadcast messages to all over the world in a distributed independent network environment, with connected Messaging Stations to relay messages. At the same time it solved the problem of messages explosion because of huge number of duplications when broadcasting messages in a network environment with routing loops. The Distributed Meta Messaging Computing of the invention presents a way to broadcast metadata with Distributed Messaging Network and to collect metadata into a Knowledge Repository. With the metadata to describe internet data or service, it will change the way to index internet comparing to search engine crawling, it provides a common way to adapt different types of metadata with namespace. Distributed Meta Messaging Computing also provides a solution that organizations can have their private intranet index to organize secret information and at the same time have their public information to be indexed in internet.

Metadata should be consistent with some format that can be processed by consumers. A namespace is included to define metadata format. The namespace includes two parts: the internet- available namespace specification and the URL (Uniform Resource Locator) to locate and access the namespace specification. A namespace specification can define a metadata format. The namespace URL will be included in the metadata. A globally unique identifier id is included in metadata to distinguish itself with other metadata. The globally unique identifier can be a unique URi (Uniform Resource Identifier), a unique URL (Uniform Resource Locator), a GUID (Globally Unique Identifier) or some other unique identifiers. The metadata also includes some URLs to locate and access the data or services that the metadata describe, where users can use these URLs to access data or services. It should also include a version number to distinguish whether it's a duplicated message or a new message. The version number can just be a timestamp. A lifetime item should be included too, which will tell the users or knowledge repositories whether the metadata is still valid. For different namespace, the maximum length of the lifetime can be different. In the metadata, the category, some key words, an abstract description are needed too for users to understand the service. Medias can also be included with links.

A message structure is defined for distributing metadata, where the message structure contains a header with information for message distributing and a body including the metadata. The header information is used for distributing messages. During distributing messages, only the header will be read and the body will not be parsed. The header needs to include the metadata id, the metadata version number, the metadata lifetime, the checksum of the metadata and the format type of the body, such as json, xml and so on. It should also include the metadata category information, hop, maximum hops and so on. The message id and version number are used by Messaging Stations to discard duplicated messages. The format type and checksum are used by users to parse the message body. The hop and maximum hops are used to avoid infinitely distributing a message in a Distributed Messaging Network. The category can be used as a filter when subscribing to or retrieving messages from a Messaging Station. For example, a user can filter to only download messages about football. The message body contains formatted metadata with format type defined in the header. Both the header and the body are open and new features can be added to them.

A Distributed Messaging Network of the invention is adapted to distribute meta messages. The Distributed Messaging Network consists of one or many connected Messaging Stations which can be deployed by the same or different organizations. The Messaging Station contains of a message collector, a message processor and a message scatter. The message collector collects messages through passively accepting messages published by users or other Messaging Stations, actively consuming or fetching messages from other Messaging Stations, or subscribing messages from other Messaging Stations. The message processor filters message headers, discards duplicated messages to avoid message explosion, and processes some other handlings. The message scatter distributes messages to other Messaging Stations or applications through actively publishing messages, open to passively accept fete hers or consumers, open to accept subscribers, and so on. The Messaging Station caches messages just like a news kiosk. The Messaging Station relays messages to connected Messaging Stations or applications. Connected Messaging Stations construct a Distributed Messaging Network.

Topics and channels can be configured in a Messaging Station during deployment, such as private and public, normal and administration, education and commerce, and so on, to apply different connection and filter policies. For example, an institute can deploy a Messaging Station with a part of topics or channels restricted to be accessed only inside of the institute and a part of topics or channels open to the world.

The Distributed Meta Messaging Computing includes a publisher which formats input metadata information to a message and then publishes the message to a Distributed Messaging Network. The publisher can be a script, an API function or a separate independent application, such as smart device applications. For example, the publisher can be a mobile application which can be used by hotels to publish their hotel's metadata information.

The Distributed Meta Messaging Computing also includes an agent repository. Users can register to the agent repository and publish metadata to the agent repository. The agent repository can filter and validate these received metadata, and then publish them to the Distributed Messaging Network. Security checks and other policies can be applied on the agent repository, which can protect the Distributed Messaging Network from some invalid messages or attackers. The agent repository can also periodically refresh metadata by fetching information from the data or service that the metadata describes, and then publish refreshed metadata to the Distributed Messaging Network.

The Distributed Meta Messaging Computing contains a Knowledge Repository which collects messages from Distributed Messaging Networks. The collected messages are parsed to metadata and then assembled in the Knowledge Repository. A user interface is included in the Knowledge Repository to facilitate user queries. In fact, depending on the filters applied on the Knowledge Repository, the Knowledge Repository itself can be a search engine, a news station, an e-commerce market which includes distributed suppliers, or something else.

A consumer client is included to consume or subscribe to the Distributed Messaging Network to get the metadata messages and parse metadata messages. User applications can use the consumer client to get metadata. User applications can also retrieve metadata from the knowledge repositories. Through the URL reference in the metadata, consumers or users can select corresponding protocols to access the external data or service which the metadata describes.

In this manner, Distributed Meta Messaging Computing distributes metadata of data or services; at the same time, Distributed Meta Messaging Computing collects metadata to construct distributed index or dictionary services for internet. A Distributed Meta Messaging Computing instance can be deployed in an organization, with a part of topics or channels open to connect a widely deployed messaging network. Open instances can also be deployed to serve a lot of aspects, such as to broadcast news, to broadcast products for companies or individual sellers, and so on. Dedicated instances can also be deployed to serve for dedicated usage. For example, a Distributed Meta Messaging Computing instance can be deployed within distributed police stations for contacting between polices.

SSL(Secure Sockets Layer)/TSL(Transport Layer Security) protocols with host certification or user certification validation can be applied to authorize different components and deployments.

The detailed description of the present invention will be followed with additional features and advantages. BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of the structure of the Distributed Meta Messaging Computing.

FIG. 2 is a block diagram of an implementation of a Messaging Station.

FIG. 3 is a flow diagram of a method to discard duplicated messages.

FIG. 4 is a block diagram of an example of a Distributed Messaging Network with connected

Messaging Stations.

FIG. 5 is a block diagram of an implementation of a Knowledge Repository.

FIG. 6 is a block diagram of an implementation of a fat Messaging Station with extended Knowledge Repository.

FIG. 7 illustrates distributing and accessing external services through the Distributed Meta

Messaging Computing. FIG. 8 illustrates some deployment examples of the Distributed Meta Messaging Computing. Figures 9-12 are prior art scenarios explained above.

DETAILED DESCRIPTION

This disclosure describes a Meta Computing architecture— the Distributed Meta Messaging Computing, it's a method, framework and system for distributing metadata via network by distributing metadata as messages through a Distributed Messaging Network.

FIG.l shows the architecture of the Distributed Meta Messaging Computing. Users can use a publisher client 110 to convert metadata to messages and publish messages to a Distributed Messaging Network 140. It can also use an agent repository 130 to convert metadata to messages and publish messages to a Distributed Messaging Network 140.

The Distributed Messaging Network 140 contains connected Messaging Stations 141. It employs mathematics of epidemics to build gossip-like network. The metadata is distributed as a message in the Distributed Messaging Network. The detail will be described in the following paragraphs. A Messaging Station 141 works as a message kiosk which caches messages and distributes messages.

Messages in the Distributed Messaging Network can be consumed through consumer client 160. A Knowledge Repository 150 is provided, which will collect messages from a Distributed Messaging Network, parse messages to metadata and assemble metadata together for user queries. The Knowledge Repository 150 can work as a backend of a search engine, an e-commerce market and so on.

SSL(Secure Sockets Layer)/TSL(Transport Layer Security) protocols with host or user certificate validation can be applied on the connections between different components and deployments. A dedicated instance can have its own certificate authority to ban external connections.

Additional features and advantages about the present invention are followed with more details.

Meta data

In the Distributed Meta Messaging Computing, metadata should be consistent with some format that can be processed by consumers. It should include a namespace to direct the parser to parse the metadata. The namespace includes a web available specification and a URL (Uniform Resource Locator) to locate the specification. The namespace specification can define a format for data, service, application interface or something else. For example, a namespace specification can define to a REST (Representational State Transfer) interface standard. A globally unique identifier id is included in metadata to distinguish itself with other metadata. The globally unique identifier can be a unique URI (Uniform Resource Identifier), a unique URL (Uniform Resource Locator), a GUID (Globally Unique identifier) or some other unique identifiers. The metadata also includes some URLs to locate and access the data or services that the metadata describe, where users can use these URLs to access data or services. It should also include a version number to distinguish whether it's duplicated message or a new message. The version number can just be a timestamp. A lifetime item should be included too, which will tell the users or knowledge repositories whether the metadata is still valid. For different namespace, the maximum length of the lifetime can be different. In the metadata, the category, some key words, an abstract description are needed too for users to understand the service. More details and medias with links can be included too.

Metadata - {'namespace': 'http://www.example.org/namespace_defmition',

'parentNamespace': 'optional; Namespace can be an extension of some popular accepted namespace, which may not be accepted by some components. This item is used to tell these components how to handle this metadata. Otherwise components will use default parser to parse the metadata ',

'id': 'http://www.example.org/resource/id' 'version': 'integer of current timestamp',

'lifetime': 'one day as an example',

'category': 'IT',

'subcategory': 'optional, can be used by different namespace parser',

'keywords': 'Metadata, Messaging, distributed Computing', 'abstract': 'The description is just an example.',

...}

Metadatas = [Metadata]

A simple example is included above. The namespace specification defines the structure and meaning of the metadata. With different namespaces, the metadata can refers to data, service, an application interface, a RPC (Remote Procedure Call), a REST interface and so on.

A special metadata category and namespace can be defined to describe the Distributed Meta Messaging Computing components. For example, when setting up a new Messaging Station, a metadata which describes the Messaging Station can be broadcasted to other Messaging Stations. The Messaging Station can also broadcast messages to ask for the connection topologies of other Messaging Stations. It's a way to construct a self-organizing messaging network.

Meta Messaging

A message structure is defined for distributing metadata, where the message structure contains a header with information for message distributing and a body including the metadata. The header information is used for distributing messages. During distributing messages, only the header will be read and the body will not be parsed. So the header needs to include the metadata id, the metadata version number, the metadata lifetime, the checksum of the metadata and the format type of the body, such as json, xml and so on. To avoid that the header is too long, if the metadata id is too long, it can be convert to shorter globally unique id such as GUID. It should also include its category information, hop, maximum hops and so on. The id and version number are used by Messaging Stations to discard duplicated messages. The format type and checksum are used by users to parse the body. The hop and maximum hops are used to avoid infinitely distributing a message in a Distributed Messaging Network. The category and keywords can be used by a filter when subscribing to or retrieving messages from a Messaging Station. The body contains metadata formatted with format type defined in the header. Both the header and the body are open and new features can be added to them.

Message - {header, body}

Header = {'id': metadata['id'] ,

'version ': metadata['version '],

'lifetime': metadata['lifetime'],

'category': metadata['category'],

'keywords ': metadataf'keywords '],

'hop': 'the number of current hops, adding 1 when passing through a Messaging

Station',

'maxHops': 'when hop reaches this number, the message will be dropped',

'checksum': 'checksum of the metadata',

'publisher': the publisher of the message,

'format': 'json, xml or others'}

body -formatted metadata

If body includes a list of metadata, header can select info from the first metadata. The header is used by Distributed Messaging Network to distribute messages. It can be generated at its first publishing location. Information in the header should not be changed except the 'hop' number. Agent Repository

Users who are not authorized to publish messages to a Distributed Messaging Network, can register themselves in an agent repository 130. Users can publish their metadata to the agent repository 130. Then the agent repository can help convert the metadata to messages and publish the messages to a Distributed Messaging Network. Filters such as security filters can be applied in the agent repository 130 to avoid some security issues.

A validator is used in an agent repository. When a registered user tries to publish metadata to the agent repository, the validator will validate the metadata format and validate the external service that the metadata describes. If the external service that the metadata describes doesn't exist or cannot be validated, the agent repository can refuse to record this metadata for publishing.

A gather can be provided by the Agent Repository. Data or services can provide an interface where metadata can be fetched. For example, a web page can have its metadata in html header; a web service can provide a RESTful interface or a json file where metadata is nested in. Then users can register this interface to the gather in Agent Repository. The gather will automatically fetch metadata from these registered interfaces.

The Agent Repository can have different crawlers too. A web crawler can browser web pages from links nested in known web pages. Crawling from links to links, a lot of web pages can be founded. For every web pages, the crawler can generate its metadata with brief information. Different types of crawlers can be employed to generate metadata for different data or services.

Messaging Station

Messaging Station is a message kiosk which collects messages, caches messages and scatters messages. FIG.2 illustrates an implementation of a Messaging Station 141. This implementation of a Messaging Station 141 includes a collector 210(or a cluster of collectors), a processor 220(or a cluster of processors), a scatter 240(or a cluster of scatters) and a topology processor 230.

The collector 210 is used to collect messages. The collector 210 can actively subscribe other messaging system with subscriber 211 or actively fetch messages from other message sources with fetcher 212. Both subscriber 211 and fetcher 212 can be configured to work in a way with a filter to collect messages. The collector 210 can also passively accept messages published by others with receiver 213.

The collector 210 has a static collection list and a dynamic collection list. The static collection list is defined from configuration. The dynamic collection list is dynamically generated by topology processor 230.

The collector 210 can also send filter parameters to the message sources to only download filtered messages.

When a collector 210 collects a message, it will not only record the message's header and body. It will also record some attributes such as the source address of the message. These attributes will not be distributed to others. They are only used internally by the current Messaging Station.

In a Messaging Station 141, a collector cluster with more than one collector 210 can be deployed, where different collectors collect messages from different sources. In a collector 210, all of the subscriber 211, the fetcher 212 and receiver 213 can be configured to be enabled or disabled.

In a collector cluster, collectors 210 are managed by groups. For example, you can define one group to collect messages from US and another group to collector messages from Europe.

The processor 220 handles all collected messages before sending them to scatter 240. It includes a filter chain 221, a duplication checker 222 and a mapper 223. Different filters can be added to the filter chain, such as a lifetime filter to discard expired messages, a hop filter to discard messages reached maximum hops, an id filter to discard messages from some domain, a category filter or keywords filter, and so on. The duplication checker 222 is used to discard duplicated messages, in order to avoid message explosion. It will be described with a flow diagram. The mapper 223 is used to map channels/topics from the collector 210 to the scatter 240. Messaging Station 141 supports channels/topics, the channel/topic information is recorded as an attribute by the collector 210 and passed to the processor 220. The mapper 223 uses this attribute to map messages to different channel/topic in the scatter 240.

The queue between processor 220 and scatter 240 is also working to cache messages. Messages are cached here for others to subscribe or fetch. A queue manager is sitting by to clean old messages when the queue reaches its threshold.

A cluster of the processor 220 can be setup. A hash function is used to distribute messages to different processors 220. For example, a hash based on the message version in the header can be used. The hash function makes sure that duplicated messages are distributed to the same processor 220, so we don't need to worry about checking duplications between different processors. When adding one processor 220 to a cluster or removing one processor 220 from a cluster, there may be some duplicated messages during synchronizing time, but it's ok. The duplication checker is only to avoid message explosion because of duplications. A small amount of duplications is acceptable.

The scatter 240 is used to distribute messages. It contains a broker 241, a waiter 242 and a publisher 243. The broker 241 can be subscribed by subscription clients. The waiter 242 is a listening service where messages can be fetched. The publisher 243 can actively publish messages.

The scatter 240 can receive filter parameters from the client side. Then it will filter messages and only send filtered messages to client. Normally the filter parameters will be only applied on the message header.

Same as the collector 210, a scatter cluster with more than one scatter 240 can be deployed too. In a scatter 240, the broker 241, the waiter 242 and the publisher 243 can be configured to be enabled or disabled.

The messaging scatter 240 is designed to adapt different scales of messages. A small Messaging Station can actively publish messages to big Messaging Stations. Big Messaging Stations can also be subscribed by others.

In a scatter 240, the broker 241, the waiter 242 and the publisher 243 are organized in groups. Every group has a full copy of all cached messages. Workers in a group will get a share of a full copy of cached messages. For example, one broker 241 can have several groups and every group can have several workers. When a user subscribes to the broker 241, the user will be assigned to a group. Then it will return the full list of workers to the user. After that, the user client will subscribe to all these workers to get a full copy of messages. The collector 210 and the client api of Messaging Stations will include functions to handle groups.

A topology processor 230 is provided in the Messaging Station. The topology processor 230 caches a list of Messaging Stations. It periodically publishes messages to these Messaging Stations to detect topology structure around itself. When a topology processor 230 receives a message like this, it will parse this message and add its own information to the end of this message to generate a new message. The added information includes the ip address of the Messaging Station, the collector address and ports, and so on. The new message will not only be added to the messaging scatter 240 to be distributed to others; the new message will also be directly published to the requester (The request message includes information about the requester), it means the Messaging Station needs to send an answer to the requester, and at the same time relay the updated request message to other Messaging Stations. The topology processor 230 inserts a topo_filter handler to the filter chain 221. This to po_f ilter handler will catch these request and response messages and move them to the topology processor 230. The topology processor 230 will parse and analysis these response messages. In fact, every response message includes a routing path from itself to others. It will be able to build a connection topology graph. The topology processor 230 will select some Messaging Stations with less duplication paths as the collector 210's dynamic connection list.

Inside the topology processor 230, the mathematics of epidemics is applied to build gossip-like dynamic connection list. The purpose is to collect as many messages as it can. Different from basic gossip protocol, the topology processor 230 constructs trees to select the dynamic source nodes to collect messages:

a) At first, the topology process 230 builds a tree where static connected nodes are the roots.

b) Then the topology processor 230 builds another tree with the left nodes.

c) The top root or one of the top root will be selected as next dynamic connection node. d) Goto b) again until no left nodes or the maximum number of dynamic connected nodes reached.

During building the topology tree, if there is a loop or circle, the topology processor 240 will select one node in the loop/circle to represent the loop/circle. Then it removes all other nodes in the loop/circle from the tree.

Based on the topology response messages, the topology processor 230 can also generate filter parameters for different message sources.

FIG.3 shows the process of a duplication checker 222. It uses a two-level cache to record the message id and message version. The latest information of messages is cached in memory cache 302. Older information is cached in Level2 cache 303. Because the message version is in fact a timestamp, the message version is used to select which cache to use. The oldest cache timestamp 310 records the oldest message version timestamp in memory cache 302. Messages older than this timestamp will be record in Level2 cache 303. Too old information in Level2 cache 303 will be cleaned.

When a new message comes at block 301, if the message version is not older than the oldest cache timestamp 310, it will be compared within memory cache 302. Otherwise it will be compared within Level2 cache 303. Duplicated messages will be discarded at block 304. The message id and message version of a new message which is not recorded in the cache will be send to cache manager 307. The cache manager 307 will update the cache with new messages 308 and balance the space between memory cache 302 and Level2 cache 303. If memory cache 302 reaches its threshold such as memory limitation, cache manager 307 will move some information from memory cache 302 to Level 2 cache 303. The cache manager 307 will also clean too old information in Level2 cache 303.

A hash item can be added to the memory cache 302 and Ievel2 cache 303. The value of the hash item can be a hash number of the message id. The duplication checker 222 can compare the hash values before comparing the message ids. Only message ids with the same hash value will be compared for a new message. It will improve the performance of message id comparing.

Distributed Messaging Network

Connected Messaging Stations 141 can construct a Distributed Messaging Network 140. The Messaging Stations 141 can be connected using subscription, fetching or publishing functions. One message station 141 can be subscribed or fetched by others. At the same time, it can also actively publish messages to others.

FIG 4 is an example of a Distributed Messaging Network. In the example of the Distributed Messaging Network, a Messaging Station 141A is subscribed by a Messaging Station 141C. A Messaging Station 141B actively publishes messages to 141C and 141E. 141D subscribes to 141C. 141E fetches messages from 141C. In this example, messages published to 141A or 141B will go to 141C, 141D and 141E. As a result, the messages are broadly distributed.

FIG. 4 is just a simple example of a Distributed Messaging Network. In real situation, it will be more complicated„ Many Messaging Stations can be connected together without organization; routing loops can be very common. The same messages can be duplicated many times in routing loops. In a real situation, one user may publish the same messages to many Messaging Stations. As a result, huge number of messages is duplicated. In a Distributed Messaging Network, it's important to have a duplication checker 222.

The Distributed Messaging Network 140 employs mathematics of epidemics to build gossip-like network using gossip protocol. It can be extended to be more and more complicated and powerful. It is an open solution to construct a worldwide architecture.

Knowledge Repository

Similar with Messaging Station 141, a Knowledge Repository 150 collects messages too. But different from a Messaging Station 141, a Knowledge Repository 150 is not designed to distribute or relay messages. It's designed to collect knowledge. It will parse the collected messages and then store the metadata in the messages to a repository, where users can query to get information.

FIG. 5 shows the structure of a Knowledge Repository 150. It contains some modules which are the same with a Messaging Station 141, such as a collector 210(or a collector cluster), a processor 220(or a processor cluster), a topology processor 230. Differently, it contains a parser 510(or a parser cluster) to parse meta messages, a SQL or NoSQL database 520 to store meta information and a post service 530 to validate meta information.

The parser 510 includes a meta parser 511 and a filter chain 512. The meta parser 511 should include most of the popular namespace plugins. It will use a corresponding namespace plugin to parse a message. If there is no corresponding namespace plugin, it will use its parent or default namespace plugin to parse the message. The filter chain 512 can be configured to use different filters at different situations.

A SQL or NoSQL database 520 can be used as the storage of a Knowledge Repository. Metadata information will be recorded in the database. The metadata id is used to identify a metadata. The latest version should be recorded and other versions can be deleted or archived. The metadata lifetime is a way to distinguish whether it's still valid or not.

In Knowledge Repository 150, there is a post service 530 which can validate the metadata. Because the metadata information is published by clients, maybe some information is exaggerated to attract users. The metadata validator 531 is designed to validate it. It uses a possibility selector to randomly select some metadata to validate. The validator 531 will use the namespace plugin to validate the metadata (The metadata id or other URLs are used to locate the service which the metadata describes). If a metadata fails to pass the validator 531 (The validator fails to validate a metadata, or the difference between the metadata and the real service is too big), the validator 531 will increase the possibility to validate metadata from the same source or the same IP address, and decrease the rank of the metadata (The rank will be used to order metadata returned from a user query. It will not decrease the rank if it's not validated, even it has a high possibility to be validated). Metadata ids which fail to pass the validator will be published by Feedback Meta Publisher 532. Other knowledge repositories will parse these feedback messages and increase the possibility to validate these metadata. Users' feedback is another point to adjust the possibility to validate a metadata.

FIG.6 shows a fat Messaging Station based on Knowledge Repository 150. A scatter 240 (or a scatter cluster) is added to the Knowledge Repository 150. The scatter 240 queries SQL/NoSQL database 520 with filters to select messages to be distributed out. The fat Messaging Station can be used as an edge Messaging Station between private organizations and publics.

The Knowledge Repository provides some interfaces for users to access metadata, such as application programming interface (API), web interface for searching, catalogue list interfaces and so on.

There are many applications where the Knowledge Repository can be employed, such as search engines, service catalogues, yellow pages and so on. Here is an example on E-Commerce. Currently there are a lot of small E-Commerce web sites which are searching ways to promote their products. If a Knowledge Repository is deployed as an E-Commerce catalogue site which collects metadata from these small E-Commerce web sites and shows collected metadata like Alipay or Amazon. Customers can browser products or services from the Knowledge Repository E-Commerce catalogue site. When they are interested in some products or services, they will be redirected the real E-Commerce web site which the metadata describes. This E-Commerce catalogue service will promote a lot of small E- Commerce sites. In this model, the Knowledge Repository can be deployed with an Agent Repository which enables the gather and some functions of the crawler, without any messaging stations. If we extend the E-Commerce catalogue to different types of services, we can build different E-Service catalogues.

Distributed Meta Messaging Computing

FIG. 7 shows an example of the architecture of Distributed Meta Messaging Computing. In the example external service 701A uses publisher client 110 to publish metadata messages directly to a Distributed Messaging Network 140. External service 701 B registers metadata information to an agent repository 130 which publishes the metadata messages to the Distributed Messaging Network 140. External app 702 uses consumer client 160 to consume messages from the Distributed Messaging Network 140. External app 702 can get the URL from the metadata to access the external service 701A. In this graph, a Knowledge Repository 150 consumes messages from the Distributed Messaging Network 140. Users 170 can query the Knowledge Repository 150 to get attractive metadata. Then users can access the external services that the metadata describes from the URL in the metadata.

FIG.8 shows an example of deployment architecture of a Distributed Meta Messaging

Computing system. In this example, an open Distributed Messaging Network 140 is deployed publicly. A small organization 801A deploys an edge Messaging Station 141 A which partly serves local organization 801A and partly publishes messages to the open Distributed Messaging Network 140. A middle organization 801B has internal Messaging Stations such as 141B2 and an edge Messaging Station 141B1. A big organization 801C has internal Messaging Stations 141C2 and 141C3. It also has an internal Knowledge Repository 150C. Metadata is first assembled in the Knowledge Repository 150C, and then filtered and published to the open Distributed Messaging Network 140 through one public Messaging Station 141C1. For very small organizations such as 801 E, 801E and 801F, they can register metadata information to Agent Repository 130 which help publishes messages to the open Distributed Messaging Network 140.

In FIG.8, an edge Messaging Station is provided. It can be a normal Messaging Station 141 with public and private channels/topics, or with filters to block private messages. It can also be a fat Messaging Station described in FIG.6, where metadata is first collected in the SQL/NoSQL database 520. Then a filter can be applied to distribute selected metadata out through scatter 240.

As a matter of fact, it shall be noted that SSL(Secure Sockets Layer)/TSL(Transport Layer Security) protocols with host or user certificate validation can be applied on the connections between different components and deployments.

The above descriptions of the various embodiments of the present invention are presented for the purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Rather, the specific features and acts are disclosed as exemplary forms of implementing the invention. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments.

Claims

1. A method to broadcast messages in a distributed independent open network environment, wherein Messaging Stations are connected as a distributed open messaging network to relay messages, wherein the method comprises:

formatting a message with a header and a body with a publisher interface;

publishing messages to a Messaging Station with a publisher interface;

collecting messages with passive listener, active subscriber or active fetcher in a Messaging Station;

relaying or distributing messages from one Messaging Station to other Messaging Stations, message systems or applications;

consuming or retrieving messages from Messaging Stations with a consumer interface; and parsing messages.

2. A method as recited in claim 1, further comprising formatting messages with a header and a body, wherein the header is used to relay messages between different Messaging Stations, wherein the header includes a globally unique identifier and version number to identify a message uniquely, wherein the globally unique id and version number are used to discard duplicated messages to avoid messages explosion when broadcasting messages in a network environment with routing loops, wherein formatting messages comprises formatting original data to message body and formatting message header, wherein the message header comprises:

an item about the message id as a globally unique identifier to identify the message uniquely in a globally connected network environment, where the globally unique identifier can be a unique URI (Uniform Resource Identifier), a unique URL (Uniform Resource Locator), a GUID (Globally Unique Identifier) or some other unique global identifiers; and

an item about the message version, wherein the message version comprises an number, wherein the version number can be a timestamp number; and

an item about the message lifetime to indicate the valid time of the message; and

an item about the publisher of the message; and

an item about the category of the message body; and

an item about the keywords of the message body; and

an item about the checksum of the message body; and

an item about the format of the message body; and

an item about the current hop number of the message; and

an item about the maximum hops that the message can be distributed; and

other items;

further comprising a hash string of the message id, wherein the hash string can be compared before comparing the message id to identify whether the current message is unique.

3. A method as recited in claim 1, further comprising processing messages by parsing the header information in a Messaging Station to discard duplications and to do other message handling, wherein processing messages comprises:

parsing the header of the messages;

filtering the messages with a filter chain by filtering the header of the message;

discarding expired messages whose lifetime expires;

discarding messages which reach their maximum hops;

discarding duplicated messages with the same message id and the same version;

discarding duplicated messages with the same message id and an older version, wherein keeping only the latest version;

mapping messages from input topics or channels to output topics or channels.

4. A method as recited in claim 1, further comprising modifying message format to conform the consumer client in a Messaging Station.

5. A method as recited in claim 1, further comprising caching messages for some time in a Messaging Station like a news kiosk, wherein caching messages comprises:

caching processed messages for consumers;

caching processed messages for applications or other Messaging Stations;

managing cache with configured policy, wherein the configured policy comprises storage limitation, cache period limitation and other system and policy limitation, wherein caching time is managed by configured policy, wherein expired messages will not be cached.

6. A method as recited in claim 1, wherein collecting messages comprises:

receiving messages published by a publisher interface; or

subscribing to Messaging Stations, message system or applications to collect messages; or fetching messages from other Messaging Stations, message systems or applications; or a combination of operations listed above.

7. A method as recited in claim 6, further comprising a collecting cluster to collect messages, wherein the cluster is managed by groups, wherein every group gets a share of workload, wherein every worker in a group get a part of workload of a group.

8. A method as recited in claim 6, further comprising collecting messages to different topics or channels.

9. A method as recited in claim 6, further comprising collecting messages with filters, wherein the filter parameters are send to the message sources, wherein messages are filtered on the message sources, wherein the collecting client only downloads filtered messages.

10. A method as recited in claim 3, further comprising a processing cluster to process messages.

11. A method as recited in claim 3, further comprising a hash function to distribute messages with the same message id from all message collecting components to the same message processing component, to avoid discarding duplicated messages between different message processing components in a processing cluster.

12. A method as recited in claim 3, wherein discarding duplicated messages comprises: caching latest message id and version in memory cache;

caching older message id and version in Ievel2 cache, wherein Ievel2 cache can be a cache with slower reading and writing speed than memory cache, wherein Ievel2 cache can be a storage file or an application or something else;

selecting function to select one cache from memory cache and Ievel2 cache to compare for a new message;

managing cache with configured cache threshold, wherein managing cache comprises adding new message id and version to the cache, moving items between memory cache and Ievel2 cache, updating cache selecting parameter, discarding very old items in Ievel2 cache;

further comprising a hash item on message id to classify items in the memory cache and Ievel2 cache to improve comparing speed, wherein checking duplicated message ids by checking the hash item at first and then only comparing message ids with the same hash values.

13. A method as recited in claim 1, further comprising parsing messages to unformatted original data, caching unformatted original data instead of messages, filtering and processing unformatted original data instead of filtering and processing the header of messages, wherein filtering and processing unformatted original data comprises filtering private or secret data or service.

14. A method as recited in claim 1, wherein relaying or distributing messages comprises: publishing messages to other Messaging Stations, message systems or receiver applications; or brokering messages to subscribed Messaging Stations, subscribed message systems or subscribed applications; or

waiting other Messaging Stations, message systems or application to fetch messages; or a combination of operations listed above.

15. A method as recited in claim 14, further comprising a cluster to relay or distribute messages, wherein the cluster is managed by groups, wherein every group gets a full copy of all cached messages, wherein every worker in a group get a share of a full copy of all cached messages, wherein the worker can be publishing messages or brokering messages or waiting to be fetched, wherein one client will be assigned to a group when relaying or distributing messages to this client and all workers in this group will be returned to the client and then the client can connect to all these workers to relay messages.

16. A method as recited in claim 14, further comprising relaying or distributing messages to different topics or channels.

17. A method as recited in claim 14, further comprising relaying or distributing messages with filter parameters, wherein the relaying or distributing service can accept filter parameters from destination side, wherein the relaying or distributing service will send filtered messages to destination.

18. A method as recited in claim 1, further comprising a configuration management component, wherein the configuration management component comprising:

static configuration to collect messages from preconfigured Messaging Stations, message systems or applications;

static configuration to relay or distribute messages to preconfigured Messaging Stations, message systems or applications;

dynamic configuration to dynamically select active Messaging Stations, message systems or applications to collect messages;

dynamic configuration to dynamically select active Messaging Stations, message systems or applications to relay or distribute messages.

19. A method as recited in claim 1, further comprising topology processing to generate dynamic configuration for collecting and distributing messages, wherein the topology processing comprises:

publishing topology request messages to other Messaging Stations, wherein the topology request message comprises the requester's ip address, port, protocol information and so on;

filtering topology request messages from collected messages, moving topology request messages to topology processing and parsing the body of these topology request messages;

adding information about the current Messaging Station to this topology request message to generate a new information, wherein added information comprises ip address, connection ports, connection protocol and so on, so the new message comprises routing path of the current message; sending the new message as a response to topology requester;

sending the new message to the relaying component as recited in claim 14, to relay the new request message to other Messaging Stations; receiving topology response messages;

analysing topology response messages to get a routing path;

collecting all routing path to generate a topology tree;

using the topology tree to generate dynamic configuration for collecting and distributing messages;

configuring the configuration of dynamic collecting and dynamic distributing;

configuring the configuration of filter parameters to send to the message sources;

further comprising generating source-publisher filter table, wherein one row in the source- publisher table comprises the name or ip address of a message source directly connected to the current messaging station and a publisher name or ip address, wherein messages whose publisher and direct source match one row in the source-publisher table will be filtered, wherein the source- publisher table is generated based on duplicated routing path.

20. A Distributed Messaging Network comprising:

a publisher to format messages and publish messages to Messaging Stations, wherein the publisher comprises application programming interface;

Messaging Stations with network connections to receive messages from publisher, cache messages for a configurable period, relay or distribute messages from one Messaging Station to other

Messaging Stations, message systems or applications;

a consumer client to collect messages from Messaging Stations and parse messages to original data, wherein the consumer comprises application programming interface.

21. A Distributed Messaging Network as recited in claim 20, wherein the Messaging Station comprises:

a collector or a cluster of collectors to collect messages, wherein the collector comprises a receiver to receive messages from publishers, or a subscriber to subscribe to other Messaging Stations or message applications, or a fetcher to fetch messages from other Messaging Stations or message applications, or a combination of operations listed here;

a processor or a cluster of processors to parse the header of messages, filter messages with a filter chain, discard expired messages, discard messages reached maximum hops, discard duplicated or old messages, map messages between input channels/topics to output channels/topics and so on; a cache to cache processed messages for a configurable period for users to get these messages, including other configurable policies;

a scatter or a cluster of scatter to relay or distribute messages, wherein the scatter comprises a publisher to publish messages to other Messaging Stations or applications, or a broker to broke messages to subscribed Messaging Stations or applications, or a waiter to serve fetchers to fetch messages, or a combination of operations listed here;

a configuration manager to manage static configuration and dynamic configuration, wherein static configuration is loaded from the configuration file, wherein the dynamic configuration is generated at running time;

a topology processor to publish topology request messages to other Messaging Stations, filter and handle topology request messages, send topology response messages to the topology requesters, relay topology request messages to other Messaging Stations, receiving topology response messages, analysis topology response messages to generate dynamic configuration and filter parameters, and further generate source-publisher filter table.

22. A Messaging Station as recited in claim 21, further comprising parsing messages to unformatted original data, caching unformatted original data and processing unformatted original data; wherein the message processor will not only process message header, but also process or filter the content of message body.

23. A method to collect metadata into Knowledge Repositories, wherein a Knowledge Repository is used to collect and manage metadata and build catalogues for data or services with metadata that describes the data or services, wherein the method comprising:

standardizing metadata that provides meta information about other data or service;

collecting metadata and assembling metadata to a Knowledge Repository;

displaying or querying metadata information in a Knowledge Repository;

locating and accessing the data or service that the metadata describes through the URL (Uniform Resource Locator) in the metadata.

24. A method as recited in claim 23, wherein a namespace is used to adapt different types of metadata, wherein standardized metadata comprising:

an item about namespace URL to direct the parser to parse the metadata, wherein the namespace comprising a web available specification and a URL (Uniform Resource Locator) to locate the namespace specification, wherein new namespace specification can be defined or extended from available namespace specification;

further comprising an item about globally unique metadata id to identify the metadata in a global distributed environment, where the globally unique identifier can be a unique URI (Uniform

Resource Identifier), a unique URL (Uniform Resource Locator), a GUID (Globally Unique Identifier) or some other unique global identifiers;

further comprising a hash string of the metadata id, wherein the hash string can be compared before comparing the metadata id to identify whether the current metadata is unique;

further comprising an item about version with number to identify different versions of metadata, wherein newer version should overwrite the old version, wherein version number can be a timestamp;

further comprising an item about lifetime to identify whether the metadata is still valid;

further comprising an item about category to classify the service that the metadata describes; further comprising an item about keywords to describe the key points of the service that the metadata describes;

further comprising an item about abstract to describe the outline of the service that the metadata describes;

further comprising an option item about default namespace to be used when current namespace cannot be identified;

further comprising an option item about subcategory to be used as a supplement of the category item;

further comprising URLs(Uniform Resource Locator) to locate the data or service that the metadata describes;

further comprising other items defined in the namespace specification;

further comprising using one of the globally unique URLs(Uniform Resource Locator) which are used locate the data or service that the metadata describes as the global unique metadata id;

further comprising items of URLs(Uniform Resource Locator) to link other data or medias.

25. A method as recited in claim 23, wherein collecting metadata comprising:

gathering metadata from registered interface, wherein data or service can provide an interface where metadata can be retrieved, wherein these interfaces can be registered in the gathering method, wherein the gathering method will automatically access these interfaces to retrieve metadata; or crawling internet to find data or service and then generate and record metadata for these data or service; or

a combine of the methods above.

26. A method as recited in claim 23, further comprising an E-Service catalogue, wherein the E-Service catalogue comprising: displaying metadata by grouping similar data or service together, wherein different filters and order solutions can be applied;

27. A method as recited in claim 23, further comprising broadcasting metadata to messaging systems like Distributed Messaging Network and collecting metadata from messaging systems, wherein the method comprising:

formatting metadata as messages and publishing messages to messaging systems like Distributed Messaging Network;

delegating to format user metadata as messages and publishing messages to messaging systems like Distributed Messaging Network;

relaying or distributing metadata messages in messaging systems;

collecting and processing messages from messaging systems, parsing messages to metadata and assembling metadata to a Knowledge Repository.

28. A method as recited in claim 27, wherein formatting metadata as messages comprises: converting metadata id to message id item in the message header;

further comprising converting metadata version to message version item in the message header; further comprising converting metadata category to message category item in the message header;

further comprising converting metadata keywords to message keywords item in the message header;

further comprising converting metadata lifetime to message lifetime item in the message header;

further comprising formatting metadata and filling it to the message body, filling the format type to the message format item in the message header to direct receivers to parse the metadata.

29. A method as recited in claim 27, wherein delegating to format user's metadata and publish metadata as a message to messaging systems like Distributed Messaging Network comprises: receiving metadata from users;

validating the metadata to make sure that the data or service which the metadata describes exists, to make sure that the metadata is valid and compatible with the namespace specification, and to make sure that the metadata is consistent with the data or service which the metadata describes; filtering metadata to avoid security issues and to avoid breaking other policies;

formatting metadata to a message;

publishing messages to messaging systems;

further comprising storing received metadata in a database;

further comprising automatically renewing metadata by accessing the data or services that the metadata describes and publishing renewed metadata to messaging systems;

further comprising automatically collecting metadata with registered data or service, wherein data or service provides an interface to get metadata, wherein the interface is registered or collected in this delegating service, wherein this delegating service automatically access the interface to collect metadata;

further comprising crawling internet to find data or service and then generate and record metadata for these data or service.

30. A method as recited in claim 27, wherein collecting and processing messages from messaging systems, parsing messages to metadata and assembling metadata to a Knowledge Repository comprises: collecting and processing messages with methods as recited in claim 3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19;

parsing processed messages and converting messages to metadata, wherein namespace item is parsed from the message body at first and then namespace plugin is loaded to convert the message body to metadata;

filtering metadata;

storing metadata in a database, wherein the database further comprising SQL database or NoSQL database, wherein metadata id is used as a direct or indirect key parameter to identify metadata in the database;

querying or retrieving metadata from databases.

31. A method as recited in claim 23, further comprising retrieving metadata from the Knowledge Repository, filtering metadata, formatting metadata to messages and publish messages to another messaging system, wherein this Knowledge Repository can be a fat Messaging Station where processing metadata instead of messages, wherein the Knowledge Repository can be an edge messaging repository where it collects all messages from intranet and then retrieves public information from the Knowledge Repository to publish to internet.

32. A method as recited in claim 23, further comprising post service to validate the metadata, wherein different knowledge repositories have different chances to validate different metadata, wherein metadata ids failed to pass the validation are distributed to connected Messaging Stations or messaging applications, wherein the post service comprises:

selecting metadata to validate, using a priority-based possibility selector to randomly select some metadata to validate;

validating the metadata to make sure that the data or service which the metadata describes exists, to make sure that the metadata is valid, and to make sure that the metadata is consistent with the data or service which the metadata describes;

managing the rank of metadata, to decrease the rank of the metadata to be retrieved if metadata fails to be validated, wherein the rank is used to order metadata when querying or retrieving. managing the validation priority of metadata, to increase the possibility priority of the metadata from the same source or publisher if metadata fails to be validated;

managing the validation priority of metadata, to increase the possibility priority of the metadata if a user has negative comments;

publishing feedback messages with metadata ids which fails to pass the validation to messaging systems;

receiving and parsing feedback messages from messaging systems, increasing the possibility priority to be validated for metadata listed in the feedback messages.

33. A method as recited in claim 23, further comprising renewing metadata when displaying or retrieving metadata, wherein it comprises:

retrieving metadata from database when displaying or retrieving metadata;

renewing metadata before showing the metadata to users, by accessing the data or service that the metadata describes;

displaying or returning renewed metadata to users.

34. A Distributed Meta Messaging Computing comprising:

a publisher to standardize metadata, format metadata as a message and publish messages to messaging systems like Distributed Messaging Network, wherein the publisher comprises application programming interface; an agent repository to collect metadata, wherein collected metadata can be transferred to a Knowledge Repository directly, wherein the agent repository can also publish metadata as messages to messaging systems;

a messaging system like Distributed Messaging Network to collect messages, cache messages, relay or distribute messages;

a consumer client to consume messages from messaging systems and convert messages to metadata, wherein the consumer comprises application programming interface;

a Knowledge Repository to store metadata, wherein the metadata can be transferred from an agent repository, wherein the metadata can be also converted from messages which are from messaging systems.

35. A Distributed Meta Messaging Computing as recited in claim 34, wherein an agent repository comprises:

a gather to gather metadata, as recited in claim 25;

a crawler to crawl internet to generate metadata, as recited in claim 25;

an interface to receive metadata from users, wherein the interface comprises application programming interface;

an validator to validate the metadata to make sure that the data or service which the metadata describes exists, to make sure that the metadata is valid and compatible with the namespace specification, to make sure that the metadata is consistent with the data or service which the metadata describes and so on;

a filter chain to filter metadata to avoid security issues and to avoid breaking other policies; a list of namespace plugins to handle namespace;

a formatter to standardize metadata with namespace plugins and convert metadata to a message;

a publisher interface to publish the message to messaging systems like Distributed Messaging Network;

further comprising storing received metadata in a database;

further comprising an interface to transfer metadata to other systems like a Knowledge Repository;

further comprising automatically renewing metadata by accessing the data or services that the metadata describes and publishing renewed metadata to messaging systems like Distributed Messaging Network;

further comprising automatically collecting metadata with registered data or service, wherein data or service provides an interface to get metadata, wherein the interface is registered or collected in this delegating service, wherein this delegating service automatically access the interface to collect metadata.

36. A Distributed Meta Messaging Computing as recited in claim 34, wherein a Knowledge Repository comprises:

a gather to gather metadata, as recited in claim 25;

a crawler to crawl internet to generate metadata, as recited in claim 25;

a retriever to retrieve metadata from other systems like an Agent Repository;

a list of namespace plugins to handle namespace;

a parser or a cluster of parsers to parse the messages with namespace plugins and convert the messages to metadata;

a filter chain to filter metadata; a database to store metadata, wherein the database can be SQL database or NoSQL databases; an interface for users to query or retrieve the metadata, locate and access the data or service that the metadata describes, wherein the interface comprises application programming interface. further comprising a processor or a cluster of processors to parse the header of messages, filter messages with a filter chain, discard expired messages, discard messages reached maximum hops, discard duplicated messages.

further comprising a configuration manager to manage static configuration and dynamic configuration.

further comprising a topology processor to publish topology request messages to other Messaging Stations or message applications, filter and handle topology request messages, send topology response messages to the topology requester, receiving topology response messages, analysis topology response messages to generate dynamic configuration.

further comprising a post service to validate the metadata, wherein the post service is recited as claim 32;

further comprising renewing metadata when querying or retrieving metadata, wherein renewing metadata is recited as claim 33.