WO2001052112A1

WO2001052112A1 - Man-machine interface method and apparatus

Info

Publication number: WO2001052112A1
Application number: PCT/US2001/001097
Authority: WO
Inventors: Adrian Gluck
Original assignee: Verbal Communications Technologies, Llc
Priority date: 2000-01-11
Filing date: 2001-01-11
Publication date: 2001-07-19
Also published as: AU2001229425A1

Abstract

A method and apparatus that allow a user to form semantically correct utterings in order to search the Internet, the web, databases, or to control target devices. The method directs the user in the construction of a semantically correct uttering by presenting the user with a list of possible verb choices. Once the user chooses an appropriate verb, the method presents the user with at least one list of possible nouns or objects that correspond to the chosen verb. Using the semantically correct uttering formed by the user, the method directs the user to an appropriate web site, Internet location, database record, or any other appropriate target. A universal controller system is designed to implement the method in order to control various target devices. The controller system allows the user to scroll amongst various verb and object choices using a selection apparatus. After the user forms a semantically correct uttering using the system, the controller sends an appropriate uttering to the target device in order to effectuate a function on the target. The controller may send utterings to the target via a wireless data transfer unit or a direct data transfer unit.

Description

MAN-MACHINE INTERFACE METHOD AND APPARATUS

FIELD OF THE INVENTION The present invention relates generally to a method and apparatus for using semantically correct utterings to improve interactions and communications between users and computer controlled systems. Specifically, the present invention relates to an improved method to conduct web site, Internet, and database searches by using semantically correct utterings to more accurately direct users to specific sites or records. Also, the present invention is directed to a man-machine interface system that enables users to more easily communicate with computer-controlled systems. The present invention is additionally directed to a remote controller system that incorporates the method for use in a multitude of human-machine communications.

BACKGROUND

Computer-controlled systems are growing in importance as computers are increasingly assisting human activities. Correspondingly, it is increasingly important for the interface between humans and computers to be efficient and effective.

I. Human-Machine Communications

Humans communicate by generating utterings that are understandable by other humans, by other living organisms, and by appropriately programmed computer-controlled systems. In a very general sense, utterings may take the form of speaking, writing, drawing, typing, pointing, body language, sign language, touching and bio-signaling. In a human-machine interface, a user generates utterings to communicate with a machine. Typical utterings in a human-machine interface context may include queries, commands, and requests.

Information Retrieval/Human-Machine Communications on the Internet

One of the fastest developing areas utilizing human-machine interfacing is the Internet and the World Wide Web. Web browsers and search engines are used in order to access web sites and other Internet-related information. However, current browsers, search engines, and web site indexing systems suffer from a number of shortcomings.

Retrieval strategies are transparent to the user

Efficient and accurate search techniques become more important as the amount of information available on the Internet increases. Many retrieval systems rely on user- transparent formulas that render access to web sites increasingly difficult. When users have no knowledge of the formulas used to generate a search, they are not as able to modify their search strategies in order to take advantage of an engine's desirable features or to avoid undesirable features. Consequently, there is a need for a system or method for a human-machine interface, in the context of web or Internet searches, that allows the user to see the search or uttering being generated at each step.

Information overload

When a user accesses web sites using one of the currently available search engines, the search can produce a large number of irrelevant results. For example, when searching for information on a home "pet", a typical search engine returns well over 500 web pages. Many of these pages do not pertain to the original search request. These pages contain references to the PET electronic imaging systems, a plastic resin, and domestic animal pets. This information is often too much to review with any degree of certainty that the information selected is relevant. Consequently, there is a need for a system that allows a user to generate a more accurate search uttering.

Lack of standards for displays of search results

Another problem encountered when attempting to search for information on the Internet is the difficulty of reading and understanding the information presented. This is due in part to the fact that many of the display layouts are not consistent with normally used standards for presenting written information.

Another problem is the lack of a standard for presenting the topic of a writing so that it becomes instantly understood by the reader. These problems result in increasing the time taken to select the information returned by the system and misunderstandings as to the nature of the information returned. As a result, there is a need for a system that may be used in various search engines and search systems to present a standard display of results that allows the user to more easily understand the results. Inability to locate specialized web sites

Many special interest web sites exist that address the problem of searching for a specific item from among a grouping or class of similar items. Using special interest web sites can speed up the process of locating information of interest once such specialized web sites can themselves be located. The user, unfortunately, faces the same problems when trying to find these sites as when trying to find any other information using today's Internet search facilities. Accordingly, there is a need for a system to direct a user to a particular specialized web site through the use of a search uttering.

Short comings of current search techniques

There is a multitude of techniques that current search engines present to users in order for the users to formulate search requests, queries, or command utterings. Unfortunately, each current search technique is deficient in one or more areas.

Boolean operations using mathematical operators such as "AND", "OR", "NOT", "NEAR" are made available by most search engines. The use of such Boolean operators improves the relevance of the responses if a user is familiar with their use. However, some users may not be entirely comfortable using Boolean operators since the Boolean commands do not perfectly mimic natural language or human semantics.

Free-form utterings require entry of data on a keyboard, a task which may be error- prone and time consuming for non-experienced typists. Another problem is that the user is not given any guidance and thus may form search requests that are either poorly understood, totally misunderstood, or even ignored by the target computer-driven system.

Question-and-answer interfaces require a large amount of time to form complete and meaningful utterings. Furthermore, as the human responses are keyed in free-form style, it is difficult for the user to know what the appropriate responses are, other than being told in slow step-by-step fashion by the system with which the user is communicating.

With fill-in forms interfaces, the user keys in data in the areas indicated on the form. While pre-formatted forms provide more guidance than free-form input, it is time- consuming for the user to select the appropriate form to use, as it is not uncommon for a system to contain hundreds of possible forms, each one corresponding to possible utterings. Another problem is that programming changes to such forms is generally costly and time-consuming. Selectable graphic icons have been used as a means of communicating with computers, as seen in the Macintosh" and Windows" operating systems. These interfaces are very good in those instances where the uttering to be formed parallels a real-world activity that involves an easily recognizable object. For example, commanding a computer to delete a file can be likened to the real- world action of throwing a folder into a garbage can. The problem with this type of interface is that the number of physical actions that have clearly-associated target objects is limited. In addition, using graphic icons to represent all of the possible objects and actions used in everyday language would be extremely time and space-consuming for the machines and unwieldy for the users.

II. The Importance of Syntax and Semantics

All of today's human-machine communication interfaces suffer from the effects of ineffectively handling the syntax and semantics inherent in any communications. The method and related apparatus of the present invention focuses on the formation of semantically correct utterings in order to increase the utility of human-machine interfaces. For the purposes of the present invention, semantically correct utterings are those utterings that stress the use of verbs, correct verb and noun orientation, syntactic structuring, and semantic disambiguation.

Verbs create "meaning" and "familiarity" in utterings

Today's computer databases and information collections are primarily indexed based on noun/objects. This requires currently used query-type utterings to use primarily nouns and adjectives in describing objects in their queries. For all languages this presents a problem in that, to be fully understandable, most utterings require at least one noun to represent the object of an action, and one verb to represent the action. The verb sets up or projects a number of thematic roles, which must then be filled by an appropriate number of nouns. There are also significant differences between nouns and verbs. Verbs determine the basic number of nouns needed in an uttering. Nouns, on the other hand, demonstrate nothing about tense and imply nothing about the semantic structure of any phrase. Verbs generally bring much more of the meaning of an uttering than the obj ect of the uttering. In fact, utterings can exist with only verbs/actions and no noun/objects present, but the reverse is not true. "I skill is a meaningful uttering while "Skill is not. Without the explicit meaning imparted by verb/action words, the responses returned are

necessarily broad in scope, excessive in number, and mostly irrelevant vis-B-vis the query

posed.

Additionally, natural languages often show deviations with respect to the normal use of noun/object keywords in information retrieval systems because of two factors: meaning and familiarity. Very often noun/object words assume additional or different meanings in various contexts. For example, the noun/object "table" means "a place to eat on" in the "to eat" verb/action context, and "a collection of data" in the "to analyze" verb/action context. Consequently, initiating an uttering with a verb/action is preferable to initiating an uttering with a noun/object. Starting an uttering with a verb or action prevents ambiguities in the uttering formed by the user. Familiarity with a term is also important because terms, whether noun/objects or verb/actions may be very familiar in one domain, and unfamiliar in others. The verb/action to implement, for example, is considered rare in standard English but is very familiar in the management and military sciences.

Word existence and word order In general, nouns and verbs do different things semantically. In commonly constructed English utterings, the verb/action precedes the noun/object.

Many of today's Internet portals attempt to categorize information by domains of interest, such as "Art and Culture", "Sports", "World", "Horoscope", "Weather" and "Fashion". Unfortunately, their implementation presents a series of problems: (a) the domains offered have confusingly different orientations: some are oriented around a verb/activity (for example "Sports"), while others are oriented around a noun/object (for example "Weather" and "Horoscope");

(b) the domains are too broad to be meaningful for more specific searches in certain cases;

(c) the average user may not completely understand the organization of the list, thus requiring the user to scan the entire list of domains in order to make a selection.

Polysemous words

One other problem with the current indexing systems is their difficulty in handling polysemous words. These are words that have multiple meanings. A search strategy that simply uses string matching to select information will likely return many irrelevant hits due to the multiple meanings of many words. The irrelevant sites and pages are those that contain the word used in a sense different from that intended by the user.

Synonymous words

Another problem is that presented by synonymy, where many equivalent or closely related meanings can be conveyed by the same distinct words. Once again, any indexing strategy that simply uses string matching to select documents is likely to miss many relevant sites. The sites missed are those that contain different words expressing similar meamngs to those in the query. This problem has been traditionally tackled by indexing and information retrieval systems by using thesauri to expand query or index terms to permit broader matching. Unfortunately, the currently available thesauri group words based on the common language use and not by the specialized scopes of typical computer- controlled systems.

Foreign Language Speakers

The Internet and many computer-controlled systems are set-up for use by English- language speakers. Foreign-language speakers may find it difficult to use such Internet- based and computer-controlled systems if they do not know the appropriate English keywords. Furthermore, replicating the man-machine interfaces for other languages is an expensive and time-consuming task. Many languages have a similar semantic and syntactic structure to English, including the use and placement of verbs and nouns and the use of plurals and singulars. Thus, an interface is needed that could be utilized to aid foreign language speakers in using these systems that are generally geared towards the English language.

III. Coordination among multiple computer-controlled systems Increased numbers of computer-controlled systems have the ability to interface directly or indirectly with other computer-controlled systems. It has been hoped that this would reduce user time spent repeatedly entering the same information. Another advantage of synchronization is that users can transport their data from one system to another with greater ease. Unfortunately, the same problems outlined above apply to this area. Additionally, a new variable is introduced: inconsistent, non-standard hardware interfacing, forcing users to remember different protocols. As a result, although technologies such as inf ared transmission, radio frequency transmission, and docking stations have been available for a long time, little use is being made of them. Although the typical family operates over forty computer-controlled systems daily, only four major applications have emerged: (a) schedule synchronizing between appropriately equipped personal computers and personal digital assistants, (b) home entertainment remote control units, (c) garage door openers, and (d) car alarm systems. The biggest reason for this failure to adopt time and effort savers is the difficulty to master the "programming" required to set them up. Consequently, there is a need for a universal controller device that implements a semantically correct uttering system of the present invention in order to allow users to control various devices using the same easily understandable methodology. IV. Conclusion

Due to the aforementioned problems, there is a need for an improved system for human-machine interfacing, particularly in the fields of Internet, web, and database searching. The search process should be visible to the user at each step, allowing the user to more accurately construct a search request, a query, or a command uttering. The system should allow the user to construct an uttering that will generate a more accurate search. The system should be compatible with various search engines and search systems in order to allow the presentation of uniform results displays to the user. The system should be able to direct the user to appropriate specialized sites or records via the uttering constructed using the system. To accomplish the aforementioned goals, the system should implement correct semantic structure and verb-noun word order to increase the users understanding of the resultant search, query, or command. Additionally, there is a need for a device that implements the system in order to allow users to operate various other devices and systems using the same semantically correct uttering system of the present invention. This would enable users to operate various devices while using the same understandable, semantically correct uttering system.

DESCRIPTION OF THE RELATED ART Heretofore, various systems and methods for searching the Internet or other databases have been proposed. Examples of some of these methods and related methods are disclosed in the following U.S. Patents:

U.S. Patent No. Patentee

5,309,359 Katz, et al.

5,404,295 Katz, et al. 5,404,506 Fujisawa, et al.

5,421,008 Banning, et al.

5,701,456 Jacopi, et al.

5,715,449 Peters, Jr., et al. 5,781,247 Wehmeyer, et al.

5,798,757 Smith

5,909,678 Bergman, et al.

5,913,215 Rubinstein, et al

5,920,854 Kirsch, et al. 5,924,090 Krellenstein

5,933,822 Sraden-Harder, et al.

5,935,060 Iliff

5,946,678 Aalbersberg

Although these patents describe various methods for searching databases and web sites, there remains a need for a method that refines the user's initial search, thereby eliminating erroneous results at the onset of the search request. The search request should be based on a semantically correct uttering, thereby increasing the user's natural familiarity with any given request. The search request should also focus on verbs and actions, rather than nouns or objects, and word order in forming the uttering. Some retrieval and access systems assign relative significance scores to the web sites or the database records being accessed. The score may depend on the frequency with which certain teπns appear within the web site or web page. For others the score depends on the number of accesses or "hits" received by a certain site or page. Others score depending on where the search terms appear. For example, a high score is given to web sites that contain the term in their home page itself. An example of this type of system is described by U.S. Patent No. 5,920,854, which proposes a search system that includes assigning relevancy scores to the results of a search. However, because of the scoring methods employed, web searches generally present information whose relevance is highly variable, thus undermining a user's ability to find information on the web.

U.S. Patent No. 5,924,090 describes a method and apparatus for searching a database of records by organizing results of an initial search into a set of most relevant categories. The user is then presented with the categories along with a list of the most relevant records found. However, the patent proposes a method that does not aid the user to form an initial search. Without a well refined initial search, the results from the search may not be as helpful to the user as the results from an initial search that fully reflects the user's request. Consequently, there is still a need for a method that allows a user to form a semantically correct search to more accurately find information in a database or on the web. Ask Jeeves and U.S. Patents 5,309,359 and 5,404,295 describe search and retrieval methods which attempt to match queries keyed in long hand with a pre-determined set of possible questions or annotations. The user does not know the contents of the predetermined set of questions or annotations. Since users are not guided in how they construct the queries, these works first parse each query and then try to match it to the most similar pre-stored query or annotation. Because of the great deal of processing required, in order to respond to the user in a reasonable time frame, the pre-determined set that must be matched has to be kept to an optimum mimmum size. This in turns reduces the probability of an exact match between the query posed and those available for answer, and increases the probability of choosing "the next best" response, which could be totally irrelevant.

Another way of accessing web sites is by using an indexing system, such as Yahoo!. These systems use human editors to manually index web sites into a set of pre- determined categories. Only web sites meeting certain criteria are included. This approach greatly improves the relevance of the sites retrieved by the user. However, the user is faced with the new problem of finding the category that is most likely to contain the topic of interest. Consequently, there remains the need for a method to improve the initial search that the user generates and to direct the user to a specific site or record based on the formed search uttering.

SUMMARY OF THE INVENTION The present invention is directed to a method that enables a user to form semantically correct utterings in order to search for records in a database or information on the web. The utterings may be in English or in a non-English language with similar semantic and syntactic structure. The method enables the user to form semantically correct utterings by emphasizing correct verb to noun placement and by eliminating ambiguities presented by polysemous and synonymous words. The present invention is additionally directed to a system for a portable universal controller device that incorporates the method. The system enables a user to control various maclunes through the use of semantically correct utterings formed by the system.

To use the method to search for information in a database, on the web, or on the Internet, the user will interact with a user interface that directs the user in the construction of a semantically correct uttering. In one embodiment, the interface is contained within a pull-down menu format.

The interface first presents the user with an initiation phrase. The user will utilize the initiation phrase in order to begin constructing the uttering. Building upon the initiation phrase, the user will select a desired verb from a master list of verbs. The user will select a verb to describe the action that the user desires to perform. hi one embodiment, the user will next select a noun or object from a list of available nouns or objects that are associated with the previously selected verb. The user will select at least one object to associate with the verb but the method may present the user with more than one level of obj ects from which to choose. The number of levels of objects will vary depending on the specific task the user intends to perform. Additionally, if there is more than one object level, the method will present each level sequentially, one at a time, to the user, h an alternative embodiment, the method may form an uttering based solely on a verb choice, without a subsequent object choice, if the desired task is simple enough in nature to be performed with only a verb.

The verbs and objects within each list may be organized alphabetically through the use of letter indices in order to better present the verbs and objects to the user. In this embodiment, only the uttering components within a selected alphabetical index will be presented to the user at any one time. To solve the problem of synonymous or polysemous words complicating a user's desired activity, the method, in one embodiment, presents the user with synonymous verbs and synonymous nouns. These synonyms will direct the user to use one verb or noun as a substitute for all synonyms of that verb or noun. The method may also incorporate a user definition subsystem that allows the user to define the user's identity, enabling the method to tailor the components and the target results to the user. Using the definition subsystem, the user will be able to specify both the type of user and the level of ability within the type of user. Once a semantically correct uttering is formed by the user, via the interface described above, the method will direct the user to a target based on the formed uttering. For example, the target may be a record in a database or a web site. The method may further include a coding subsystem to facilitate the compilation of targets and direction of users to those targets. The coding subsystem assigns a code to each available uttering component. When a complete uttering is formed, the individual codes of the components are read together in order to form a unique identification number. The resultant number may be used to more easily categorize targets and direct users to those targets.

In one embodiment of the system, the method is embedded in a web browser. This may be accomplished, for example, by placing the components of the method inside a browser's bookmarks or favorites file. In another embodiment of the method, the method is installed within a search engine or an indexed 'directory. In this embodiment, the user will use the method in order to form semantically correct utterings that allows the user to form more refined search commands.

Another embodiment of the method allows the user to create a series of scripts that may be used to generate a series of utterings automatically. The user will initially create a script comprised of a series of uttering. The script is then added to the list of possible objects for an uttering. The user may then run the script at a later time by choosing the particular script as an object for an uttering. The present invention is also directed to a universal controller system that implements the aforementioned method to allow the user to control target devices using semantically correct utterings. The system is contained within a compact housing that is designed to allow the system to be portable. A program unit located with the housing operates the method that enables the user to construct semantically correct utterings. To allow the user to interact with the program unit, a selection apparatus is provided. The selection apparatus allows the user to select from amongst the available verb and object components in order to form a semantically correct uttering. In one embodiment, the selection apparatus is comprised of selection actuators that the user utilizes to select the uttering components. Alternatively, the system may also utilize any combination of a touch screen, selection actuators, or a microphone to enable the user to select verb and object choices.

Once a semantically correct uttering is formed, the controller system transmits the uttering to a target device in a form that is compatible with the target device. In one embodiment, the system transmits the uttering via a wireless data transfer unit. In another embodiment, the system transmits the uttering via a direct data transfer unit.

It is an object of this invention to provide a more accurate and easier-to-use human-machine interface using semantically correct and complete utterings, closely paralleling natural language utterings. It is a further object of this invention to assist users by giving them a comprehensive view of their progress in constructing utterings, by displaying the possible verbs and nouns/objects available for the most current entry selection. This object may be furthered through the use of rapid pop out menus. It is a further object of this invention to provide a faster, more accurate and more efficient method for indexing large amounts of information stored in multiple databases and/or at multiple locations, such as the Internet.

It is a further object of the present invention to reduce the amount of typing of information in order to minimize the time required to form utterings, to reduce the necessity for possessing good typing skills, and thus significantly reducing the impact of typing errors.

It is a further object of this invention to minimize the time taken by the user in constructing an uttering by only offering as choices within menus those verbs and nouns/objects that can be logically associated with the immediately preceding choice.

It is a further object of this invention to resolve meaning conflicts when using polysemous words and the problem of not retrieving information indexed around polysemous words.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the method and apparatus, and to show that it may be carried into effect, reference will be made, by way of example, to the accompanying drawings, in which:

FIG. 1 is a schematic illustration of a preferred embodiment of the method to form a semantically correct uttering to search the Internet, the web, or a database.

FIG. 2 illustrates a typical semantically correct uttering formed through the use of the method with corresponding targets.

FIG. 3 illustrates an alphabetical index system that may be used to categorize uttering components within the method. FIG. 4 illustrates a user definition subsystem that may be utilized to tailor the method to a specific user profile.

FIG. 5 illustrates an embodiment of the method wherein the uttering components and the uttering are encoded with unique identification numbers. FIG. 6 illustrates the exterior of a portable universal controller system that utilizes the method.

FIG. 7 is a schematic representation of the components of an embodiment of the controller system.

FIG. 8 is a flow chart illustrating the operation of a portable universal controller system.

DETAILED DESCRIPTION OF THE INVENTION Turning to FIG. 1, the method first presents a user with an initiation phrase 1 that represents the start of the semantically correct uttering that will be used to communicate with the computer-controlled system, perform a search, give a command, or pose a query. The initiation phrase 1 will generally be presented to the user as an implied action such as "I want to." In addition to representing the start of the uttering, the initiation phrase 1 establishes a role for the user that allows the user to better understand the formation of the uttering. For example, the initiation phrase 1 "I want to." implies that the user wishes to command a computer-controlled system to perform a task.

Next, the user will select a verb from a list of verbs 2 that the method presents to the user. The list of verbs 2 is a master list of volition verbs or action words that are applicable to the specific application to which the method is being applied. In one embodiment, the list of verbs 2 will contain synonyms and polysemous variants of verbs that will direct the user to use one particular verb for all the synonyms and polysemous variants of that verb.

In a preferred embodiment, after choosing a verb, the user will be presented with a list of phrases 3 from which the user will choose a phrase that will serve as the target of the verb aheady chosen by the user. The list of phrases 3 will contain nouns or objects that are associated with the verb that the user previously selected from the list of verbs 2. As with the list of verbs 2, the list of phrases 3 may also contain synonyms and polysemous variants of the phrases that will direct the user to use one particular phrase for all the synonyms and polysemous variants of that phrase. The method presents the list of verbs 2 to the user prior to the list of phrases 3, thereby maintaining semantically correct word order between the verb and the noun or object.

In another embodiment, the method presents the user with lists of sub-phrases 4 for further modifying and refining the semantically coπect uttering already formed by the combination of the verb and the phrase. There may be multiple levels of sub-phrases from which the user may select in order to construct the semantically coπect uttering. After the user selects a verb and a phrase to construct the semantically coπect uttering, this embodiment of the invention presents the user with further levels of sub-phrases that modify the semantically coπect uttering. As with the list of verbs 2 and the list of objects 3, synonyms and polysemous variants of sub-phrases may be added to the lists of sub- phrases 4 to ensure that the user forms the most accurate uttering possible.

In all of the embodiments of the method, the method will direct the user to a target destination or list of targets 10, which may be web sites or database records, based on the combination of the verb and the phrase chosen by the user. If the uttering is very simple, the method may direct the user to a target or target list 10 based only on a selected verb, without further selections of objects/nouns. The combination formed by the user will be semantically coπect, thereby increasing the likelihood that the user will find the information that the user initially desired. The emphasis on forming the semantically coπect uttering with a verb, initially, also increases the likelihood of the user forming an effective uttering.

An example of a completed uttering with a resultant list of target destinations is illustrated in FIG. 2.

The list of target destinations 10 may be constructed manually by operators who locate appropriate web sites or records in order to generate and update the list periodically. This may be accomplished through the use of a preexisting search engine, for example. Alternatively, the list of target destinations 10 may be constructed through the use of a web crawler or database crawler that automatically generates and updates the list based on certain keywords or metatags located within the target web sites, Internet locations, or records. Turning back to FIG. 1 , the method will preferably utilize a series of quickly cascading menus which automatically pop-out or pull-down after each uttering component is selected, enabling the user to select the next uttering component needed to construct the uttering. This will enable the user to construct utterings using a generally continuous motion, such as by using a mouse. An embodiment of the method, illustrated in FIG. 3, utilizes alphabetical letter indices within each list presented to the user. The letter indices improve the organization of the lists of verbs, objects, and sub-phrases by clustering the components of each list within an appropriate alphabetical letter index. Additionally, in one embodiment the lists may be presented in a more compact fashion by initially displaying only the individual alphabetical letter indices of each list rather than all of the components of the list. Once the user selects a particular index, only the components within that alphabetical index selected will be presented to the user.

Turning to FIG. 4, an alternative embodiment of the method involves appending a subsystem of uttering components to the beginning of the previously described uttering formation process. Prior to presenting the user with an initiation phrase such as "I want to", this embodiment of the method will instead allow the user to define the user's identity. This may be accomplished by allowing the user to first choose from a list of identities that may include "student", "professional", "web surfer", "private individual", and "traveler", for example, once the user chooses an identity, the subsystem will next present the user with varying levels of skill, knowledge, or specialties within the chosen identity in order to more accurately define the user identity. For example, if the user initially chooses a "student" identity, the subsystem may subsequently present the user with a list that includes K-4 grade", 5-8 grade", "9-12 grade", "college", and "graduate school" from which the user may select in order to refine the selected identity. Once the user-specific identity is defined using the subsystem, the previously described method, starting with an initiation phrase, will be used to generate the semantically coπect uttering. Using the user- defined identity, the method will be able to tailor both the uttering components and the resultant target list to the level of knowledge of the user. For example, if the chosen user- specific identity is a 9-12 grade student, the method may direct the user only to sites or records that are appropriate for a 9-12 grade student.

Turning now to FIG. 5, an embodiment of the method, includes a coding system for each component of the uttering that allows for easier classification of target destinations. In this embodiment, each uttering component is assigned a unique code. When the user completes an uttering, each individual code of each uttering component is read together to form a master identification code. Web site operators may use a master identification code to allow for accurate indexing of their site by the method. For example, using a master identification code that coπesponds to the subject of their site, web site operators may place coπesponding metatags within the source of their site. The use of metatags that coπespond to the code system used by the method would enable the operators of the uttering formation method to more efficiently index web site targets through the use of web crawlers or other search apparatus that are capable of detecting the metatags. In the example illustrated in FIG. 5, the master identification code formed by the example uttering would be [001240005075]. This code is formed by the individual codes of the uttering components "I want to" [001], "shop - using" [2401], "auctions" [005], "track & search services" [075]. An online auction tracking and search site may place the resultant code, [001240005075], within the source-of the site in order to allow the operators of the method to coπectly index the site and direct users to the site when the user forms an uttering with the matching code.

Another embodiment of the method involves the construction of a script by the user that enables the user to execute a number of uttering commands through the use of one uttering. For example, a user might construct the uttering "I WANT TO - CREATE - SCRIPT" followed by a unique name given by the user. The unique name may include any descriptive or abstract terms. The user then constructs the utterings to be included in the script in sequential fashion. After all of the utterings are entered the script may be executed either on a certain date an time, or at any time by uttering "I WANT TO - RUN - SCRIPT - (script name). The script name used is the same as that given to the script at the time it was created. Users may modify existing scripts at any time by using the uttering I WANT TO - CHANGE SCRIPT (script name). This capability would be useful for executing more involved tasks such as making traveling aπangements. A typical traveling aπangement script might include the following utterings:

I WANT TO PLAN - TRIP I WANT TO RESERVE - HOTEL

I WANT TO RESERVE - HOTEL - KIDS CAMP

I WANT TO REDEEM - FREQUENT TRAVELER POINTS

I WANT TO RESERVE - AIR FLIGHT

I WANT TO SHOP - FLIGHT INSURANCE I WANT TO RESERVE - AUTO

To end the session the user may use an uttering such as "I WANT TO - END -

SCRIPT."

The utterings constructed by the method will preferably be semantically coπect in English or other natural languages. The method may also be applied to generate semantically coπect utterings in any other language that has a known predefined syntactical structure, such as computer languages.

In a prefeπed embodiment, the method is implemented by installing the method within an Internet web browser such as Microsoft Explorer or Netscape Navigator. For example, the method components may be placed in a sub-folder within the Favorites folder of Microsoft Explorer or the Bookmarks folder of Netscape Navigator.

Additionally, the method may be implemented in a similar manner using any other web browser that has analogous functionality to the currently available browsers.

In another prefeπed embodiment, the method is embedded within preexisting Internet search engines or indexes. The user would then be able to use the method to construct a semantically coπect uttering that would refine a search request within the search engine or index.

Alternatively, the method may also be incorporated within a node computer apparatus. Through the use of a node computer, or a network computer, users may access an application software provider or a server in order to utilize various applications including word processors and spreadsheets. These applications are stored remotely by the application software provider or on the server. The method of constructing semantically coπect utterings may be incorporated within a node computer apparatus to enable a node computer user to more efficiently access and operate the remotely stored applications.

Preferably, the interface is activated wholly by selection and pointing using the mouse. The dialog mode is intrinsically menu selection, except when specific information like names and sub-selections is required. There are no special icons, symbols or characters needed to be entered at any time. When input is complete the user simply moves the pointer to the next menu option, always working in straight up and down, and left and right directions.

Throughout the dialog, the user is provided with all the information needed to determine what options are available, and what their functional capabilities are. If a mistake is made, the user can quickly get back to the last option selected or start from the beginning by manipulating the mouse in the desired direction, for example.

Turning to FIGS. 6 and 7, a universal controller device system 50 designed to utilize the method of this invention in order to control various target devices is illustrated. FIG. 6 illustrates the exterior of one embodiment of the universal controller system 50. The controller system 50 operates a target device by transmitting an uttering that is derived from a semantically coπect uttering formed by a user while operating the controller 50. The housing 51 of the controller is compact, enabling the controller 50 to be a portable device. To further emphasize the portable nature of the system 50, an attachment eyelet 200 may be located on the housing 51 that allows a user to use the system 50 as a key chain.

Turning to FIG. 7, a schematic diagram of the various possible components of the controller 50 is illustrated. A program that implements the method of generating semantically coπect utterings may be located within a fiπnware memory unit 52. A central processing unit (CPU) 53 controls the program located within the firmware memory 52. The CPU 53 accesses the firmware 52 in order to operate the method of generating semantically coπect utterings.

Optionally, a media reader slot 300 may be provided that enables a user to insert preprogrammed cards to vary the functionality of the system 50. For example, a card may be preprogrammed with a method program of verb or noun choices and control commands that are specifically tailored to a particular target device. In this embodiment, the CPU 53 will then access the card in the slot 300, rather than the program located in the firmware memory unit 52, in order to execute the method of forming semantically coπect utterings and to retrieve the appropriate commands for the particular target device. When accessing a program located in a media card, the CPU 53 may store the accessed program within an erasable memory unit 54.

Also, the slot 300 may be replaced with an interface that allows the user to connect a disk drive, cartridge drive, tape drive, or other removable media device to the controller system. As with a program stored on a media card, the CPU 53 would access a program stored on removable media associated with the removable media device. The CPU 53 would then store and execute the program in the same manner as described for a program stored on a media card.

Regardless of whether the CPU 53 executes a program stored in the firmware memory unit 52, a program stored in a media card, a program stored in removable media, or a program transfeπed from a media card or removable media to the erasable memory unit 54, the CPU also utilizes the erasable memory 54 to store data that is used to operate the method program. This data includes the user's selections of verbs and nouns/objects and the resultant semantically coπect utterings.

When the controller 50 is in use, the CPU 53, by executing the method program, presents the user with an initial phrase to start the uttering. The CPU 53, again accessing the method program, next presents the user with verb choices to use in the uttering. The CPU 53 stores the user's verb choice within the erasable memory 54. In one embodiment, the CPU 53 subsequently presents at least one list of phrases or sub-phrases that the user selects in order to complete the uttering. The CPU 53 stores the user's phrase choice or choices and the resultant semantically coπect uttering within the erasable memory 54. A selection apparatus is provided that enables the user to interact with the controller 50 in order to made selections amongst the verbs, the phrases, and the sub- phrases that the CPU 53 presents to the user. The selection apparatus may be a plurality of actuators 55, a display screen 100 that has touch screen functionality, a microphone 70, or any combination of these units. In one embodiment, a plurality of actuators 55 enables the user to interact with the controller 50 in order to make selections amongst the verbs, the phrases, and the sub-phrases that the CPU 53 presents to the user. The actuators include an up actuator 55a, a down actuator 55b, a left actuator 55c, a right actuator 55d, and an enter/send actuator 55e. hi one embodiment, an input/output controller 58 may be provided to coordinate the inputs from the selection apparatus 55 to the CPU 53 and the outputs from the CPU 53 to the appropriate transmitting apparatus.

The lists of verbs, phrases, and sub-phrases, and the user's subsequent selections and resultant semantically coπect uttering are all displayed on a display screen 100 that encompasses a substantial portion of the exterior of the housing 51, as shown in FIG. 6. As previously described, the screen 100 may also include touch screen functionality that allows the user to make selections by touching the screen, thereby using the screen 10 as the selection apparatus. Turning back to FIG. 7, the screen 100 may be controlled by a number of different components. In one embodiment, an audio-video board 101 controls the functioning of the screen or touch screen 100. Alternatively, the manufacturer of the controller system may opt to design a system wherein an input/output controller 58 controls the screen 100 in addition to processing data from the selection apparatus. The system may also be designed to allow the CPU 53 to control the screen 100, thereby eliminating the need for a dedicated audio-video board 101 or an input/output controller 58 to operate the screen 100.

In one embodiment of the controller 50, the controller 50 is modified to enable a visually impaired person to operate the controller 50. In this embodiment, a microphone 70 is used as part of the selection apparatus. The microphone 70 enables a visually impaired user to hear the selections of verbs and nouns/objects that will comprise the semantically coπect uttering. A speaker 71 verbally presents the possible selections of verbs and nouns to the user. In this embodiment, the CPU 53 will use the method program in order to read each possible selection of verbs and nouns at each step in the method to the user. The user may scroll through the possible verb and noun choices using the plurality of actuators 55. The CPU 53 will indicate the particular choice that is highlighted through the use of the actuators 55 via a signal sent to the input/output controller 58, which in turn sends a signal to the speaker 71. Additionally, instead of scrolling through each choice one by one, the user may hold down a particular actuator and the CPU 53 will skip through large portions of the selections. When the user releases the particular actuator, the CPU 53 will send a command to the input/output controller 58 to read the selection that is highlighted at that time. Once the user makes a selection of a verb or a noun through the use of the actuator apparatus 55, the CPU 53 will also send a command to repeat the user's choice to the input/output controller 58 which activates the speaker 71 to verbalize that choice back to the user.

Alternatively, the user may select a verb or noun choice through the use of a microphone 70 as the selection apparatus. If the user utilizes the microphone 70 to select a verb or noun choice, the input/output controller 58 will relay the choice that was spoken into the microphone 70 to the CPU 53. The CPU 53 will then utilize the spoken choice to form the uttering.

In all of the embodiments of the controller, once the user indicates that the uttering _. is complete, the CPU 53 translates the uttering into a command that a target device is able to recognize. The controller 50 will transmit a coπesponding command to the target device using either a wireless transmitting unit 56 or a direct data transfer unit 57. When communicating with a target device via the wireless transmitting unit 56, the controller 50 will preferably send commands to the target device through either infrared or radio frequency transmissions. When communicating via the direct data transfer unit 57, the controller 50 will utilize a physical connection with the target in order to send commands to the target. Turning to FIG. 8, a flow chart illustrates the functioning of one embodiment of the remote controller system. This example of the controller system is merely illustrative and does not limit the scope of the invention in any manner. Initially a user will depress an actuator to turn on the controller. The selection actuators are connected to an input/output controller. When initially depressed, the depressed selection actuator will transmit a signal to the input/output controller, which in turn transmits a signal to the CPU to alert the CPU that the user desires to use the system. The CPU then accesses a method program in order to display possible selections of verbs and nouns to the user on a display apparatus. The CPU may access the method program stored in a firmware memory unit located within the controller. Alternatively, the CPU may access a method program stored on a media card that is inserted in a media reader slot of the controller.

The user may select uttering components using either the actuators or the actuators in conjunction with a microphone. In one embodiment, the controller repeats each of the user's selections back to the user through the use of a speaker located on the controller's housing. When the user has completed the construction of an uttering, the user depresses the send actuator. When the send actuator is depressed, the CPU will send the uttering to the input/output controller which in turn transmits the uttering to a target device. The input/output controller may transmit the uttering through the use of either a direct data transfer unit or a wireless data transfer unit. Both the direct data transfer unit and the wireless data transfer unit are also capable of receiving a confirmatory acknowledgement signal from the target device. Upon receipt, of the acknowledgement signal, the controller CPU access the method program, located either in the firmware unit or in a media card inserted in the media reader slot, and prepares to receive new selections from the user. If the target device sends an eπor signal to the controller, the controller will display an eπor message to the user.

When the controller has not been used for a predetermined period of time, the controller enters a power saving sleep state. The CPU may be present to enter the power saving state if the CPU does not receive any inputs within a certain amount of time. Alternatively, the method program that the CPU executes may contain verb and object/noun selections that enable a user to foπn an uttering to control the unit's sleep state. A sample uttering that would instruct the CPU to place the controller in the sleep state could be: I want to - inactivate - remote controller. Machine-based systems that could use the concepts of the invention disclosed herein include the following, without limitation: the Internet; desktop, laptop, and palmtop personal computers; personal digital assistant (PDA) devices; word, data, image, and sound communications networks; manufacturing automation systems; process control systems; avionic, automotive and other vehicle command, control, and guidance systems; air, space, road, rail and water traffic control systems; telephone, smart phone, voice- response, email, facsimile, videophone, and videoconferencing systems; radio, television, facsimile, email, and Internet broadcasting systems; military and defense command and control systems; power transmission command and control systems; self-serve, automated and attended vehicle fueling systems; logistics and distribution command and control systems; self-serve, automated and attended teller systems; cash, check, debit, credit and smart card register, control, receipt, dispensing and communication systems.

The human-machine interface of this invention is application and adaptable to any semantic uttering such as queries, commands, informational utterings, query replies and computer programming utterings; and is applicable to any Internet search, control and command system; any computer assembler and program compiler; database system; data warehousing system; informational reference system; and computer retrieval system.

The particular examples set forth herein are instruction and should not be interpreted as limitations on the applications to which those of ordinary skill are able to apply this invention. Modifications and other uses are available to those skilled in the art which are encompassed within the spirit of the invention as defined by the scope of the

following claims.

Claims

WHAT IS CLAIMED IS:

1. A method for enabling the construction of a semantically coπect uttering by a user through a user interface comprising the steps of displaying an initiation phrase with which the user will begin to construct the uttering, selecting a verb from a list of verbs, the list of verbs extending from the initiation phrase, the verb describing an action the user desires to perform, selecting at least one object phrase from at least one list of objects associated with the verb, the at least one list of objects presented to the user after the list of verbs, to form the semantically correct uttering in conjunction with the verb, and directing the user to targets based on the semantically coπect uttering formed by the method.

2. The method of Claim 1 further comprising the step of adding synonymous verb choices to the list of verbs that direct the user to one verb choice that will be used for all synonyms of the one verb choice.

3. The method of Claim 1 further comprising the step of adding synonymous object choices, to the at least one list of objects, that direct the user to one object choice that will be used for all synonyms of the one object choice.

4. The method of Claim 1 wherein the user interface is embedded within an electronic browser.

5. The method of Claim 1 wherein the user interface is installed within a search engine.

6. The method of Claim 1 wherein the user interface is incorporated into a node computer apparatus.

7. The method of Claim 1 further comprising the step of coding the uttering with an identification code that matches predetermined criteria to enable more accurate presentation of targets to the user.

8. The method of Claim 1 further comprising the step of defining a user-specific identity in order to tailor the method to the user.

9. The method of Claim 1 further comprising the step of categorizing uttering component choices within alphabetical indices to more efficiently present the component choices to the user.

10. A method for enabling the construction of a semantically coπect uttering by a user through a user interface comprising the steps of displaying an initiation phrase with which the user will begin to construct the uttering, selecting a verb from a list of verbs, the list of verbs extending from the initiation phrase, the verb describing an action the user desires to perform, and directing the user to targets based on the semantically coπect uttering formed by the method.

11. The method of Claim 10 further comprising the step of adding synonymous verb choices, to the list of verbs, that direct the user to one verb choice that will be used for all synonyms of the one verb choice.

12. The method of Claim 10 further comprising the step of coding the uttering with an identification code that matches predetermined criteria to enable more accurate presentation of targets to the user.

13. The method of Claim 10 further comprising the step of defining a user-specific identity in order to tailor the method to the user.

14. The method of Claim 10 further comprising the step of categorizing uttering component choices within alphabetical indices to more efficiently present the component choices to the user.

15. A system for enabling the construction of a semantically coπect uttering by a user through a user interface comprising an initiation phrase with which the user will begin to construct the uttering, a list of verbs extending from the initiation phrase from which the user will select a verb to describe the action that the user desires to perform, at least one list of object phrases from which the user will select an object to use with a previously selected verb in order to form the semantically coπect uttering, and targets to which the user will be directed based on the semantically coπect uttering.

16. The system of Claim 15 wherein the list of verbs includes synonymous verb choices that direct the user to utilize one verb choice for all synonyms of the one verb choice.

17. The system of Claim 15 wherein the at least one list of object phrases includes synonymous object phrase choices that direct the user to utilize one object plirase choice for all synonyms of the one object phrase choice.

18. The system of Claim 15 further comprising a coding subsystem to match the uttering with predetermined criteria to enable more accurate presentation of targets to the user.

19. The system of Claim 15 further comprising a user definition subsystem that allows the user to define a user-specific identity in order to tailor the method to the user.

20. The system of Claim 15 further comprising alphabetical indices to categorize uttering component choices in order to more efficiently present component choices to the user.

21. A system for enabling a user to construct a semantically coπect uttering through a user interface comprising an initiation phrase with which the user will begin to construct the uttering, a list of verbs extending from the initiation phrase from which the user will select a verb to describe the action that the user desires to perfoπn, and targets to which the user will be directed based on the semantically coπect uttering.

22. The system of Claim 21 further comprising adding synonymous verb choices, to the list of verbs, that direct the user to one verb choice that will be used for all synonyms of the one verb choice.

23. A universal controller system that enables a user to control target devices utilizing a method for constructing semantically coπect utterings, comprising a compact housing that enables the controller system to be a portable unit, a program unit containing a program enabling the user to construct a semantically coπect uttering by initially selecting a verb choice and then selecting at least one target phrase to form the uttering, a selection apparatus situated on the housing that allows the user to select amongst verb choices and phrase choices to select verb and phrase combinations, thereby forming the semantically coπect uttering, and a transmitting apparatus to send the semantically coπect uttering, in the form of an uttering recognizable by a target device to be controlled using the controller, to the target device to effectuate a function on the target device.