GB2327327A - Data processing system with display and user input device - Google Patents

Abstract

A two-way activator or two-way switch is provided for the control of electronic media presentations through user-interactive computer controlled display interfaces. The operation of the switch control is dependent on the duration of time of the user input to the two-way switch. In response to an input of a first time duration to the switch, the system being controlled performs a first function. Then, responsive to an input of a second time duration, the system performs a function which is different in kind from the first function.

Description

DATA PROCESSING SYSTEM WITH DISPLAY AND USER INPUT DEVICE The present invention relates to user interactive computer supported display technology and particularly to user interactive systems and methods for providing display interfaces for example for electronic media presentations and for the editing of such presentations.

The current decade has been marked by a social and technological revolution driven by the convergence of the data processing industry with the consumer electronics industry. This advance has been even further accelerated by the extensive consumer and business involvement in the Internet over the past two years. As a result of these changes, it seems as if virtually all aspects of human endeavor in the industrialized world require human-computer interfaces. As a result of these profound changes, there is a need to make computer directed activities accessible to a substantial portion of the world's population which, up to a year or two ago, was computer-illiterate, or at best computer indifferent. In order for the vast computer supported market places to continue and be commercially productive, it will be necessary for a large segment of computer indifferent consumers to be involved in computer interfaces.

Thus, the challenge of technology is to create interfaces to computers which are as close to the real world as possible.

One major market for user friendly computer controlled display interfaces has been in the selection and delivery of a variety of electronic media presentations which is part of the multimedia marketplace. In the media presentation product fields, it is very important to provide viewers or users with on-screen controls for making and editing various media presentations. These controls must be easy to use. As the number of functions available to interactive display users to control electronic media increases, the complexity of the display screens would be expected to increase. The display interface technology is continually seeking new implementations which simplify display interfaces in order to offset the increased complexity brought about by increased function.

Accordingly, the present invention provides a data processing system including a display device and user input time-dependent two-way activation means comprising: means responsive to a user input to said two-way activation means for a first time period to perform a first display function, and means responsive to a user input to said two-way activation means for a second time period longer than said first time period to perform a display function different in kind from said first display function.

Such an approach is user friendly, so that even non-computerliterate users can be provided with an interface environment which is easy to use and intuitive.

In a preferred embodiment, said two-way activation means is a push button device, said user input for said first time period is a press and release of said push button within said first time period, and said user input for said second time period is a press and hold of said push button for a period of time longer than said first time period.

The preferred embodiment further includes means for storing the content of a media presentation, and interactive means for controlling said media presentation including said two-way activation means.

Typically said media presentation is an audio or video presentation. In the former case, said first display function may be a change from one recorded audio track to another, and said different display function the forwarding of position along one recorded track; in the latter case, said first display function may be a jump from one recorded video segment to another, and said different display function the playing of one video segment.

The invention further provides a method of operating a data processing system including a display device, responsive to user input by time-dependent two-way activation means, said method comprising the steps of: responsive to a user input to said two-way activation means for a first time period, performing a first display function, and responsive to a user input to said two-way activation means for a second time period longer than said first time period, performing a display function different in kind from said first display function.

The invention further provides a computer program having data structures included on a computer readable medium for operating a data processing system including a display device in response to user input by time-dependent two-way activation means, comprising: means responsive to a user input to said two-way activation means for a first time period to perform a first display function, and means responsive to a user input to said two-way activation means for a second time period longer than said first time period to perform a display function different in kind from said first display function.

Viewed from another aspect, the invention provides a data processor controlled display system having user input time-dependent two-way activation means comprising: means responsive to a user input to said two-way activation means for a first time period to perform a first display function, and means responsive to a user input to said two-way activation means for a second time period longer than said first time period to perform a display function different in kind from said first display function.

Preferably said two-way activation means is a push button device, said user input for said first time period is a press and release of said push button, and said user input for said second time period is a press and hold of said push button.

Thus the approach described herein provides in a preferred embodiment for the selection of, access to and control of the stored content of an electronic media presentation through a data processor controlled display interface which includes two-way activation means for controlling display functions in which there are means responsive to a user input of a first time period to the two-way activation means to perform a first display function, and means responsive to a user input to the same two-way activation means but of a second time period longer than the first time period to perform a display function entirely different in kind from the first display function. Since the user input is represented on the screen as an object, a single display object may be used to perform two functions entirely different in kind. This serves to simplify the display screen as otherwise the two functions would have to be represented on the display screen by two objects which would make the display screen more crowded and thus more complex.

The two-way activation means is preferably a push button device such as a mouse button or a key which is preferably represented on the display screen as an up/down two position switch. This displayed two position switch may also be directly controlled by the user if the input is through a touch-screen.

It is important to recognise that the two functions controlled by the two-way switch are different in kind i.e., the functions are not just variations in degree of the same function, it already being known in the art to use the same switch for variations of the same function, e.g., fast and slow forward. It will also be appreciated that the data processing system described herein may comprise a multimedia computer workstation, a digital television control system (eg set top box), some form of handheld computing or electronic device (a "tier 0" system), or any other appropriate system that could benefit from the two-way switch described herein.

A preferred embodiment of the invention will now be described in detail by way of example only with reference to the following drawings: Fig. 1 is a block diagram of a data processing system; Fig. 2 shows a typical three-dimensional workplace embodiment of a portion of a media entertainment center; Figs. 3A and 3B are representations of the displayed audio CD presentation and two-way switch control during a first mode of operation of the switch; Figs. 4A and 4B are representations of the displayed audio CD presentation and two-way switch control during a second mode of operation of the switch; Figs. 5A and 5B are representations of a displayed video presentation and two-way switch control during a first mode of operation of the switch; Figs. 6A and 6B are representations of a displayed video presentation and two-way switch control during a second mode of operation of the switch; Figs. 7A and 7B are flowcharts of an embodiment of a process for running a presentation under the control of a two-way switch.

Referring to Fig. 1, a typical data processing system is shown having a central processing unit (CPU), such as one of the PowerPC microprocessors available from IBM Corporation (PowerPC is a trademark of IBM Corporation), interconnected to various other components by system bus 12. A read only memory (ROM) 16 is connected to CPU 10, via bus 12 and includes the basic input/output system (BIOS) that controls the basic computer functions. Random access memory (RAM) 14, I/O adapter 18 and communications adapter 34 are also interconnected to system bus 12. It should be noted that software components including the operating system 41 and application 40 are loaded into RAM 14 which is the computer system's main memory. I/O adapter 18 may be a small computer system interface (SCSI) adapter that communicates with the disk storage device 20, i.e. a hard drive. Communications adapter 34 interconnects bus 12 with an outside network enabling the data processing system to communicate with other such systems over a local area network (LAN), wide area network (WAN), or the like. I/O devices are also connected to system bus 12 via user interface adapter 22 and display adapter 36.

Keyboard 24, trackball 32, mouse 26 and speaker 28 are all interconnected to bus 12 through user interface adapter 22. Display adapter 36 includes a frame buffer 39 which is a storage device that holds a representation of each pixel on the display screen 38. Images may be stored in frame buffer 39 for display on monitor 38 through various components such as a digital to analog converter (not shown) and the like. By using the aforementioned I/O devices, a user is capable of inputting information to the system through the keyboard 24, trackball 32 or mouse 26 and receiving output information from the system via speaker 28 and display 38.

As will be hereinafter described in greater detail, the input means in the preferred embodiment may be the trackball, mouse or keyboard, each of which has a key or equivalent member which may pressed or held down for shorter or longer periods. Where the I/O is through a touch screen, then the image area on the touch screen representing the two-way activator, e.g., push-button may be held down for varying periods in doing the input.

The operating system 41 runs on CPU 10 and provides control and is used to coordinate the function of the various components of Fig. 1.

Operating system 41 may be one of the commercially available operating systems such as DOS, Windows, or the OS/2 operating system available from IBM Corporation (OS/2 is a trademark of IBM Corporation; Windows is a trademark of Microsoft Corporation). A program application 40 runs in conjunction with operating system 41 and provides output calls to the operating system 41 which implements the various functions to be performed by the application.

In the preferred embodiment, program application 40 is developed using the Virtual Reality Toolkit (VRT) from Superscape Ltd (having US offices in Palo Alto, California), a virtual reality three-dimensional image creation program which allows images to be rendered. Extensive details of VRT programming techniques may be found in the Superscape VRT, Reference Manual, Version 4-00, 2nd Edition, January 29, 1996. Such virtual reality software can be used to create a three-dimensional workspace that is perceived as extending in three orthogonal directions.

Typically a display has a two-dimensional display surface and the perception of a third dimension is effected by visual clues such as perspective lines extending toward a vanishing point.

It will be appreciated by those skilled in the art that such virtual reality programs typically use object oriented programming techniques involving the definition, creation, use and instruction of "objects". These objects are software entities comprising data elements and routines, or methods, which manipulate the data elements. The data and related methods are treated by the software as an entity and can be created, used and deleted as such. The data and functions enable objects to model their real world equivalent entity in terms of its attributes, which can be presented by the data elements, and its behavior which can be represented by its methods. Other object oriented languages which might be used to develop applications in this context include (for example) C++.

The preferred embodiment will now be explained in detail using for an initial example, an audio CD presentation system such as that described in the US patent application, "Interactive Display Interface for Media Presentation." Bardon et. al., filed 23 May 97, (Attorney Docket No. AT9-97-050), a copy of which is placed on the file of this application. Such an audio CD presentation system includes a computer controlled three dimensional display interface for controlling electronic media presentations. The two-way activation means described herein may be used in the mouse or key driven on-screen control system described in this earlier application. Thus the system for controlling the CD presentation in this earlier application is shown within the context of a virtual reality three-dimensional workspace as shown in Fig. 2. The workspace 42 is shown as an image of an entertainment center, a portion of which, 43 is shown in viewpoint interface 44 which is presented to the viewer on display monitor 38 of Fig. 1. In accordance with conventional techniques, the user may control the viewpoint 44 through a conventional I/O device such as mouse 26 of Fig. 1 which operates through user interface 22 to call upon VRT programs in RAM 14 cooperating with the operating system 41 to create the images in frame buffer 39 of display adapter 36 to control the display on monitor 38. Using conventional virtual three-dimensional workspace navigation techniques, the viewpoint interface of the user shown in Fig. 2 is changeable as the user moves closer or backs away from objects in the workspace or moves to the right or left. All this may be controlled by a suitable I/O device such as mouse 26 of Fig. 1.

The workspace of Figure 2 also includes a CD library in the form of CD carousel 58 from which the user operating with conventional selection pointing techniques has browsed through carousel 58 to select a desirable CD to play. Once the appropriate selection of the object has been made (eg in accordance with the process described in the above-mentioned earlier application), the selected CD appears as shown in Fig. 2. The CD control image 45 contains a realistic image of the CD jacket 46 as well as all of the controls 47 necessary to run the CD audio presentation. It also contains a numerical indicator of the track number and time 48.

Using these controls, the user may run the audio CD without any CD player.

Referring now to the CD objects 45 shown in Figs. 3A and 3B, in Fig. 3A there is shown the CD control image 45 from Fig. 2. This control image has an image of the CD jacket 46 and a control area 47. For purposes of the present illustration we will focus on switch image 49 which is a two way switch. In Fig. 3A mouse controlled cursor arrow 50 is shown in contact with two way switch 49. As previously described, cursor arrow 50 may be controlled by mouse 26 (Fig. 1) in the conventional manner. For purposes of the illustration in Figs. 3A and 3B, we will assume that the user through the use of the cursor 26 in Fig.

1 is pressed to a switch 49 and is holding that switch down for a period of time. This operates switch 49 in its first mode wherein it fast forwards the present track in the audio CD. Indicator 48 shows that the present track is track "1." The results of holding switch 49 down for a period of time are shown in Figure 3B, in which the audio CD has remained on track 1 as shown by indicator 48 but track 1 has been fast forwarded from an initial 0:00 in Fig. 3A to 1:06 in Fig. 3B.

Now with reference to Figs. 4A and 4B the other mode of operation of a two way switch 49 to control the audio CD will be illustrated.

Again, as indicated by cursor arrow 50, switch 49 is selected but this time it is not held down for any substantial period but merely given a short click via the appropriate button of mouse 26 (Fig. 1). The result of this short click is that the selected track jumps from track 1 as shown in indicator 48 of Fig. 4A to track 2 shown again in indicator 48 of Fig. 4B. However no movement in either track has occurred since the indicator in both Figs. 4A and 4B stands at 0:00.

If instead of a CD audio presentation, the media presentation is a video presentation, similar control results can be obtained. For a simplified illustration of a video presentation, reference is made to Figs. 5A, 5B, 6A and 6B. In these illustrations, control panels 51 are used to control video presentations which could be shown on a full screen, or particularly for editing purposes, on a browser window 52 which may conveniently be adjacent to control panel 51. It should be understood that for a video presentation, a control panel might be selected with a somewhat different arrangement in control panel 51; however, for convenience in illustration, the video control panel 51 is substantially similar to the audio control panel 47 (Figs. 2-4B). In the first mode of operation two way switch 53 in Fig. 5A is depressed and held through cursor arrow 54 for a period of time. As a result of holding down switch 53 the video remains on the same scene, scene 1 which is equivalent to the audio track 1, but the video is advanced. Comparing indicator 56 in Figs 5A and 5B we note that scene one is still in effect but the scene has advanced to 1:06. This is illustrated in the browser window by the movement of boat 55 within this browser window. On the other hand, as illustrated in Figs. 6A and 6B, if two way switch 53 is selected by cursor arrow 54 but the controlling mouse is only clicked and released, then the scene completely changes, eg from the sunny beach scene shown in Fig. 6A to the snowy mountains scene shown in Fig. 6B.

The change in scene is also indicated by the change in indicator 56 from scene one to scene two.

Now with reference to Figs. 7A and 7B, these flowcharts illustrate the process flow of a method in accordance with the present invention.

The flowcharts are in two parts, with the steps in Fig. 7A relating to the development of the virtual reality three dimensional workspace, in particular to a set of objects corresponding to media presentations within which are used the two way switches described above. Thus first, step 60, (Fig. 7A) the desired three dimensional workspace is laid out including appropriate objects. For example, this would be workspace 42 in Fig. 2 which would be created and stored. Next, step 61, the objects including objects for the media presentation are set up and stored. This includes the CD control image 45 in Fig. 2, including the control panel 47. Next, step 62, one or more two way switches are set up. An example of such a two way switch is switch 49 in Figs. 3A and 3B. Also, step 63, means for selecting and holding such two way switches are provided1 including the drivers and other linkages connecting mouse 26 Fig. 1 via cursor arrow 50 to move two way switch 49 into a select or non select position. Then, step 64, media presentations are stored on the system, such as the audio CD presentation previously described with respect to Figs. 2 through 4B, or the video presentation described in Figs. 5A through 6B. Program means are then provided for supporting display functions of first and second kinds in response to the time period for which the two way switch such as switch 49 is held, as previously described for the audio presentation in Figs. 2 through 4B and for the video presentation in Figs. 5A thru 6B. Lastly, conventional navigational means are provided, step 66, for changing user viewpoints.

The process now proceeds to point A in Fig. 7B. At this point, the previously created and stored three-dimensional workspace program is run, step 67. Then using the conventional navigation means for changing viewpoints, step 68, the system is navigated to a particular position or viewpoint at which a particular object representative of a media presentation is available eg, the viewpoint in Fig. 2 wherein CD ROM carousel 58 is available. Next in decision step 69 a determination is made whether a media presentation has been selected. For the purposes of the present example, let us assume that the user has selected a CD ROM presentation and, step 70, the appropriate display interface is set up, including the two way switch, as previously described for running an audio CD presentation, Fig 3A. Next, in decision step 71, a determination is made as to whether the switch has been pressed. If the switch has been pressed, then a determination is made, step 72, as to whether the amount of time the switch has been held down is greater than a set time. This time differential determines whether or not the pressing of the switch is a mere click or whether the switch is being held down.

If the decision is yes, i.e. the switch is being held down, the system proceeds to step 71 where the CD ROM is fast forwarded within its track as shown in Fig. 3B. On the other hand if the determination from step 72 is that the input duration is short, then the program jumps through the CD ROM to the next track i.e. from track 1 to track 2 as illustrated in Figs. 4A and 4B. The system then returns to decision step 71 via entry point B where a determination is again made as to whether the switch is pressed. If the decision from decision step 71 is that the switch has not been pressed then a determination is made in decision step 75 as to whether the session is over, and if so, the system proceeds to end the session, step 77. If the session is not over the system returns to decision step 71 to await a pressing of the switch. Going back a few steps to decision step 69 where a determination was made as to whether a media presentation was selected, if a media presentation was not selected, then the system proceeds to decision step 76 where a determination is made as to whether the session is over. If the session is over then here again the system proceeds to step 77 and the session is ended. On the other hand if the determination from decision step 76 is that the session is not over, then the system returns to step 68 and navigation takes place to a subsequent position.

The skilled person will appreciate that the flow charts of Figs. 7A and 7B could be readily applied to the use of the two-way switch in the control of video presentations as set forth in Figs 5A through 6B, or in any other suitable situation. Further, it will be appreciated that although the preferred embodiment has been described in the context of a a three-dimensional virtual environment, it should be understood that the same approach could be employed within a more conventional twodimensional windows environment.

Claims (10)

1. A data processing system including a display device and user input time-dependent two-way activation means comprising: means responsive to a user input to said two-way activation means for a first time period to perform a first display function, and means responsive to a user input to said two-way activation means for a second time period longer than said first time period to perform a display function different in kind from said first display function.

2. The system of claim 1, wherein said two-way activation means is a push button device, said user input for said first time period is a press and release of said push button within said first time period, and said user input for said second time period is a press and hold of said push button for a period of time longer than said first time period.

3. The system of claim 1 or 2, further including means for storing the content of a media presentation, and interactive means for controlling said media presentation including said two-way activation means.

4. The system of claim 3 wherein: said media presentation is an audio presentation, said first display function is a change from one recorded audio track to another, and said different display function comprises forwarding of position along one recorded track.

5. The system of claim 3 wherein: said media presentation is a video presentation, said first display function is a jump from one recorded video segment to another, and said different display function comprises playing of one video segment.

6. A method of operating a data processing system including a display device, responsive to user input by time-dependent two-way activation means, said method comprising the steps of: responsive to a user input to said two-way activation means for a first time period, performing a first display function, and responsive to a user input to said two-way activation means for a second time period longer than said first time period, performing a display function different in kind from said first display function.

7. The method of claim 6, wherein: said two-way activation means is a push button device, said user input for said first time period is a press and release of said push button within said first time period, and said user input for said second time period is a press and hold of said push button for a period of time longer than said first time period.

8. A computer program having data structures included on a computer readable medium for operating a data processing system including a display device in response to user input by time-dependent two-way activation means, comprising: means responsive to a user input to said two-way activation means for a first time period to perform a first display function, and means responsive to a user input to said two-way activation means for a second time period longer than said first time period to perform a display function different in kind from said first display function.

9. The computer program of claim 8, wherein: said two-way activation means is a push button device, said user input for said first time period is a press and release of said push button within said first time period, and said user input for said second time period is a press and hold of said push button for a period of time longer than said first time period.

10. A computer multimedia presentation device substantially as described herein with reference to the accompanying drawings.