WO1995000917A1

WO1995000917A1 - Integrated multimedia information management system

Info

Publication number: WO1995000917A1
Application number: PCT/US1994/006766
Authority: WO
Inventors: Grant K. Holcomb
Original assignee: Multimedia Design Corporation
Priority date: 1993-06-21
Filing date: 1994-06-21
Publication date: 1995-01-05
Also published as: AU7207994A

Abstract

A multimedia management system, comprising a plurality of host computer platforms (9) and operating system software therefor, and video display screens (13-1, 13-2) and means for connecting the screens to the host computer platforms (9) to provide easy access to a plurality of media sources, including video, audio, and communication sources. In order to enable facile access to the media sources, a pointing device provides horizontal and vertical coordinates to a selected computer platform, and a controllable interface system couples the media sources to a selected host computer platform. A large screen display (14) can be coupled to the same information as displayed on one of the video display screens. The controllable interface system includes a programmable microprocessor and a bus system, a plurality of RS-232 interface cards (24, 25-1 through 25-5, 26 and 27) connected to the bus system. Video (20) and audio (19) switching matrices selectively couple the plurality of media sources to a selected host computer and two or more display screens are positioned for common viewing by a user and the pointing device (10-1) is a touch panel device. Speakers (18) and a data entry device for recording responses are also included.

Description

Integrated Multimedia Information Management System

I. Background of the Invention

1. Field of the Invention. This invention relates in general to multimedia systems, and in particular to the integration of computers, computer software, video electronics, audio electronics, and communications electronics.

2. Description of Related Art. If a political leader, business executive, manager, or teacher could retrieve, organize, query and present all known forms of information through a single integrated system, the impact on their efficiency would be astronomical. The ability to make informed decisions and to decisively influence others depends directly upon the availability of accurate and timely information. The ability to teach, inspire, and to enlighten others depends as much upon the information as the form in which it is presented. Finally, the strength of the economy and the ability to conduct commerce is directly related to the rate at which information and ideas flow, an issue as critical as the means by which manufactured products are transported. There exists several technologies, when properly merged, that can solve existing information management problems.

The present invention is directed as a system and method of integration of existing technologies that will allow technically unskilled individuals to access, manipulate, and present multimedia information in a fashion that dramatically increases the rate of information flow to its final destination while decreasing the consumption of resources. Numerous corporations have focused their efforts on a new technology presently called "multimedia". Multimedia is the integration of text, graphics, animation, video, and audio through the use of a computer. These companies have invested heavily to prove that multimedia will dramatically increase the rate of information flow, increase efficiency, reduce overhead, increase retention rates during training, and reduce the consumption of natural resources. "A picture is worth a thousand words" and when combined with text and audio increases the rate at which an individual can assimilate information. This combined with interactivity, allowing the user to navigate through information in a fashion more natural to the human mind, is the next generation of information management. However, with such a large number of independent efforts to mature this new technology, the vast majority of busy professionals that know they need the technology have found it impossible to find an easy to use turnkey solution.

To pick a single company's multimedia product means picking from a limited number of media formats. It also means dependence upon a skilled staff to operate the system that is purchased. What exists today is a proliferation of standards that are mutually exclusive. A video expert deals with storage formats like VHS, S-VHS, Hi8mm, Super Hi8mm, Betcam, D2, 8" laserdiscs, 12" laserdiscs (CLV or CAV) and several other formats. Video can be routed in an NTSC, PAL, Y-C, RGB, or HD format. One must deal with television standards like NTSC, PAL, and HD and concerns over frequency ranges like VHF, UHF, and numerous cable channel frequencies. Computer video standards include EGA, VGA, SVGA, XGA, Mac, and many high resolution formats. Most video projection technologies or large screen displays such as flat Plasma Display Panels (PDP), etc... only support a limited number of these formats. Audio comes in the form of cassette tape, DAT, CD, and numerous digital file formats. Audio is transmitted using various modulation techniques to include AM and FM at various frequencies. The proliferation of networking electrical standards like LocalTalk, Ethernet, TokenRing, and FDDI as well as numerous protocols and file servers, forces a user to make more choices that further isolates them from some day tapping into a national information highway. Computer operating systems like DOS, Windows, Windows NT, OS/2, Macintosh System 7.1, Unix, X Windows, and Next all create information islands. The data storage, file format, MODEM, FAX, and optical disc technologies are also so numerous that to pick any single format increases the "Tower of Babel", unregulated, incompatible information maze that grows on a daily basis in this country. The present invention makes it possible to become independent of these compatibility issues.

II. Summary of the Invention

The principal object of the present invention is to provide a system of integrating existing computer, video, audio, and communications technologies. In a preferred embodiment this integration will take the form of a multimedia information management station that allows a user to manage multimedia information in a fashion that requires no programming, no scripting, no menu bars, and no exhaustive training.

Another principal object of the present invention is to eliminate the dependence upon a single variant of the following formats, standards, and technologies in comprising the multimedia information management station: a. Computer platform (Apple®, IBM® and Compatibles, Sun®, etc.). b. Computer operating system (Macintosh®, Windows®, OS/2®, Unix®, etc.). c. Video format. d. Audio format. e. Optical disc format. f. Networking standard. g. Wireless communications standard, h. Software application file format. i. Video display technology, j. Video projection technology, k. Computer based pointing device.

It is also an object of the present invention is to provide support for the control of the below listed components and processes through a single computer "pointing device". The "pointing device" being in the form of, and not limited to, a touch screen, mouse, track ball, digitizing pad, infrared remote, or radio frequency remote, all of which pass horizontal and vertical coordinates to a host computer. a. Read only and writeable video laserdisc players. b. Read only and writeable CD or CD-ROM players. c. Video cassette or tape recorders. d. Audio cassette or tape recorders, e Video cameras. f. Microphones. g. Audio mixers. h. Audio speakers (volume control), i. Video projection systems. j. MODEMS. k. Facsimile machines.

1. Document scanners. m. Slide scanners. n. Video digitizers and manipulation of digitized video. o. Audio digitizers and manipulation of digitized audio. p. Computer networks. q. Video encoders. r. Video switching matrices. s. Audio switching matrices. t. Room lighting. u. Room temperature. v. Cable TV selection. w. Televideo conferencing. x. Satellite communications. y. Radio frequency communications. z. Multimedia station ergonomics.

A further object of the present invention is to provide a means to present multimedia information in a non-linear fashion supporting a level of interactivity that allows both the presenter/instructor and audience/student to search for and assimilate information in a more intuitive and natural fashion.

The foregoing objects are accomplished by the proper integration of the following components: a. Control System - The control system includes a microprocessor with a bus architecture that supports control of a wide variety of cards to include RS-232, Infrared In/Out, Volume Control, Relays, MIDI, SMPTE time code, voltage ramping etc. The microprocessor must be programmable and support at minimum an 8 bit data bus. Many manufactures offer solutions with various processors and card options which are suitable for the requirement of this component of the invention. b. Computer(s) - The computer can be of any manufacturer with any operating system that preferably supports the following:

(1) 32 Bit Bus architecture (16 bit if cost is more important than performance).

(2) 40 MBytes or more of RAM (less if cost is a factor).

(3) Hard disk drive with preferably a minimum 200 MByte capacity.

(4) CD-ROM with 300 KByte/second transfer rate (w/CD support).

(5) Support for two 24 Bit color video displays at 640x480 resolution.

(6) Video digitizing at 30 frames per second.

(7) Audio digitizing at up to 44.1 KHz, 16 bits per sample.

(8) Two serial communications ports.

(9) Pointing device.

(10) Keyboard, pen based, or voice recognition text entry.

(11) SCSI bus port.

(12) Stereo audio out port.

(13) Graphical user interface software library.

(14) Support of LocalTalk, Ethernet, TokenRing, or FDDI. c. Video Laserdisc Player/Recorder. d. Video Cassette Recorder. e. Speaker/Amplifier combination. f. Video Encoder. g. Video switching matrix, h. Audio switching matrix, i. TV Tuner. j. Connection patch panel (internal/external source connections), k. Optional miscellaneous components as required to include the following:

(1) Audio cassette or tape recorders.

(2) Video cameras.

(3) Microphones.

(4) Audio mixers.

(5) Video projection systems.

(6) MODEMS.

(7) Facsimile machines.

(8) Document scanners.

(9) Slide scanners.

In the preferred embodiment these components are integrated into a single wheeled cabinet that allows the system to be transportable. Preferably, the cabinet top supports two video display screens (with touch screen capability), keyboard, and trackball. One display screen is dedicated to viewing the "presentation" or displaying exactly what is projected if the output is connected to a video projector, large screen video monitor, or flat PDP displays through the supplied connection patch panel. The other video display screen is a virtual interface or control display screen that controls all components integrated within the cabinet. During a presentation, the user can randomly select any portion of a multimedia presentation from the control display screen allowing the presentation to be driven by feedback from the audience. This allows the presentation to be non-linear. During the creation of a multimedia presentation, or multimedia training material, the two video display screens are used as virtual interfaces for manipulating the multimedia information that comprises the final presentation.

The cabinet houses the control system, the video switching matrix, audio switching matrix, the user's choice of video devices, the user's choice of audio devices, and the users choice of computer platforms. The preferred embodiment includes an Apple Macintosh using a Motorola 68040 microprocessor (or better) and an IBM compatible using an Intel 80486 microprocessor (or better). Sun, Silicon Graphics, and other work stations could be used. Between these two platforms, the Macintosh OS, DOS, Windows, Windows NT, OS/2, and Unix operating systems can be supported, giving the user access to the largest possible spectrum of computer based information. The entire system is integrated to the point that the user need only plug in a retractable power cord and depress the single power switch to begin using the system.

The software for the Control System resides in the Control System's memory and manages all requests from the computers to adjust volume, control a video device, switch the video matrix, dim the lights, close a curtain, etc... The Control System also supports a large quantity of digital alphanumeric touch pads or data entry devices which would allow students or members of an audience to pass data to the software executing on the computers. This supports significant interactivity between the presenter and their audience. The software that resides on the computers allows a pointing device to control volume, create multimedia presentations, play video tapes, run a televideo conferencing session, create a commercial for cable TV, give a test to students, etc... The interface chosen for the software that resides on the computers has been developed to ensure a limited amount of complexity is present at any time, actions are intuitive, and the user need not understand programming, scripting, or the concept of menu based commands in order to make use of the system.

III. Brief Description of the Drawings

The above and other objects advantages and features of the invention will become more apparent when considered with the following specification and accompanying drawings wherein:

Figure 1-A is a diagrammatic illustration of a preferred embodiment of the invention showing the multimedia management system controlling what is projected onto a large screen display with the seating of the audience,

Figure 1-B is a diagrammatic illustration of a user's perspective of the preferred embodiment of the invention showing two touch screen video displays, the keyboard, and a trackball or pointing device,

Figure 1-C is a diagrammatic illustration of a user's perspective of the preferred embodiment showing a user controlling the system via the touch screen video display,

Figure 1-D is diagrammatic view showing an audience perspective of the preferred embodiment showing the stereo speakers,

Figure 2 is a block diagram of a multimedia information management system incorporating the invention,

Figure 3 - 7 are explanatory block diagrams, Figure 3 showing the composition of the control system and the paths of control for computer video signals; Figure 4 illustrates the signal paths of the NTSC video out routing; Figure 5 illustrates the signal paths of the video in routing; Figure 6 illustrates the signal paths of the stereo audio out routing; Figure 7 illustrates the signal paths of the stereo audio in routing,

Figure 8 is a view showing a typical presentation to a user on the virtual interface or control screen with a touch entry device and associated control icons displayed thereon for controlling all components integrated in the system,

Figure 9 is a view showing the typical presentation menu according to the invention,

Figure 10 is a view showing a typical slide menu according to the invention, and

Figure 11 is a view showing the typical edit menu according to the invention.

IV. Detailed Description

System Overview. Referring to Figures 1-A through 1 -D, cabinet 1 is preferably provided with wheels W, a podium configuration on top of the cabinet supporting left and right touch capable video display screens 4L and 4R, keyboard 5 and a trackball 6. A compact video camera (not shown) can be mounted by a flexible goose neck support post (not shown) to provide live information for display on screen 2 via the user's choice of video projection system (not shown). The left 4L or right 4R touch capable video display screens can be used as the virtual interface or control display screen, while the other screen lets the user see what is being projected to the audience . Cabinet 1 houses the control system 9, computer platforms 10-1 and 10-2, audio and video switching matrices 19 and 20, the users choice of video and audio devices, video A/B switches 11-1, 11-2, and 11-3, video distribution amplifiers 12-1 and 12-2 which are coupled to left and right video display 4L and 4R (13-1 and 13-2 in Figure 2), and a video encoder 15. The cabinet may also house output only video/audio devices 16 (video laserdisc for example), video cassette recorders 17 for making a permanent record of the information presented to the audience or as a source of information to be merged by the system of this invention. Device 17 can also be an audio source. Cabinet 1 also houses the audio amplifier/speaker combination 18, and external connectivity jacks or connection panel 21 and internal connectivity 22 for the internal video/audio paths. Drop leafs 1DL and 1DR provide additional work space for the user.

Video displays 4L and 4R (13-1 and 13-2 in Figure 2) for space reasons are preferably high quality flat panel displays and are mounted in panel TP which, for ergonomic reasons, can be swung forward on a hinge TPH and lockable in the adjusted position by a latch rod (not shown). Moreover, while displays 4L and 4R are shown laterally aligned, they may be vertically aligned. In fact they need not be aligned and may be in individually adjustable mounts so the user can adjust them to best accommodate his or her comfort.

As shown in Figure 1A the cabinet is positioned at any desired position by the user in relation to an audience 3, the members which of which are provided with data entry devices (not shown) to provide interactive responses to the material being presented to them. The speakers 8 are preferably facing the audience 3. Likewise the large area display screen 2 is positioned for easy viewing by the audience. As discussed above, the large area display screen will have the same information displayed therein as one of screens 4.

Control Integration and Computer Video Routing. The control system 9 manages the communication between and control of all devices within the system. Comprised of a microprocessor 24 that controls a wide variety of control and interface cards on a common bus 23. The bus architecture 23 need only handle data consisting of ASCII text strings and to address at least 255 devices to ensure sufficient expendability to support a wide range of applications. Microprocessor 24 is programmable, has ROM to hold a minimal operating system, and has battery backed RAM to hold an executable program. The executable program runs in a continuous loop monitoring a register. The register holds ASCII commands strings sent by the attached computer platforms 10-1 and 10-2. These command strings consist if an initial device code, a device number if more than one of a specific device exists, and a command. The executing program interprets this command and executes a subroutine appropriate for the device. A command structure has been developed that reduces the complexity of the code in computer platforms 10-1 and 10-2. For example, to play a video cassette, only VCR00PL is sent. The device is VCR, the particular VCR is 00 (indicating only one or the first VCR), and the command PL for play. PL is sent to play any type of device regardless of what is "plays". The subroutine that exists for VCR00 knows what commands to send over the RS-232 card (25-4 Figure 3) to get that device to play. Since many types and brands of devices have the same functionality the level of complexity is abstracted by using a lookup table for commands like PL (play) so that the computer platforms need only send PL and the control system does the lookup and control. A plurality of RS-232 interface cards 25-1 to 25-5 are provided and support communication with computer platforms 10-1 and 10-2, video devices 16 and 17, audio devices, and switching matrices 19 and 20. Relay card 26 allows TTL voltage levels to be set to control external devices. Volume control card 27 takes in an audio voltage level and ramps it up or down at a selected rate via ASCII commands sent by the microprocessor 24. This allows a touch screen to control audio volume (see Figure 8). The computer platforms 10-1 and 10-2 are integrated into the system via RS-232 serial port connections 25-1 and 25-2. The computer platforms 10-1 and 10-2 support two displays, one for letting a presenter or multimedia author see the final result of the presentation and the other to provide the controls that manage the presentation being given or developed. Two input one output switch boxes 11-1 , 11-2, and 11-3, under control of the relay card 26, allow a user via a touch screen to decide which computer platform controls the video displays 4L and 4R and video projector 14.

This allows rapid access to large amounts of information in various formats via the touch screen. The video distribution amplifiers 12-1 and 12-2 are required to allows the analog computer video signal to drive both video display and video projector 14. It will be appreciated that digital displays can be incorporated in the invention. Video displays 13-1 and 13-2, which can be color multisync monitors, either CRT, liquid crystal or plasma, support the virtual interface for control of all components integrated by the system. Any video projection system

14 or large screen display, can be used to get the desired information to a large audience. A video encoder 15 is used to convert computer video signals to a form that can be routed, transmitted, or stored to traditional video tape formats.

Any video output device that can be controlled via infrared or RS-232 can be integrated into the system of this invention. Any amplified stereo speaker system

18 can be controlled by the system of this invention. A video and audio switching matrices 19 and 20 are used to route the various stereo and audio signals to and from the devices that need it.

NTSC Video Out Routing (Figure 4). Each computer platform incorporates two video digitizers combined with two video display cards to display video in the computer video format on the two video displays 4L and 4R. Both displays require the video 30-1 and 30-2 be routed to them for digitization. This configuration supports each display to contain a different video source. Any device that can record video 31 can receive any available video source. Most video projectors handle both computer video signals as well as NTSC video which can both be routed to the projector. Providing a means to route video external 33 to the system allows a user to create a presentation then get it to any device that can transmit, route, or store that video 31.

NTSC Video In Routing (Figure 5). The video switching matrix 20 can accept the video output (34, 35, 36, 37) of any device in the system. The encoded computer video signal 36 can be routed anywhere desired once put in the NTSC video format. Accepting extemal video signals 37 supports the addition of cameras, televideo conferencing systems, etc...

Stereo Audio Out Routing (Figure 6). The computer platforms 10-1 and 10-2 must be able to digitize stereo audio 38-1 and 38-2 for its integration into a presentation or multimedia database. Any device that can record audio can receive any available audio source. The volume control card 27 receives the desired audio signal 40-1, adjusts its volume by ramping up or down the voltage, them outputs that signal 40-2 to a fixed gain amplifier 18. Providing a means to route audio external to the system allows a user to create a presentation then get it to any device that can transmit, route, or store that audio.

Stereo Audio In Routing (Figure 7). The computer platforms 10-1 and 10-2 output their own audio (internal speaker) 42-1 and 42-2, audio from internal CD players, and audio that is stored in digital format and converted to analog. The audio switching matrix 19 can accept the audio output of any device in the system. Accepting external audio signals 45 supports the addition of any device from radios to microphones that output an analog audio signal.

Computer Platform Software (Figure 8-11). The software residing on computer platforms 10-1 and 10-2 has two integrated modules. The first is a virtual control interface (Figure 8) that allows generic control of all components within the system. The user can control a particular device by selecting its icon 46 and controlling it with the various controls 49. If the device provides video it can be viewed 50. A user has full control over audio, both its volume 47 and its source 48. The user can play a VHS tape, play a CD, play a laserdisc, create a QuickTime movie, control a camera, change a cable TV channel, etc... Also a user can create an Edit Decision List (EDL) that stores from and to addresses (video/audio clip or sequence) from video tapes, laserdiscs, or CDs. The EDL for a video tape or other media allows a user to play, delete, record, digitize, or move clips or sequences of video and audio.

The second module allows a user to rapidly create a multimedia presentation taking advantage of all of the components within the system. A slide show metaphor is used. A group of slides forms a presentation. An element of a presentation, the slide, contains backgrounds, titles, other forms of text, video, audio, animations, charts, graphics, and interactive buttons. Figure 9 shows the presentation menu 51 and the ability to create, edit, delete, select, and rename a presentation. When a presentation is opened the slide menu (Figure 10) shows the interface for creating and editing slides. Slides are shown in miniature for ease of identification 57 and the slide to be edited or shown must be selected 58. A master slide is used when a slide element is added that is to be part of all slides 56. Finally, the edit menu Figure 11 shows a slide being edited. This interface allows all the elements of the slide to manipulated as desired. This slide example shows and image imported from a Kodak Photo CD 61, a chart that was made from within the system 62, and the first frame of a video clip 63 which will become active when selected by the user. The slide is shown scaled down to allow the user access to controls, the slide can be shown full screen at any time desired. The system is designed for ease of use and does not require the user to program, write any scripts, or be familiar with any sequence of keyboard commands.

While preferred embodiments of the invention have been shown and described, it will be appreciated that various modifications and adaptions of the invention will be obvious to those skilled in the art. What is claimed:

Claims

1 . A multimedia management system, comprising: a plurality of host computer platforms and operating system software therefor, two or more video display screens and means for connecting said video display screens to said host computer platforms, a plurality of media sources, including, video, audio, and communication sources, a pointing device including means providing horizontal and vertical coordinates to a selected computer platform, and a controllable interface means coupling said media sources to a selected host computer platform.

2. The multimedia management system defined in claim 1 including large screen display means and means for causing said' large display screen means to display the same information that is on a selected one of said two or more display screens.

3. The multimedia management system defined in claim 1 wherein said controllable interface means includes a programmable microprocessor and a bus system, a plurality of RS-232 interface cards connected to said bus system, said RS-232 interface cards being connected on a one-to-one basis with said plurality of host computer platforms and said plurality of media sources.

4. The multimedia management system defined in claim 3 including a video switching matrix and an audio switching matrix for selectively coupling said plurality of media sources to a selected host computer.

5. The multimedia management system defined in claim 1 wherein two of said two or more display screens are positioned for common viewing by a user of said system, and said pointing device is a touch panel device associated with one of said two display screens positioned for common viewing by said user.

6. The multimedia management system defined in claim 2 wherein said large screen display means is selected from a group comprising projection systems and a large area flat matrix display panels and is positioned for viewing by an audience.

7. The multimedia management system defined in claim 6 including speaker means associated with said large screen display means.

8. The multimedia management system defined in claim 7 wherein members of said audience are provided with a data entry device to record their responses to information presented on said large screen display and/or by said speaker means.