WO1997000491A1

WO1997000491A1 - Method and apparatus for providing independent audio in multiple listener digital sound systems

Info

Publication number: WO1997000491A1
Application number: PCT/US1996/010450
Authority: WO
Inventors: Andrew L. Carter
Original assignee: Apple Computer, Inc.
Priority date: 1995-06-15
Filing date: 1996-06-17
Publication date: 1997-01-03
Also published as: AU6334396A

Abstract

The present invention has application to multiple listener digital systems. A central processing unit (CPU) is coupled to a mass storage device, such as for example, a read-only memory multimedia cartridge, CD-ROM, floppy disk or hard disk drive. The mass storage device stores an application audio program which is executed by the CPU and further stores a library of sound data. A sound synthesizer circuit is coupled to the CPU for generating digital audio signals from the sound data stored in said mass storage device. Under the control of a single executable program, the CPU selectively provides sound data for a listener of the audio program to the sound synthesizer circuit. A digital to audio converter (DAC) is coupled to receive the digital audio signals from the sound synthesizer circuit and convert the digital audio signals into analog audio signals. A listening device, such as a headphone, is coupled to the DAC to convert the analog audio signals into sound waves to be heard by the listener. Under program control the CPU directs certain of the sound data to each of the listeners, such that the listeners hear separate and independent audio as required by the audio program.

Description

METHOD AND APPARATUS FOR PROVIDING INDEPENDENT AUDIO IN MULTIPLE LISTENER DIGITAL SOUND SYSTEMS

1. FIELD OF THE INVENTION: The present invention relates to the field of digital audio systems, and more particularly, to audio systems used in digital devices such as personal computers, workstations, network servers and dedicated multimedia game devices and any other digital device where it may be desirable to provide separate audio to individual listeners.

2. ART BACKGROUND:

Historically, multimedia game systems utilized two dimensional landscapes and iconic bit mapped action characters. Game players viewed this two dimensional world through a low resolution cathode ray tube (CRT) which typically comprised a home television set. Software content was limited by low resolution graphics, low processor speeds, and in some instances, monaural audio provided to the game players.

Recent innovations to both home cartridge-based game systems and game programs executed by personal computers (PCs) have changed many ofthe systems' characteristics. Today, richly-textured, three dimensional graphics supported by hardware chip sets supplement or replace older bit mapped two dimensional graphics. CD-ROM players have expanded storage capacity and the addition of modems for network game playing expands the number and geographical sphere of players. These recent innovations have effectively eliminated the traditional boundaries between games viewed on televisions and those viewed on personal computers.

However, although the sophistication of game programs and the displayed graphics have markedly improved from earlier generations of cartridge and PC-based systems, audio quality has generally lagged. More particularly, prior art monaural or dedicated game devices and personal computer systems typically provide a single monaural or stereo output to all ofthe players ofthe game simultaneously. Thus, each of the players of a multimedia game perceive the same audio experience. Although the audio may be stereophonic and 16 bit CD quality, it does not vary between the players.

As will be described, the present invention provides an enhanced audio experience for each ofthe players of a multimedia game heretofore unknown in the prior art. By providing separate and independent audio to each of the players of either a dedicated game machine or personal computer, the game experience is enhanced and tailored to the particular player. The present invention permits separate and independent audio to be perceived by each player, or groups of players, which may be a function ofthe game, of the players' language, the players' ability to handle more complex audio feedback, and or the players' technical understanding. It will be appreciated that the present invention is useful for a wide variety of applications apart from games. Multiple user application programs, such as for education, may prove more effective if the individual users may be provided with individualized sound tracks and the present invention may be employed in all such applications. It will be further appreciated that the present invention may be used in conjunction with non-interactive application programs. Indeed, the present invention may be employed with any type of digital device where it is desirable to provide separate and independent sound to individual listeners or groups of listeners.

SUMMARY OF THE INVENTION

The present invention has application to any digital system such as a personal computer, dedicated multimedia game system, network server, workstation and the like wherein a single executable program at least partly controls the routing of separate and independent sound data to particular listeners. A central processing unit (CPU) may be coupled to a mass storage device, such as for example, a read-only memory game cartridge, CD-ROM, floppy disk or hard disk drive or remote network server. The mass storage device stores a program (hereinafter referred to as an "audio program") that includes audio components which is executed by the CPU and may also store a library of sound data. It will be appreciated that the CPU may be coupled with any type of sound source such as a real time sound source. In a preferred embodiment, a sound synthesizer circuit is coupled to the CPU for generating digital audio signals from the sound data stored in said mass storage device. Under program control, the CPU selectively provides sound data for a listener of the audio program to the sound synthesizer circuit. A digital to audio converter (DAC) is coupled to receive digital audio signals from the sound synthesizer circuit and convert the digital audio signals into analog audio signals. In the present embodiment, a listening device, such as a headphone, is coupled to the DAC to convert the analog audio signals into sound waves to be heard by the listener. Under program control the CPU directs certain portions of the sound data to each of the listeners, such that the listeners hear separate and independent audio as required by the program.

In a first embodiment, the DAC provides a stereo output having left and right channels. The CPU directs sound data for a first listener to the left channel and sound data for a second listener to the right channel. The two listeners each control separate multimedia controllers and wear headphones coupled to the multimedia controllers. The headphones for the first listener are coupled to the left output channel of the DAC and the headphone for the second listener are coupled to the right output ofthe DAC. Thus, each listener hears a separate and independent monaural channel of audio. In a second embodiment, the stereo output ofthe DAC is coupled to an analog multiplexer which is under the control of the CPU. A plurality of listeners operate multimedia controllers which are coupled to the multiplexer. The CPU directs sound data destined for each listener independent of sound data destined for other listeners. The CPU controls the multiplexer selection such that the appropriate multimedia controller is selected to receive the desired audio at the appropriate time. As in the first embodiment, the listener's headphones are coupled to the multimedia controller such that the listener hears audio intended for that listener independent of the other listeners.

In a third embodiment, the sound synthesizer circuit is coupled to a sound bus. Separate DACs are provided for each hand controller and are coupled between the audio bus and the hand controllers. A DAC control bus is provided which is coupled to the CPU and enable circuits for each of the DACs. In operation, the CPU executing the audio program provides sound data to the sound synthesizer circuit which provides its digital audio signal output to the audio bus. Each of the DACs are selectively enabled by a control signal coupled over the control bus by the CPU, such that substantially concurrent with the sound synthesizer circuit providing digital audio signals to the sound bus, the CPU enables the appropriate DAC corresponding to the desired destination multimedia controller and headphone.

In a fourth embodiment, the sound synthesizer circuit is coupled to a digital input device controller which is in turn coupled to a bi-directional high speed digital bus. Each multimedia controller is coupled to the bus and includes a microcontroller and interface logic to receive and send data over the bus. The microcontroller is coupled to a DAC to which a listener's headphones are in turn coupled. In operation, an audio program executed by the CPU utilizes sound data stored in the mass storage device. In accordance with program commands, the CPU forwards sound data to a particular multimedia listener operating one of a plurality of multimedia controllers coupled to the bus. The sound synthesizer circuit converts the sound data into digital audio signals, which are encapsulated into packets by the digital input device controller in accordance with the particular protocol of the bus. The packet includes address information unique to the destination multimedia controller. The packet is received by the destination multimedia controller and decapsulated to recover the digital audio signals which are then coupled to the DAC. The DAC converts the digital audio signals into analog audio signals which are provided to the listener's headphones.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a block diagram illustrating a first embodiment of the present invention wherein monaural audio is provided to each listener. Figure 2 is a block diagram of a second embodiment of the present invention in which stereo sound is provided to each listener under the control of a central processing unit.

Figure 3 is a block diagram illustrating a third embodiment of the present invention wherein multiple digital to audio converters (DACs) are coupled to an audio bus, and wherein each of the DACs are under the control of a central processing unit.

Figure 4 illustrates a block diagram of a fourth embodiment of the present invention wherein each of the listeners' controllers are coupled to a high speed digital bus. Figure 5 is a block diagram of a multimedia controller utilized by the present invention's fourth embodiment shown in Figure 4.

DETAILED DESCRIPTION OF THE INVENTION The present invention discloses methods and apparatus for providing separate and independent audio experiences to persons listening to the audio output of a digital system. In the following description, numerous specific details are set forth, such as system architectures, representative devices, and exemplary hardware configurations, etc. to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details.

In other instances, well-known circuits and structures are not described in detail in order not to obscure the present invention unnecessarily. For example, since the present invention relates to the audio component of a digital system, various video, control and other subsystems which are common to various types of digital systems are not shown in the figures or described. It will be appreciated by one skilled in the art that these additional subsystems for video, video compression, system integration and control may be necessary to integrate the present invention into a particular digital system although the present invention is not limited to digital systems including video subsystems. Similarly, power, control and other lines and devices are not shown in the drawings, but their addition will be apparent to one skilled in the art based upon the description of the present invention in this Specification. Moreover, although the present invention is described with reference to a personal computer, it should be noted that the teachings of the present invention may be dedicated multimedia systems, other general purpose digital computers, DVD players (digital video disk players), set top boxes and the like, which are configured by either hardware or software to run programs that include, either directly or indirectly, sound information. As used in this specification, the term "multimedia controller" includes human input devices such as hand-held game controllers, joysticks, remote controls, keyboard and similar devices.

Referring now to Figure 1, a first embodiment of the present invention is illustrated. A central processing unit (CPU) 10 runs a single instance of an executable program that at least partly controls the routing of separate and independent sound data to particular listeners. The present invention is thus in contrast to network games such as "Marathon" where multiple copies of an executable program run on separate computers and each copy ofthe program provides sound data to a user through his or her associated computer. Although, according to the present invention, a single executable program, as opposed to multiple copies of the same program, provides separate sound to listeners, it will be appreciated that the single program may be run across multiple processors within a single digital device or within a network. For example, a single computer may use multiple processors to execute a main program. For another example, a server may provide distinct audio programs for separate users of a game, movie or such. Each user may have a separate computer that translates, extracts or otherwise processes the audio program for that user but the primary program is running on the server.

The CPU 10 is coupled through supporting circuitry (not shown) to an input/output (I/O) circuit 12. The I/O circuit 12 is used to communicate information in appropriately structured form to I/O devices and to provide a video output to a display 14. Where the present invention is applied to a personal computer, the display 14 may comprise a computer monitor. As shown, the visual output of an audio program, such as for example, a multimedia game, an educational program or the like is displayed on the display 14. Although Figure 1 illustrates a multimedia system, it will be understood that the present invention is applicable to digital systems that do not have video components. A mass data storage device 16 is coupled to the CPU 10. The mass data storage device 16 may comprise a hard disk drive, a read-only memory (ROM) multimedia cartridge, a CD-ROM, remote network server or other read-only or writable mass storage device. The mass storage device 16 stores the application program which is executed by CPU 10. As such, the CPU 10 is under program control in accordance with the instructions stored in the mass storage device 16.

Mass storage device 16 further stores a library of sound data which is used by the particular program executed by CPU 10. The sound data stored on the mass storage device 16 may comprise sound data, analog sound data, MIDI code sequences and the like. Therefore, it is contemplated that whatever form mass storage device 16 takes, the necessary application program, images (if any), and sounds required by the particular application are stored therein and are accessible to the CPU 10. It will further be appreciated that in the embodiments disclosed in this Specification, that the application program, and/or the sound data, may be downloaded from a network for storage in the mass storage device 16 or for direct execution by the CPU 10. It will be further appreciated that in the embodiments the CPU 10 need not be coupled to a mass storage device that stores sound data and may be coupled with any type of sound source such as a real time sound source which may be natural or synthetic.

As illustrated in Figure 1, a sound synthesizer circuit 18 is coupled to, and under the control of, CPU 10. In practice, the sound synthesis circuit 18 may comprise digital hardware or be implemented in software to achieve substantially the same functionality. The sound synthesizer circuit generates the required audio output under the control of CPU 10 utilizing the sound data stored in the mass storage device 16. The output ofthe sound synthesizer circuit 18 is coupled to a digital to audio converter (DAC) 20, which receives the digital output ofthe sound synthesizer circuit 18 and converts the digital audio data into analog electrical signals.

As shown in Figure 1, the output of the DAC 20 comprises left (L) output 22 and right (R) output 24. In prior art systems, the left 22 and right 24 outputs comprise a stereo audio output which was perceived by all listeners ofthe multimedia system. However, in accordance with the teachings of the present invention, the left 22 and right 24 outputs ofthe DAC 20 are coupled to separate multimedia controllers 26 and 28, as shown in the figure. As is well-known, multimedia controllers 26 and 28 are used by listeners of the multimedia application to interact with CPU 10 through the I/O circuit 12. Depending on the particular multimedia application program executed by CPU 10, each of the respective listeners must provide appropriate inputs using multimedia controllers 26 and 28. As illustrated, a headphone 29 is coupled to the multimedia controller 26, and a headphone 32 coupled to the multimedia controller 28. In the embodiments disclosed in this Specification, it will be appreciated that the separate sound data need not be provided through a headphone but may be provided, for example, to separate groups of listeners in different rooms. It will further be appreciated that, as previously mentioned, the present invention is in no way limited to interactive applications or programs that include a video component. In accordance with the teachings of the present invention, rather than providing a stereophonic audio output to both listeners, this first embodiment provides independent audio outputs in monaural form to each of the respective listeners. Utilizing the teachings ofthe present invention, it is contemplated that the sound data stored in the mass storage device 16 will be selectively provided, that is routed, to each ofthe listeners by CPU 10 as required by the particular audio program. Thus, the left and right channels output different audio experiences to the respective listeners.

For example, the application may comprise a program that allows two or more users to simultaneously create a multimedia application. One user may be assigned the task of creating shape outlines while the other user may be assigned the task of filling in the outlines. The users may be provided with audio feedback to indicate certain events, such as when a boundary has been inappropriately crossed. To avoid confusion, it would be desirable to provide the users with separate audio so that each user may concentrate on his or her own creation process.

Thus, a user operating multimedia controller 26 would hear sounds unique to his or her own creations. Similarly, a user operating multimedia controller 28 would hear sounds unique to his or her own creations. Each user would not necessarily hear the audio provided to the other user, thereby avoiding confusion, although in certain applications it may be desirable for both users to hear some common sounds. These common sounds may be supplied under the control or direction of CPU 10, or alternatively, may be provided using, a separate line, or using a sum or difference technique known in the Art. The invention also contemplates providing two or more channels of sound in each output so each user may experience stereo or multichannel sound.

Continuing to refer to Figure 1, it will be noted that both headsets 29 and 32 are illustrated as being coupled to the multimedia controllers 26 and 28, respectively, through a wire coupling. It will be appreciated by one skilled in the art that in the embodiment of Figure 1, as well as in the other embodiments disclosed in the Specification, both the multimedia controllers and headphones may be coupled to the multimedia system in a wireless manner. For example, it is contemplated that the multimedia controllers may communicate with the multimedia system, and/or the headphones may receive audio signals, either from the multimedia system or from the multimedia controllers using, for example, infrared or RF transmission mediums. Referring now to Figure 2, therein is disclosed a block diagram illustrating a second embodiment ofthe present invention. As in the embodiment shown in Figure 1, a CPU 40 is coupled to an I/O circuit 42. The I/O circuit 42 is coupled to a display 44 for displaying the visual output of an application program executed by the CPU 40. A mass storage device 46 stores the audio program for execution by the CPU 40 as well as sound data to provide the audio experience for the application program. A sound synthesizer circuit 50 is coupled to the CPU 40 for generating a digital representation of the required sounds under the control of the CPU 40, and utilizing the sound data stored in the mass storage device 46.

As shown, a DAC 52 is coupled to the sound synthesizer circuit 50 for converting the digital output of the sound synthesizer circuit 50 into an analog output. An analog multiplexer 55 is coupled to the output of the DAC 52. In addition, a multiplex control line 56 is provided between the CPU 40 and the analog multiplexer 55. The analog multiplexer 55 is coupled to multimedia controllers 60, 62, 64 and 66. Headphones 68, 70, 72 and 74 are coupled to the respective multimedia controllers as shown in the figure. Although the analog multiplexer 55 is illustrated as a four-way multiplexer in Figure 2, it will appreciated by one skilled in the art that the multiplexer 55 may have fewer or greater number of multimedia controllers coupled to it as required by a particular multimedia system or application. The multimedia controllers illustrated in Figure 2 are coupled to the CPU 40 over a multimedia control bus 80 through the I/O circuit 42. It will be appreciated that the multimedia control bus 80 may comprise an analog parallel bus or, with appropriate circuitry and supporting logic, a digital serial or parallel bus. In addition, although Figure 2 illustrates the multiplex control line 56 coupled between the CPU 40 and the analog multiplexer 55, it will be noted by one skilled in the art that additional circuitry may be required (and is not shown) in order to provide control over the analog multiplexer 55 by the CPU 40. However, the additional support circuitry required to implement the present invention as illustrated in Figure 2 is well-known, and therefore, not described in this Specification. In operation, separate and independent audio experiences are provided to each of the users operating the multimedia controllers 60, 62, 64 and 66 by multiplexing the audio output from the DAC 52. For example, audio to be routed to a user operating multimedia controller 60 is provided by the CPU 40 to the sound synthesis circuit 50 and the DAC 52. The output of the DAC 52 is coupled to the analog multiplexer 55 which is selected by the CPU 40 to provide its output at the appropriate time to the multimedia controller 60 and headphones 68.

Although not shown in the figure, it may be desirable depending on the speed of the audio output to buffer the output ofthe analog multiplexer 50, or sound synthesis circuit 52, to achieve proper timing and realism in multimedia playback. In addition, in certain applications, it may be desirable to group multiple users into teams which receive the same audio experience. For example, in the embodiment illustrated in Figure 2 the multimedia application may group users operating the multimedia controllers 60 and 62 as a first team, and users operating the multimedia controllers 64 and 66 as a second team. As such, the analog multiplexer 55 need only switch between two groups of users thereby reducing the speed requirements of the system. In multimedia applications, such as for example a helicopter gun ship squadron battling a tank squadron, the users operating within each group of multimedia controllers would hear only audio signals appropriate for their group. Thus, the users commanding the helicopter gun ships will hear only helicopter gun ship audio, and the tank users will hear only tank communications and other audio associated with the tank group. Of course, it will be understood that there may be instances where it is desirable to provide some common audio to all users. In such event, either a separate common line may be used between the multiplexer and the multimedia controllers, or altematively, the multiplexer may sequentially provide the common audio to each multimedia controller.

Referring now to Figure 3, a block diagram is provided illustrating a third embodiment of the present invention. As in the embodiments of Figures 1 and 2, a CPU 100 is coupled to an I/O circuit 102. The I/O circuit 102 is coupled to a display 104 for displaying the application program executed by the CPU 100 under the control of software stored in, for example, a mass storage device 106. The mass storage device 106 further stores sound data required by the particular audio program executed by the CPU 100. As shown, a sound synthesizer circuit 108 is coupled to the CPU 100 for generating digital representations of sounds required by the program using, for example, the audio data stored in the mass storage device 106. The sound synthesizer circuit 108 is coupled to an audio bus 110. DACs 112, 114, 116 and 118 are coupled to receive the output ofthe sound synthesizer circuit 108 over the audio bus 110. Although four DACs are illustrated in Figure 3, it will appreciated that depending upon the particular multimedia system and/or program, any number of DACs may be coupled to sound bus 110.

As illustrated, multimedia controllers 120, 122, 124 and 126 are coupled, respectively, to the DACs 112, 114, 116 and 118. Headphones 130, 132, 134 and 136 are coupled to the respective multimedia controllers. A multimedia control bus 140 is provided to couple signals originating from each of the multimedia controllers to the CPU 100 through the I/O circuit 102. The multimedia control bus 140 may comprise an analog parallel bus, or alternatively, with appropriate digital logic, a digital serial or parallel bus. As shown in Figure 3, DACs 112, 114, 116 and 118 are each coupled to a DAC control bus 142. The CPU 100 is also coupled to each of the DACs over the DAC control bus 142.

In operation, CPU 100 executing the application program stored in the mass storage device 106 provides sound data to the sound synthesizer circuit 108. The output of the sound synthesizer circuit 108 is coupled to the audio bus 110. Sound data destined for a particular user is routed to that user by selectively enabling each of the DACs 112, 114, 116 and 118 by a control signal provided over the DAC control bus 142 from the CPU 100. Audio data directed to the user operating, for example, the multimedia controller 120 is provided over the audio bus 110 by the sound synthesizer circuit 108. Substantially concurrent to the sound synthesizer circuit 108 providing audio destined for multimedia controller 120 the CPU 100 provides an enable signal to the DAC 112 over the DAC control bus 142. Since only DAC 112 is enabled (DACs 114, 116 and 118 being disabled) audio data destined for multimedia controller 120 is only received by multimedia controller 120 and not the other multimedia controllers coupled to the system. Similarly, audio data destined for any ofthe other multimedia controllers illustrated in Figure 2 is provided to the desired multimedia controller through the selective enabling of the corresponding DAC by the CPU 100. Common audio data which is destined for all (or groups) of multimedia controllers is provided by enabling all, or a selected set, ofthe DACs.

It will be appreciated by one skilled in the art that it may be desirable to buffer the output of the sound synthesizer circuit 108 to ease the timing requirements for the enabling and disabling of the DACs. It is contemplated that the buffering of the DACs may be accomplished at a much faster rate than the sound output requirements ofthe multimedia controllers coupled to the system. In this manner, the audio output of each multimedia controller would approach that of a real time system, such that the user does not perceive any undesirable delays in receiving his audio output, and each listener will perceive simultaneous audio experiences despite the serial nature of the output ofthe sound synthesis circuit 108. Referring now to Figure 4 in conjunction with Figure 5, there is shown a fourth embodiment of the present invention. As in the embodiments illustrated in Figures 1 through 3, a CPU 200 is coupled to an O circuit 202 and to a mass storage device 204. The I O circuit 202 is coupled to a display 206 for displaying the visual portion of an application program executed by the CPU 200 and stored in the mass storage device 204. Sound data representing the audio portion of the program is also stored in the mass storage device 204, as previously described with reference to the other embodiments. A sound synthesizer circuit 208 is coupled to the CPU 200 for generating the required sound signals utilizing the sound data stored in the mass storage device 204.

A digital input device controller 210 is coupled to receive the digital audio output from the sound synthesizer circuit 208. In practice, it is contemplated that the digital input device controller may comprise hardware under the control ofthe CPU 200. The digital input device controller 210 is coupled to a digital bi-directional high speed bus 212. Multimedia controllers 214, 216, 218 and 220 are also coupled to the high speed bus 212. As illustrated in Figure 4 and as will be described more fully below, digital audio is coupled over the high speed bus 212 to the multimedia controllers, and listener commands are coupled through the digital input device controller 210 to the CPU 200. Headphones 222, 224, 226 and 228 are coupled to their respective multimedia controllers to provide independent audio to each ofthe listeners. As will be appreciated, the embodiment illustrated in Figure 4 is almost entirely digital. Selections made by users using the multimedia controllers 214, 216, 218 and 220 are coupled over the high speed bus 212 to the digital input device controller 210 and the CPU 200. Similarly, digitized audio to be directed independently to each ofthe users is coupled over the high speed bus 212 and addressed to the desired multimedia controller. It is contemplated that the high speed bus 212 may comprise the Apple® Firewire™ (IEEE 1394-1995), the Intel® Universal Serial Bus, or other bus having similar high speed capabilities.

Also shown in Figure 4 are optional microphones 230, 232, 234 and 236 coupled to each of the multimedia controllers. The embodiment illustrated in Figure 4 provides not only full stereo audio to each of the users, but also permits audio feedback from each of the users to the other users as may be desired in a particular multimedia or other application. It will be appreciated that other forms of sound, such as monaural sound, may be provided to each of the users.

In operation, a program executed by the CPU 200 utilizes sound data stored in the mass storage device 204 to generate unique independent sounds to each listener as provided by the application. The digital input device controller 210 handles the two-way communication of data and sound between the multimedia controllers 214, 216, 218 and 220. Illustrated in Figure 5 is a representative multimedia controller 214 which includes a serial bus interface cirr -40 coupled over line 242 to the high speed bus 212. A microcontroller 242 > .pled to the serial bus interface 240. The microcontroller 242 may comprise a ge* , ai purpose microprocessor, or altematively, may comprise a custom microcontroller. _ DAC 246 is coupled to the microcontroller 242 and to headphones 248. Sound data is provided from the sound synthesizer circuit 208 through the digital input device controller 210 and coupled over the high speed bus 212 to the serial bus interface 240. The microcontroller 242 further couples this digitized audio to the DAC 246 which converts the digitized audio into analog signals. A user utilizing the multimedia controller 214 will then hear independent audio intended for, and directed to, that listener using the headphones 248.

The multimedia controller 214 includes various input devices such as push button switches 260 and four-way rocker switches 262. A listener operating the multimedia controller 214 provides multimedia input commands to the CPU 200 through the use of switches 260 and 262. Signals generated by the switches 260 and 262 are coupled to the microcontroller 242 and are ultimately coupled through the serial bus interface 240 to the high speed bus 212. The digital input device controller 210 receives these input command signals and couples them to the CPU 200. Similarly, for audio input at the user level, microphone 230 is coupled to an analog to digital (A/D) converter 270, which is in turn coupled to the microcontroller 242. Audio input may thereby be provided to other users, or to the CPU 200, depending on the multimedia application.

Thus, it will be appreciated that the embodiment illustrated in Figures 4 and 5 is almost entirely digital in nature and provides separate independent audio to each of the users operating the various multimedia controllers. Since each multimedia controller is independently addressable, sound data packets, which include an indicator, such as an address, that correlates the particular data packet with a particular destination, may be directed independently to the multimedia controllers. Input command signals may be coupled back to the CPU 200 by the listener using switches 260 and 262, such that these desired input command signals are coupled in a packetized form over the high speed bus 212 for receipt by the CPU 200. Digital audio signals destined for a particular multimedia controller and listener are encapsulated by the digital input device controller 210 in accordance with the protocol of the bus 212. The packet is routed to the destination multimedia controller and decapsulated to recover the digital audio signals. The digital audio signals are then coupled to the DAC which provides analog audio signals to the listener's headphones. Accordingly, the embodiment illustrated in Figure 4 permits very high levels of performance and flexibihty. Individuals and groups of listeners may selectively receive communications between one another depending upon the apphcation. For example, in the context of an educational application wherein different student groups share access to the same application, those group members may receive and transmit secure communications between one another through the use of headphones (for example, headphone 248) and the microphones (for example, microphone 230) coupled to the multimedia controllers. In addition, as digital systems become more sophisticated, it is contemplated that each ofthe listeners may communicate through voice recognition software directly with the CPU 200, and thereby achieve a level of dialog and interaction with the application program.

It will be appreciated that the embodiments illustrated herein may be networked through local area networks (LAN), wide area networks (WAN), telephone, optical fiber, two-way cable, and/or satellite links with other multimedia listeners and/or computers. It is contemplated that the networking of systems employing the teachings ofthe present invention will permit independent and separate audio experiences to each of the listeners receiving sound information through the network.

Further, the present invention is applicable to any architecture wherein a single executable program directs the listening experience for two or more listeners. The single executable program may be run across multiple processors, such as where a program operating on a first processor invokes a remote procedure call to a second processor. The program may also be resident on a server, such as a world wide web server, that provides multimedia content to a client. Even though the client may be running an executable program to interpret the instructions of the server program, the server program, which corresponds to the audio program ofthe present invention, directs the listening experience.

It is also possible for the server program to merely download data or an apphcation program. The client may then inteφret the downloaded content (eg. display any visual portions and convert the audio data to sound) or run the application program according to the teachings of the present invention.

As described in this Specification, the present invention may be implemented in one of a variety of embodiments depending upon the level of sophistication and technical performance required. Although the present invention has been described with reference to a few exemplary Figures 1 through 5, it will be apparent that many alternatives, modifications and variations may be made in light of the foregoing description.

Claims

What is claimed is: I . A computer game for providing separate and independent audio to a first and a second user, comprising: storage means for storing sound data; processor means operating under the control of a program and coupled to said storage means, for selecting and retrieving first sound data for said first user and second sound data for said second user; sound synthesizer means coupled to receive said first and second sound data and converting said sound data into first and second digital audio signals, respectively; digital to audio converter (DAC) means for receiving said first and second digital audio signals from said sound synthesizer means and converting said first and second digital audio signals into corresponding first and second analog audio signals; routing means coupled to receive said first and second analog audio signals and selectively providing said first analog audio signals to said first user and said second analog audio signals to said second user; a listening device coupled to said routing means and provided to said first and second users to convert said first and second analog audio signals to sound waves such that said first user hears said first analog audio signals and said second user hears said second analog audio signals.

2. The apparatus of claim 1 wherein said DAC means comprises a single digital to audio converter having a stereo output comprising first and second output channels, said first analog audio signals provided to said routing means on said first output channel and said second analog audio signals provided to said routing means on said second output channel.

3. The apparatus of claim 1 wherein said routing means comprises a multiplexer, said multiplexer controlled by said processor means to selectively couple said first analog audio signals to said listening device of said first user and said second analog audio signals to said listening device of said second user.

4. The apparatus of claim 1 wherein said DAC means comprises a first digital to audio converter coupled to receive said first digital audio signals and a second digital to audio converter coupled to receive said second digital audio signals from said sound synthesizer means.

5. The apparatus of claim 4 wherein said first and second digital to audio converters are selectively enabled by said processor means, such that said first digital to audio converter is enabled and said second digital to audio converter is disabled when said sound synthesizer means outputs said first digital audio signals, and said first digital to audio converter is disabled and said second digital to audio converter is enabled when said sound synthesizer means outputs said second digital audio signals.

6. A computer game for providing separate and independent audio to a first and a second user, comprising: storage means for storing sound data; processor means operating under the control of a program and coupled to said storage means, for selecting and retrieving first sound data for said first user and second sound data for said second user; sound synthesizer means coupled to receive said first and said second sound data and converting said sound data into first and second digital audio signals, respectively; controller means coupled to said sound synthesizer to receive said first and second digital audio signals, said controller means further coupled to a bus to provide said first digital signal to a first user and said second digital sound to a second user; a first and a second receiving device coupled to said bus for receiving said first and second digital audio signals, respectively; conversion means coupled to each of said receiving devices for converting said received first and second digital audio signals into corresponding first and second analog audio signals; a listening device coupled to each of said conversion means for converting said first and second analog audio signals to sound waves such that said first user hears said first analog audio signals and said second user hears said second analog audio signals.

7. The apparatus of claim 6 wherein said first and second receiving devices comprise game controllers and said conversion means includes digital to audio converter (DAC) means for converting said first and second digital audio signals into said corresponding first and second analog audio signals.

8. The apparatus of claim 7 wherein each of said game controllers includes input signal means for receiving input signals from a said user of said game controller, said input signals including real time sound wherein one said real time sound comprises speech.

9. The apparatus of claim 8 wherein said bus comprises a serial bus and each of said game controllers includes a serial bus interface circuit for receiving said first and second digital audio signals over said bus and for communicating said input signals from said user of said game controller to said CPU.

10. The apparatus of claim 9 wherein said audio inputs are coupled to said CPU to be selectively coupled back to said game controllers to provide real time player generated sounds to said users.

11. A method for providing separate and independent audio to first and second listeners, comprising the steps of: providing, substantially under the control of a single executable computer program, first sound data for a first listener; providing, substantially under the control of said program, second sound data for a second listener, said second sound data being different from said first sound data at some time during the execution of said program; receiving said first sound data by said first listener; receiving said second sound data by said second listener; such that said first and second listeners may be provided with distinct sound data.

12. The method of claim 11 further comprising the step of providing at least one of said first or second sound data with an indicator that correlates said first or second sound data with a corresponding one of said first or second listeners.

13. The method of claim 12 further comprising the step of providing said first and second sound data in the form of packets and wherein said indicator comprises a packet address.

14. The method of Claim 12 wherein said indicator comprises an "enable" signal that enables a digital to analog converter associated with one of said first and second listeners respectively.

15 ethod of claim 11 wherein at least one of said first and second listener . , es a corresponding one of said first or second data in analog form.

161 The method of claim 11 wherein said single executable program operates on a first digital device.

17. The method of claim 16 further comprising the step of providing at least one of said first or second data from said first digital device to a second digital device and wherein a corresponding one of said first or second listeners receive said first or second data respectively from said second digital device.

18. The method of claim 17 wherein said first digital device comprises a server computer and said second digital device comprises a client computer.

19. The method of claim 11 wherein said steps of receiving said first and second data respectively comprise the steps of receiving said first and second data with first and second multimedia controllers.

20. The method of claim 11 wherein said single executable program comprises a game program.

21. The method of claim 11 wherein said single executable program comprises a computer program controlling an audio program which includes multiple, distinct audio programs suitable for separate, simultaneous listeners.

22. The method of claim 11 wherein said receiving steps comprise receiving said first or second data respectively in different locations.

23. The method of claim 11 wherein said at least one of first and second sound data is stored on a mass storage device before being provided to a corresponding one of said listeners.

24. The method of claim 11 further comprising the step of providing said first and second listeners with visual data associated with said first and second sound data.

25. The method of claim 11 wherein at least one of said steps of providing said first or second sound data comprises the step of routing said first or second sound data to a corresponding one of said first or second users.

26. An apparatus for providing separate and independent audio to a first and a second listener, comprising: a first digital device including: first providing means for providing, substantially under the control of a single executable program, first sound data to a first listener; second providing means for providing, substantially under the control of said program, second sound data to a second listener, said second sound data being different from said first sound data at some time during the execution of said program; first receiving means for receiving said first sound data by said first listener; second receiving means for receiving said second sound data by said second listener; such that said first and second listeners may be provided with distinct sound data.

27. The apparatus of claim 26 wherein at least one of said first or second providing means comprises means for providing said first or second sound data with an indicator that correlates said first or second sound data with a corresponding one of said first or second listeners.

28. The apparatus of claim 26 wherein at least one of said first and second listeners receives a corresponding one of said first or second data in analog form.

29. The apparatus of claim 26 further comprising a first digital device and wherein said single executable program operates on said first digital device.

30. The apparatus of claim 29 further comprising a second digital device and wherein: said first or second providing means includes means for providing at least one of said first or second data from said first digital device to said second digital device; and said second digital device includes means for providing said first or second data from said second digital device to a corresponding one of said first or second listeners.

31. The apparatus of claim 30 wherein said first digital device comprises a server computer and said second digital device comprises a client computer.

32. The apparatus of claim 26 wherein said first or second receiving means comprises a multimedia controller.

33. The apparatus of claim 26 wherein said single executable program comprises a computer program controlling an audio program which includes multiple, distinct audio programs suitable for separate, simultaneous listeners.

34. The apparatus of claim 26 wherein said first receiving means comprises a speaker disposed in a first location and said second receiving means comprises a speaker disposed in a second location.

35. The apparatus of claim 26 further comprising a mass storage device for storing at least one of said first or second sound data before said first or second sound data is provided to a corresponding one of said first or second listeners.

36. The apparatus of claim 26 further comprising means for providing said first and second listeners with visual data associated with said first and second sound data.

37. The apparatus of claim 26 wherein at least one of said first or second providing means comprises means for routing said first or second sound data to a corresponding one of said first or second users.

38. A method for providing separate and independent audio to a first and second user, comprising the steps of: storing sound data; selecting and retrieving first sound data for said first user and second sound data for said second user; receiving said first and second sound data and converting said sound data into first and second digital audio signals, respectively; receiving said first and second digital audio signals and converting said first and second digital audio signals into corresponding first and second analog audio signals; receiving said first and second analog audio signals and selectively providing said first analog audio signals to said first user and said second analog audio signals to said second user; converting said first and second analog audio signals to sound waves such that said first user hears said first analog audio signals and said second user hears said second analog audio signals.

39. A method for providing separate and independent audio to a first and a second user, comprising the steps of: storing sound data; selecting and retrieving first sound data for said first user and second sound data for said second user; converting said first and second sound data into first and second digital audio signals, respectively; receiving said first and second digital audio signals, respectively over a bus; converting said received first and second digital audio signals into corresponding first and second analog audio signals; converting said first and second analog audio signals to sound waves such that said first user hears said first analog audio signals and said second user hears said second analog audio signals.