GB2282031A - Video and audio information signal processing - Google Patents

Abstract

An information signal processing apparatus processes a digital video signal DVS and a digital audio signal DAS by using memories 10, 24. The video signal is directly written into one memory 10 and read out from the memory at a designated rate. The audio signal is clock frequency converted into field units corresponding to fields of the video signal, then written into another memory 24. The stored audio data is read out from the other memory 24 at the designated rate, and clock frequency re-converted from the field unit format. The resulting video and audio signals can then be supplied to external monitoring means. <IMAGE>

Description

INFORMATION SIGNAL PROCESSING APPARATUS AND METHODS This invention relates to information signal processing apparatus and methods, the information signal including video data and audio data.

There is known a video random access memory (RAM) recorder which, amongst other functions, controls the replay rate of a video signal according to a user settable input, such as the position of a control lever. The video RAM recorder is primarily used in the sports and outside broadcast market providing instant replay and slow motion facilities.

Due to the distinct advantages of the RAM recorder over standard video tape recorder (VTR) technology, namely flexibility in simultaneous record and playback and near instantaneous random access capabilities (that is, no time is required for rewind cueing or prerolling), it is very suitable for many other applications, for example in post production, where edit intensive work such as multi-layering can be performed far more swiftly using a RAM recorder than with a traditional VTR. A talk-show delay is another very suitable application for RAM technology. Here instant cueing can be utilized for live on-line censoring of unsuitable material.

In both these applications, audio plays an important part.

However, due to the differences in the properties of digital audio and video signals, RAM recorders have generally been designed specifically to handle video only or to handle audio only. Therefore using such known RAM recorders, it is not readily apparent how to handle video signals and audio signals in synchronization.

According to one aspect of the invention there is provided apparatus for processing an information signal including video data and audio data, the apparatus comprising: means for converting the audio data into field units corresponding to fields of the video data; first memory means for memorizing the video data in field units; second memory means for memorizing the converted audio data in field units; control means for controlling the write timing and the read out timing of the audio data into/from the second memory means with respect to the write timing and the read out timing of the video data into/from the first memory means; and means for re-converting the audio data from the field unit format.

According to another aspect of the invention there is provided a method of processing an information signal including video data and audio data, the method comprising: converting the audio data into field units corresponding to fields of the video data; memorizing the video data in field units in a first memory means; memorizing the audio data in field units in a second memory means; controlling the write timing and the read out timing of the audio data into/from the second memory means with respect to the write timing and the read out timing of the video data into/from the first memory means; and re-converting the audio data from the field unit format.

The invention will now be described by way of example with reference to the accompanying drawings, throughout which like parts are referred to by like references, and in which: Figure 1 is a block diagram of video and audio signal processing apparatus according to one embodiment of the invention; Figure 2 is a schematic diagram of one type of digital audio signal to be handled by the apparatus shown in Figure 1; Figure 3 is a schematic diagram of the line relationship between video data and audio data during memory writing; Figure 4 is a schematic diagram of the field relationship between the video data and audio data during memory writing; Figure 5 is a schematic diagram of the line relationship between the video data and audio data during memory read out; Figure 6 is a schematic diagram of the field relationship between the video data and audio data during memory read out; and Figure 7 is a schematic diagram of three possible sequences of read out data from a memory board.

Referring initially to Figure 1, there is shown a video and audio RAM recorder apparatus according to an embodiment of the invention. An incoming digital video signal DVS such as an NTSC digital video signal, a PAL digital video signal or component digital video signal is supplied via a video input board 12 (including a timing circuit) to an input port (c) of a memory board 10. A video field sync signal VFS and a video sample rate clock signal VSRC are supplied to the timing circuit of the video input board 12. An active video memory write signal AVMW from the timing circuit of the video input board 12 is supplied to an input port (d) of the memory board 10. An output video signal from the memory board 10 is supplied to a video output board 14 (including a timing circuit).An active video memory read signal AVMR from the memory board 10 is supplied to the timing circuit of the video output board 14, as well as VFS and VSRC signals. An output video signal from the video output board 14 is supplied to a monitor (not shown).

An incoming biphase digital audio signal DAS is supplied to a receiver 16. The digital audio signal DAS is preferably in the form of an AES/EBU format signal as shown in Figure 2 and as described in "Serial Transmission Format for Two-channel Linearly Represented Digital Audio Data", AES3-1985 or ANSI S4-40-1985. An output signal from the receiver 16 is supplied to a serial-to-parallel converter 18.

An output signal from the serial-to-parallel converter 18 is supplied to an eight wide three-to-one multiplexer 20. An output signal from the eight wide three-to-one multiplexer 20 is supplied to an audio field delay first-in-first-out (FIFO) store 22. An output signal from the audio field delay FIFO 22 is supplied to another memory board 24.

An output signal from the memory board 24 is supplied to a swing buffer memory 26. An output signal from the swing buffer memory 26 is supplied to a eight way four-to-one demultiplexer 28. An output signal from the eight way four-to-one demultiplexer 28 is supplied to a gain profile multiplier 30. An output signal from the gain profile multiplier 30 is supplied to a parallel-to-serial converter 32. An output signal from the parallel-to-serial converter 32 is supplied to a transmitter 34. An output digital audio signal from the transmitter 34 is supplied to the monitor (not shown).

The video field sync signal VFS and the video sample rate clock signal VSRC are also supplied to a timing signal generator 36 which provides timing signals at audio rate (AR) and video rate (VR). Audio rate timing signals from the timing signal generator 36 are supplied to the receiver 16, the serial-to-parallel converter 18 and the eight wide three-to-one multiplexer 20. A write signal (at audio' rate) and a read signal (at video rate) from the timing signal generator 36 are supplied to the audio field delay FIFO 22. An active audio memory write signal AAMW from the timing signal generator 36 is supplied to the memory board 24. An active audio memory read signal AAMR from the memory board 24 is suppLied to a write address generator 56. A write address signal from the write address generator 56 is supplied to the swing buffer memory 26.Audio rate (AR) timing signals from the timing signal generator 36 are also supplied to the read address generator 60, to the eight way four-to-one demultiplexer 28, the parallel-to-serial converter 32 and the transmitter 34. Video rate (VR) timing signals from the timing signal generator 36 are supplied to the gain profile multiplier 30, the write address generator 56 and a read address generator 60. A read address signal and a switching signal from the read address generator 60 are supplied to the swing buffer memory 26.

An instruction signal from an input control means 62 is supplied to a CPU 64 which also receives a mode signal from the timing generator 36.

The input control means 62 may, for example, include a control lever 66 for controlling the rate and/or direction of read out. Control signals including addresses from the CPU 64 are supplied to the memory board 10 and the memory board 24. A mode signal is also supplied by the CPU 64 to the write address generator 56.

The swing buffer memory 26 has a first switch means 38, a second switch means 40, a field 0 RAM memory 42 and a field 1 RAM memory 44.

The (audio) output signal from the memory board 24 is supplied to the first switch means 38. The switched signal from the first switch means 38 is selectively supplied to the field 0 RAM memory 42 and the field 1 RAM memory 44. Output signals from the field 0 RAM memory 42 and the field 1 RAM memory 44 are supplied to the second switch means 40. The output switched signal from the second switch means 40 is supplied to the eight way four-to-one demultiplexer 28.

The active video memory write signal AVMW generated by the timing circuit of the video input board 12 according to the video field sync signal VFS and video sample rate clock signal VSRC is supplied to the memory board 10 as a write timing signal. The incoming digital video signal DVS is stored in the memory board 10 under control of the CPU 64. The stored video signal is read out from the memory board 10 at a controlled video field rate under control of the CPU 64. The read out video field rate is set by the operator by using the control lever 66 of the input control means 62. Then digital video data read out from the memory board 10 is supplied to an external video monitoring apparatus via the video output board 14.

In the specific embodiment, the memory board 10 has two 8-bit wide input data ports and two 8-bit wide output data ports (not shown), for example as provided in the Sony DEM-1000 video RAM recorder. The CPU 64 allows an independent start address to be provided relating to the data at each of these four ports. For video, these start addresses are provided during field blanking. At the start of a field, data is applied to the input ports. The active video memory write signal AVMW defines a period of active input data. Commencing at the two write start addresses, the input digital video data is sequentially written into the memory board 10. Meanwhile data is read from the memory board 10 sequentially commencing at the two read start addresses and directed to the output ports.An active video memory read signal AVMR is provided by the memory board 10 and indicates when the output port is providing active data. The AVMR signal is supplied to the video output board 14 as a valid data timing signal. The memory board 10 utilises dynamic RAM (DRAM) technology which requires time for refreshing each memory cell for data retention. During this time, the memory board 10 cannot be written to or read from. In the video signal, the field and line blanking intervals provide ample time for this when the active video memory write signal AVMW is inactive.

The incoming biphase digital audio signal DAS is supplied to the receiver 16. The audio signal is decoded into a serial data stream containing audio sample data along with various channel interface bits.

Under the control of the timing signal generator 36, this data has been locked to the input digital video signal such that a known number of audio samples may be associated with each input digital video field.

The format of the receiver board level interface may be similar to the format of the commonly used I2S bus, for example. However, it should contain at least each audio sample with the corresponding Validity, Channel status, User, Parity and Z preamble bits, and be arranged in a consistent "left right left right" multiplex in the normal manner.

The output of the receiver 16 is supplied to the serial-toparallel converter 18. In the serial-to-parallel converter 18, the output signal of the receiver 16 having a data rate of 32x2xfs (fs being the sampling clock frequency) is converted to 24-bit parallel data having a data rate of 2xfs under the control of the timing signal generator 36. In the serial-to-parallel converter 18, all of the audio data (20 bits), one User bit, one Validity bit, one Channel status bit and one Z preamble, or channel status synchronization bit, are selected from the output data of the receiver 16.

The output signal from the serial-to-parallel converter 18 is supplied to the eight way three-to-one multiplexer 20, in which 24-bit parallel data is time division multiplexed into three 8-bit words having a clock frequency of 3x2xfs. The output signal of the eight wide three-to-one multiplexer 20 is then supplied to the audio field delay FIFO store 22 and memorized therein at 3x2xfs clock rate according to the FIFO write clock from the timing signal generator 36.

Until this point, the data has been processed at a rate equivalent to an integer multiple of the audio sample rate fs.

The data memorized in the audio field delay FIFO store 22 is read out at the clock rate of the digital video signal according to the FIFO read clock from the timing signal generator 36. That is, the audio field delay FIFO store 22 is used to convert the word rate to that of the video sample rate and format it in the time domain to conform to the restrictions of the memory board 24. For example, as the architecture of the memory board 24 is identical to that of the memory board 10, it follows that, as previously mentioned, one restriction is that significant time is required for dynamic memory refresh, during which data cannot be written or read. This refresh must occur at regular intervals and thus audio data is presented to the memory board 24 in line units, see Figure 3.

The output signal read from the FIFO store 22 is then supplied to the memory board 24. The CPU 64 controls the write addressing of the memory board 24 and controls the read out addressing and thus the read out field order of the memory board 24 according to the instruction signal from the input control means 62 provided with the control lever 66 or keyboard (not shown).

Once per field, the CPU 64 downloads the start addresses for the audio data. This is followed by the audio data being presented, under the control of the timing signal generator 36, to the memory board 24 in a format similar to lines of the video data. The predetermined number of audio words are downloaded in this manner with an associated active audio memory write strobe AAMW equivalent to the active video memory write strobe AVMW used for the digital video signal DVS. Due to the continuous manner in which the audio words continue to be written into the audio field delay FIFO store 22 it follows that the capacity of the store must reflect the predetermined number of samples of audio associated with the field of video plus a buffer margin determined by the time taken to download the audio from the previous field into the memory board 24.

As the number of audio data words per field is far less than that for video, it is possible to store more than one stereo channel of audio per memory board channel, with each field of video signal. For example, two channels can be accommodated by duplicating the write side data path of Figure 1, including providing two FIFO stores. The first start address is loaded and data from the first FIFO is downloaded in the normal manner. During this period, the read clock to the second FIFO remains inactive. After the data from the first FIFO has been downloaded, a second start address is supplied by the CPU 64 and data from the second FIFO is then multiplexed, for example via tristate bussing into the same memory board input port. This time the read clock of the first FIFO remains inactive. The FIFO capacities must also be increased to accommodate the additional buffering of data while not just the first FIFO empties but also the second, that is, until the last audio sample associated with that field of video has been written to the memory board 24.

In all operations, the CPU 64 keeps track of where particular fields of audio data are placed within the memory, and in certain instances of the number of samples stored for that field, in dependence on the mode signal. For example, when locking 48kHz sampled audio data to 625/50 video data, an exact number of audio samples can be calculated for each field but for 525/59.94 video the audio sample number varies by one sample every fifth field.

The audio data memorized in the memory board 24 and the video data memorized in the memory board 10 may be read out from the memory boards 24 and 10 at field rates greater than, equal to, or less than unity in both forward and reverse directions while at all times maintaining a recognizable audio output with a usable channel status bit (that is, a channel status which contains regular valid blocks which relate to, with reasonable accuracy, the audio samples with which they are re-transmitted). For example, it is possible to maintain any changes in emphasis (which may, for example, be caused by careless editing of two different sources) indicated on the received audio samples to a region close to the sample where the original change in emphasis occurred when re-transmitting the retrieved audio. In normal unity speed playback, the channel status is transparently retrieved and replayed.The read out field timing and rate are controlled by the CPU 64 according to the instruction signal from the input control means 62.

Audio data read out from the memory board 24 is then supplied to the swing buffer 26, and in particular to the field 0 RAM memory 42 and the field 1 RAM memory 44 alternately through the first switch means 38 controlled by the read address generator 60. Then the data is memorized in the field 0 RAM memory 42 and the field 1 RAM memory 44 alternately at the rate of the digital video signal according to the write address signal from the write address generator 56 on a field-byfield basis. Data in the field 0 RAM memory 42 and data in the field 1 RAM memory 44 are field-by-field alternately read out from the field O RAM memory 42 and the field 1 RAM memory 44 at a rate of 4x2xfs according to the read address signal from the read address generator 60.Data read out from the field 0 RAM memory 42 and the field 1 RAM memory 44 are supplied to the second switch means 40 which alternately supplies the data to the eight way four-to-one demultiplexer 28.

The active audio memory read strobe signal AAMR provided by the memory board 24 indicates the active data timing on the output ports of the memory board. As it happens, this is essentially the AAMW signal delayed by several clock cycles. While the AAMR signal is active, data is read from the memory board 24 into the swing buffer 26 as described above. During every odd field, for example, the field 0 RAM 42 is being written to while the field 1 RAM 44 is being read from. Then, on every even field the opposite situation occurs, that is the field 0 RAM 42 is read from and the field 1 RAM 44 is written to. The control of this swing operation is one of the functions of the read address generator 60.

The write address generator 56 is reset to zero at the beginning of every field by a write address signal from the timing generator 36.

Following resetting whenever the AAMR signal is active, the write address is incremented by one and thus the data supplied by the memory board 24 sequentially fills one of the field RAMs.

At the end of each field, the read address generator 60 selects the relevant field RAM 42/44 to next read from, via the RAM control signals and the switch means 40. The data is then read from the field RAM 42/44 under the control of the read address generator 60. The read address pattern varies depending on two main factors, namely the number of samples in the present audio field and the direction of playback. For forward playback, the data is simply read out sequentially until the correct number of words for that field have been read at which point the swing control toggles and the next field begins.For reverse playback, it is necessary for the audio sample data and the corresponding Validity bit to be read out starting with the last 'left' sample followed by the last 'right' sample, followed by the second last 'left' sample and so on until the correct number of samples have been replayed. The channel status Z preamble and the User bits will, however, normally all have to be read out as they would in forward playback mode, otherwise they will be interpreted incorrectly further downstream in the audio path, for example in a mixer, DVTR or similar.

To minimize the number of read cycles per audio sample when formatting the audio data on the write side of the memory board 24, it is preferable to store the channel status, Z preamble and User bit together in the same byte, thus standardizing to four read cycles per sample from the swing buffer. For example, with these three bits stored in the third of the three bytes for each sample, for forward playback the read address pattern generated by the read address generator 60 might be: 0,1,2,2, 3,4,5,5, 6,7,8,8, 9,10,11,11 ........

and for reverse, might be: 5754,5755,5756,2, 5757,5758,5759,5, 5748,5749,5750,8, 5751,5752,5753,11,11 The result of this reverse pattern is that although the channel status is not exactly what it should be, with minimal hardware overhead it is maintained with semi-usable accuracy.

The data thus retrieved from the field RAMs 42/44 is applied as a continuous stream to the eight way four-to-one demultiplexer 28 where the four separate bytes of the data are re-assembled into the 24-bit data. In the eight way four-to-one demultiplexer, four 8-bit words time division multiplexed from the switch means 40 are demultiplexed to 24-bit parallel data having a clock frequency of 2xfs. Then the output signal of the eight way four-to-one demultiplexer 28 is supplied to the gain profile multiplier 30.

The multiplier 30 can be used to smooth a non-continuous field boundary portion of the data from the eight way four-to-one demultiplexer 28. Then the data from the multiplier 30 is supplied to the parallel-to-serial converter 32. In the parallel-to-serial converter 32, the 24-bit parallel data is converted to digital serial data. Then the digital serial data is transmitted to external audio equipment by the transmitter 34 after re-encoding the channel for output. The read address generator 60, the eight way four-to-one demultiplexer 28, the gain profile multiplier 30, the parallel-toserial converter 32 and the transmitter 34 are operated according to timing signals generated by the timing signal generator 36 according to the video field synchronization signal VFS and the video sample rate clock VSRC. The timing signal generator 36 and the CPU 64 regulate the read side processing.

Operation of the apparatus shown in Figure 1 will now be described with reference to Figures 3 to 7.

Figure 3 shows the line relationship between an active video line and active audio data in a typical component digital system. As shown in Figure 3, 768 samples of the active audio data and 768 samples of the active video line data are memorized in the memory boards 24 and 10 respectively.

Figure 4 shows the field relationship between an active video field and active audio data. 768x240 samples of the active video data may typically be memorized in the memory board 10 as a field in accordance with the AVMW strobe. 768xL (L depending on the sampling frequency fs of the audio signal) samples of the audio data may then be memorized in the memory board 24 during one video field in accordance with the AAMW strobe.

The active video strobe signal AVMW shown in Figure 4 is generated by the timing circuit of the video input board 12 as derived from the video field synchronization signal VFS and the video sample rate clock signal VSRC, and is supplied to the memory board 10 as a timing signal for writing. The active audio strobe signal AAMW shown in Figure 4 is generated by the timing generator 36 as derived from the video field synchronization signal VFS and the video sample rate clock signal VSRC and is supplied to the memory board 24 as a timing signal for writing audio data.

As mentioned above, in the writing side of the apparatus, digital video signals are memorized in the memory board 10 directly at the video clock rate. The clock frequency of the digital audio data is up converted in order that it may be the same as the clock frequency of the digital video signal before memorizing. The resulting digital audio signals are then memorized in the memory board 24 at the video clock rate.

Figure 5 shows the line relationship between an active video line and active audio data for read out. As shown in Figure 5, 768 samples are read out from the memory board 10 as video lines. 768 samples of the audio data are read out from the memory board 24 during one video line.

Figure 6 shows the field relationship between an active video field and digital audio data read from the memory board 24.

The active video strobe signal AVMR from the memory board 10 is supplied to the video output board 14 as a timing signal, and is used to format the digital video signal in the output board 14.

The active audio strobe signal AAMR from the memory board 24 is supplied to the write address generator 56. The write address generator 56 generates a write address signal according to the active audio strobe signal AAMR . Then data read from the memory board 24 is memorized in the field.0 RAM memory 42 or the field 1 RAM memory 44 according to the write address signal and the position of the switch 38.

As mentioned above, in the reading side of the apparatus, digital video signals memorized in the memory board 10 are read out from the memory board 10 at the video clock rate, and supplied to the external monitoring apparatus via the video output board 14. On the other hand, the word rate of the digital audio data read out from the memory board 24 is reduced to an appropriate digital audio data rate. The resulting digital audio data is then supplied to external audio apparatus.

Control of which fields of audio data are retrieved from the memory board 24 is performed via the CPU 64. That is, the read start addresses are supplied from the CPU 64. Normally the retrieved audio data would be that which relates directly to the video field with which it was recorded. Alternatively, for example, audio tracks can be slipped by supplying different start addresses. For non-unity playback speed, the audio data will normally only be continuous on a field basis, for example, one field of audio data in every two would normally be retrieved at double speed playback. In the double speed playback mode, referring to the Figure 7, the CPU 64 controls the reading out operation of the memory board 24 in order that one field of audio data in every two would normally be retrieved.In a half-speed playback mode, again as shown in Figure 7, the CPU 64 controls the reading out operation of the memory board 24 in order that each field of audio data would be consecutively retrieved twice. This type of processing would generally degrade reproduction of the audio data since discontinuities would occur at field boundaries. The gain profile multiplier 30, under the control of the timing generator 58 and in response to information from the CPU 64, is therefore used on the data path to fade the reproduced audio signal around the field boundaries where discontinuities occur. Incidentally, this multiplier 30 can also be used, under separate control, for crude fading of the audio when editing or for muting unwanted audio channels. The input control means 62 supplies a reproducing speed and direction instruction signal to the CPU 64 according to the position of the lever 66.

The read side circuit for retrieving two or more channels of audio from the memory board 24 per field simply involves duplication of that part of the circuit shown in Figure 1 which comprises the swing buffer memory 26, the eight way four-to-one demultiplexer 28, the gain profile multiplier 30, the parallel-to-serial converter 32, the transmitter 34, the write address generator 56 and the read address generator 60. The timing signal generator 36 may serve this duplicate circuity in a similar manner to that described above. It should be noted, however, that additional control must be included to reset the duplicate write address generator at the appropriate later time, for example when the CPU 64 is providing the second set of start addresses to the memory board 24.

According to the apparatus as described above, it is possible to handle video and audio signals in tandem. As the construction of the memory board 24 can be the same as the construction of the memory board 10, it is possible to use a single type of memory board as both video and audio memory boards.

Claims (11)

1. Apparatus for processing an information signal including video data and audio data, the apparatus comprising: means for converting the audio data into field units corresponding to fields of the video data; first memory means for memorizing the video data in field units; second memory means for memorizing the converted audio data in field units; control means for controlling the write timing and the read out timing of the audio data into/from the second memory means with respect to the write timing and the read out timing of the video data into/from the first memory means; and means for re-converting the audio data from the field unit format.

2. Apparatus as claimed in claim 1, wherein said audio data converting means comprises first clock frequency converting means for converting the clock frequency of the audio data in the information signal to a second clock frequency being the clock frequency of the video data, the clock frequency converted audio data being memorized in the second memory means at the second clock frequency, and wherein said audio data re-converting means comprises second clock frequency converting means for converting the clock frequency of the audio data read out from the second memory means at the read out video clock rate to the first-mentioned clock frequency.

3. Apparatus as claimed in claim 2, wherein said first clock frequency converting means comprises a serial to parallel converter for converting serial audio data to parallel audio data, and FIFO memory means for memorizing the parallel audio data and for outputting the parallel audio data at video clock rate.

4. Apparatus as claimed in claim 2 or claim 3, wherein said second clock frequency converting means comprises RAM memory means for memorizing the audio data read out from the second memory means at the video clock rate and outputting the parallel audio data at the audio clock rate.

5. Apparatus as claimed in claim 4, wherein said RAM memory means comprises a first store and a second store, field units of the audio data being alternately written into the first and second stores and alternately read out from the first and second stores such that, during audio data writing of one field unit into one of the stores, stored audio data of another field unit is read out of the other of the stores.

6. Apparatus as claimed in any one of the preceding claims, comprising an input means for supplying a control signal to the control means thereby controlling the read out timing of the first and second memory means.

7. Apparatus as claimed in any one of the preceding claims, wherein the control means is operable to control the read out rate of the first and second memory means thereby to control the rate of the video and audio data read out from the first and second memory means.

8. Apparatus as claimed in any one of the preceding claims1 wherein the control means is operable to reverse the order of the video and audio data in field units read out from the first and second memory means.

9. Apparatus for processing an information signal including video data and audio data, the apparatus being substantially as hereinbefore described with reference to the accompanying drawings.

10. A method of processing an information signal including video data and audio data, the method comprising: converting the audio data into field units corresponding to fields of the video data; memorizing the video data in field units in a first memory means; memorizing the audio data in field units in a second memory means; controlling the write timing and the read out timing of the audio data into/from the second memory means with respect to the write timing and the read out timing of the video data into/from the first memory means; and re-converting the audio data from the field unit format.

11. A method of processing an information signal including video data and audio data, the method being substantially as hereinbefore described with reference to the accompanying drawings.