EP2206113B1 - Vorrichtung und verfahren zum erzeugen eines multikanalsignals mit einer sprachsignalverarbeitung - Google Patents
Vorrichtung und verfahren zum erzeugen eines multikanalsignals mit einer sprachsignalverarbeitung Download PDFInfo
- Publication number
- EP2206113B1 EP2206113B1 EP08802737A EP08802737A EP2206113B1 EP 2206113 B1 EP2206113 B1 EP 2206113B1 EP 08802737 A EP08802737 A EP 08802737A EP 08802737 A EP08802737 A EP 08802737A EP 2206113 B1 EP2206113 B1 EP 2206113B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- signal
- channel
- speech
- implemented
- ambience
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 51
- 238000012545 processing Methods 0.000 title description 9
- 239000003607 modifier Substances 0.000 claims abstract description 31
- 230000002238 attenuated effect Effects 0.000 claims abstract description 18
- 230000003595 spectral effect Effects 0.000 claims description 17
- 238000004458 analytical method Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000002123 temporal effect Effects 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 5
- 238000007619 statistical method Methods 0.000 claims 1
- 238000001514 detection method Methods 0.000 abstract description 13
- 230000005236 sound signal Effects 0.000 description 27
- 230000001629 suppression Effects 0.000 description 23
- 230000007613 environmental effect Effects 0.000 description 17
- 239000000203 mixture Substances 0.000 description 11
- 230000008569 process Effects 0.000 description 10
- 238000000605 extraction Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 8
- 238000012986 modification Methods 0.000 description 7
- 230000004048 modification Effects 0.000 description 7
- 230000009467 reduction Effects 0.000 description 7
- 238000001228 spectrum Methods 0.000 description 5
- 238000003786 synthesis reaction Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000000354 decomposition reaction Methods 0.000 description 4
- 238000000926 separation method Methods 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 230000003321 amplification Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 238000003909 pattern recognition Methods 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 230000001755 vocal effect Effects 0.000 description 3
- 241001136792 Alle Species 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 2
- 235000009508 confectionery Nutrition 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000005284 excitation Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000001052 transient effect Effects 0.000 description 2
- 238000012952 Resampling Methods 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 239000006227 byproduct Substances 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013016 damping Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 238000009527 percussion Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 108010066082 tartrate-sensitive acid phosphatase Proteins 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
- H04S5/005—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation of the pseudo five- or more-channel type, e.g. virtual surround
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
- G10L21/0364—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude for improving intelligibility
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/78—Detection of presence or absence of voice signals
Definitions
- the present invention relates to the field of audio signal processing, and more particularly to the generation of multiple output channels from fewer input channels, such as audio channels.
- B one (mono) channel or two (stereo) input channels.
- a corresponding device is for example from the EP 1 021 063 out.
- Multi-channel audio is becoming more and more popular.
- Such playback systems generally consist of three speakers L (left), C (center) and R (right), which are typically located in front of the user, and two speakers Ls and Rs located behind the user, and typically one of them LFE channel, which is also called low-frequency effect channel or subwoofer.
- LFE channel which is also called low-frequency effect channel or subwoofer.
- Such a channel scenario is in Fig. 5b and in Fig. 5c indicated. While the positioning of the loudspeakers L, C, R, Ls, Rs should be made with respect to the user as shown in FIGS.
- the positioning of the LFE channel (in FIG Figs. 5b and 5c not shown) is not as critical as the ear can not locate at such low frequencies and thus the LFE channel can be located anywhere where it does not bother due to its considerable size.
- Such a multi-channel system provides several advantages over a typical stereo reproduction, which is a two-channel reproduction, such as in Fig. 5a is shown.
- Even outside of the optimal central listening position results in improved stability of the front listening experience, which is also referred to as a "front image”, due to the center channel. This results in a larger “sweet spot”, where "sweet spot” stands for the optimal listening position.
- the listener has a better feeling of "immersing" in the audio scene due to the two rear speakers Ls and Rs.
- the first option is to play the left and right channels through the left and right speakers of the multi-channel playback system.
- a disadvantage of this solution is that you do not exploit the variety of existing speakers, so that you do not take advantage of the presence of the center speaker and the two rear speakers advantageous.
- Another option is to convert the two channels into a multi-channel signal. This can be done during playback or by a special preprocessing, which advantageously exploits all six speakers of the existing example 5.1 playback system and thus leads to an improved listening experience when the upmixing or the "upmix" of two channels to 5 or 6 channels error-free is carried out.
- the direct sound sources are reproduced by the three front channels so that they are perceived by the user at the same position as in the original two-channel version.
- the original two-channel version is in Fig. 5a shown schematically, using the example of various drum instruments.
- Fig. 5b shows a highly mixed version of the concept, in which all the original sound sources, so the drum instruments again from the three front speakers L, C and R are played, in addition to the two rear speakers special environmental signals are output.
- the term "direct sound source” is thus used to describe a sound coming only and directly from a discrete sound source, such as a drum instrument or other instrument, or generally a particular audio object, as shown schematically in, for example, US Pat Fig. 5a represented by a drum instrument. Any additional sounds, such as due to wall reflections, etc. are not present in such a direct sound source.
- the sound signals coming from the two rear speakers Ls, Rs in Fig. 5b are emitted only from environmental signals that are present in the original record or not.
- Such ambient signals or "ambience” signals do not belong to a single sound source, but contribute to the reproduction of the room acoustics a recording and thus lead to the so-called “immersion” feeling of the listener.
- Fig. 5c Another alternative concept, called in-the-band concept, is in Fig. 5c shown schematically.
- Each type of sound ie direct sound sources and ambient sounds, are all positioned around the listener.
- the position of a sound is independent of its characteristics (direct sound sources or ambient sounds) and depends only on the specific design of the algorithm, as described in eg Fig. 5c is shown. So was in Fig. 5c the upmix algorithm determines that the two instruments 1100 and 1102 are positioned laterally relative to the listener while the two instruments 1104 and 1106 are positioned in front of the user.
- the two rear speakers Ls, Rs now also contain portions of the two instruments 1100 and 1102 and no longer just ambient sounds, as in Fig. 5b was still the case where the same instruments were all positioned in front of the user.
- the Ambience Extraction technique also exists using non-negative matrix factorization, especially in the context of a 1-up-N upmix, where N is greater than two.
- a time-frequency distribution (TFD) of the input signal is calculated, for example by means of a short-time Fourier transformation.
- An estimate of the TFD of the direct signal components is derived by a numerical optimization technique called non-negative matrix factorization.
- An estimate of the TFD of the ambient signal is determined by calculating the difference of the TFD of the input signal and the estimate of the TFD for the direct signal.
- the re-synthesis of the time signal of the surround signal is performed using the phase spectrogram of the input signal. Additional post processing is optionally performed to enhance the listening experience of the generated multichannel signal. This procedure is detailed in C. Uhle, A. Walther, O. Hellmuth and J. Herre in "Ambience separation from mono recordings using non-negative matrix factorization", Proceedings of the AES 30th Conference 2007 , described.
- Matrix decoders are known under the keyword Dolby Pro Logic II, DTS Neo: 6 or HarmanKardon / Lexicon Logic 7 and in almost every audio / video receiver which is sold nowadays. As a by-product of their intended functionality, these processes are also able to perform a blind upmix. These decoders use interchannel differences and signal adaptive control mechanisms to produce multichannel output signals.
- frequency domain techniques described by Avendano and Jot are also used to identify and extract the ambience information in stereo audio signals. This method is based on the calculation of an interchannel coherence index and a non-linear mapping function, thereby making it possible to determine the time-frequency regions which are mainly composed of ambient signal components.
- the surround signals are subsequently synthesized and used to feed the surround channels of the multi-channel playback system.
- One component of the direct / ambient high-mix process is the extraction of an environmental signal that is injected into the two back channels Ls, Rs.
- a signal there are certain requirements for a signal to be used as an environment-like signal in the context of a direct / environment upmix process.
- a prerequisite is that no relevant parts of the direct sound sources should be audible in order to be able to locate the direct sound sources safely in front of the listener. This is especially important if the audio signal contains speech or one or more distinguishable speakers. Speech signals generated by a crowd, on the other hand, do not necessarily disturb the listener unless they are located in front of the listener.
- a prerequisite for the sound signal of a movie (a soundtrack) is that the listening experience should conform to the impression created by the images. Audible clues to the localization should therefore not be in contrast to visible clues to the localization. Consequently, if a speaker is seen on the screen, the corresponding language should also be placed in front of the user.
- audio signals d. H. is not necessarily limited to situations where both audio and video signals are presented simultaneously.
- Such other audio signals are for example broadcast signals or audiobooks.
- a listener is accustomed to producing speech from the front channels, and would likely turn around to restore his usual impression if speech were coming from the back channels at once.
- German patent application DE 102006017280.9-55 proposed to subject a once-extracted environmental signal to transient detection and to induce transient suppression without substantial energy losses in the environmental signal.
- a signal substitution is made to replace areas with transients by corresponding signals without transients, but with approximately the same energy.
- the US 6,914,988 For example, it is an object of the present invention to provide a concept for generating a multi-channel signal having a number of output channels which on the one hand provides flexibility and on the other hand provides a high quality product.
- This object is achieved by a device for generating a multi-channel signal according to claim 1, a method for generating a multi-channel signal according to claim 23 or a computer program according to claim 24.
- the present invention is based on the finding that speech components are suppressed in the rear channels, ie in the surrounding channels, so that the rear channels are speech component-free.
- an input signal is highly mixed with one or more channels to provide a direct signal channel and to provide an environmental signal channel or, depending on the implementation, the modified surround signal channel.
- a speech detector is provided to search for speech components in the input signal, the direct channel or the surround channel, such speech components being temporal and / or frequency sections, or even in components of orthogonal decomposition, for example.
- a signal modifier is provided to modify the direct signal produced by the high mixer or a copy of the input signal to suppress the speech signal components there while the direct signal components in the corresponding sections comprising speech signal components are less or not attenuated. Such a modified surround channel signal is then used to generate loudspeaker signals for corresponding loudspeakers.
- the surround signal generated by the high mixer is used directly because the speech components are already suppressed there since the underlying audio signal also had already suppressed speech components.
- the direct channel is calculated not based on the modified input signal but on the basis of the unmodified input signal to selectively suppress the speech components, only in the environment channel, but not in the direct channel, in which the speech components are explicitly desired.
- a signal-dependent processing is thus carried out in order to remove or suppress the speech components in the rear channels or in the ambient signal.
- two essential steps are taken, namely the detection of the occurrence of speech and the suppression of speech, wherein the detection of the occurrence of speech in the input signal, in the direct channel or in the surrounding channel can be made, and wherein the suppression of speech in the surrounding channel directly or indirectly can be made in the input signal, which is then used to generate the surround channel, this modified input signal is not used to generate the direct channel.
- the resulting signals for the rear channels viewed by the user comprise a minimal amount of speech, to get the original sound image before the user (front image).
- the position of the speakers would be positioned outside the front area, somewhere between the listener and the front speakers or, in extreme cases, even behind the listener. This would result in a very disturbing sound perception, especially if the audio signals are presented simultaneously with visual signals, as is the case for instance in films. Therefore, many multi-channel movie soundtracks contain hardly any speech components in the back channels.
- speech signal components detected and suppressed at a suitable location If a particular set of speech components were to be reproduced through the back channels, the position of the speakers would be positioned outside the front area, somewhere between the listener and the front speakers or, in extreme cases, even behind the listener. This would result in a very disturbing sound perception, especially if the audio signals are presented simultaneously with visual signals, as is the case for instance in films. Therefore, many multi-channel movie soundtracks contain hardly any speech components in the back channels.
- Fig. 1 shows a block diagram of an apparatus for generating a multi-channel signal 10, which in Fig. 1 is shown to include a left channel L, a right channel R, a center channel C, an LFE channel, a left rear channel LS, and a right rear channel RS. It should be noted, however, that the present invention is also suitable for any other representations than for this selected 5.1 representation, for example, for a 7.1 representation or for a 3.0 representation, in which case only a left channel, a right channel and a center channel is generated.
- the multi-channel signal 10 with the example six channels, which in Fig.
- 1 is generated from an input signal 12 or "x" having a number of input channels, where the number of input channels is 1 or greater than 1 and equal to 2, for example, when a stereo downmix is input. In general, however, the number of output channels is greater than the number of input channels.
- the apparatus shown includes a high mixer 14 for up-converting the input signal 12 to produce at least one direct signal channel 15 and one environmental signal channel 16 or optionally a modified ambient signal channel 16 '.
- a speech detector 18 adapted to use as input the analysis signal, the input signal 12, as provided at 18a, or to use the direct signal channel 15, as provided at 18b, or to use another signal, which is similar in terms of the temporal / frequency appearance or in terms of its characteristics, as far as speech components, to the input signal 12.
- the speech detector detects a portion of the input signal, the direct channel or z.
- the environmental channel as shown at 18c, in which a speech component occurs.
- This language part can be a significant part of speech, so z.
- a speech portion whose speech property has been derived depending on a particular qualitative or quantitative measure, wherein the qualitative measure and the quantitative measure exceeds a threshold, which is also referred to as speech detection threshold.
- a language property is quantified with a numeric value, and this numeric value is compared to a threshold.
- a decision is made per section, which can be made by one or more decision criteria.
- decision criteria may be, for example, various quantitative features that be compared / weighted with each other or processed somehow in order to come to a yes / no decision.
- the apparatus shown in FIG. 1 further comprises a signal modifier 20 configured to modify the original input signal, as shown at 20a, or adapted to modify the environmental channel 16.
- the signal modifier 20 When the surround channel 16 is modified, the signal modifier 20 outputs a modified surround channel 21, while when the input signal 20a is modified, a modified input signal 20b is output to the high mixer 14 which then outputs the modified surround channel 16 '. B. generated by the same Hochmischvorgang that has been used for the direct channel 15. Should this hyperbolic process also result in a direct channel due to the modified input signal 20b, then this direct channel would be discarded since a direct channel derived from the unmodified (without speech suppression) input signal 12 and not from the modified input signal 20b is used as the direct channel ,
- the signal modifier is configured to modify portions of the at least one environmental channel or the input signal, which portions may be temporal or frequency portions or portions of orthogonal decomposition, for example.
- the portions corresponding to the portions detected by the speech detector are modified such that the signal modifier, as illustrated, generates the modified surround channel 21 or the modified input signal 20b in which a speech portion is attenuated or eliminated, wherein the speech portion in the corresponding portion of the direct channel has been less, or at best, not attenuated at all.
- the in Fig. 1 The apparatus shown has a speaker signal output device 22 for outputting of speaker signals in a playback scenario, such as the one in FIG Fig. 1 5.1 scenario shown by way of example, but also a 7.1 scenario, a 3.0 scenario or another or even higher scenario is also possible.
- a playback scenario such as the one in FIG Fig. 1 5.1 scenario shown by way of example, but also a 7.1 scenario, a 3.0 scenario or another or even higher scenario is also possible.
- the at least one direct channel and the at least one modified surround channel are used, where the modified surround channel may either originate from the signal modifier 20, as shown at 21 or originate from the high mixer 14, as at 16 'is shown.
- two modified surround channels 21 could be fed directly into the two loudspeaker signals Ls, Rs, while the direct channels are fed only to the three front loudspeakers L, R, C, thus allowing complete separation between ambient signal components and direct signal components.
- the direct signal components are then all in front of the user and the surrounding signal components are all behind the user.
- ambient signal components can typically also be introduced to a smaller percentage in the front channels, so that z. B. in Fig. 5b shown direct / ambient scenario arises in which not only surround channels ambient signals are generated, but also from the front speakers z. L, C, R.
- surrounding signal components will also be mainly from the front speakers z. B. L, R, C output, but also direct signal components are at least partially fed into the two rear speakers Ls, Rs.
- a placement of the two direct signal sources 1100 and 1102 in Fig. 5c At the locations shown, the proportion of the source 1100 in the speaker L will be about the same as in the speaker Ls, so according to a typical panning rule, the source 1100 can be placed midway between L and Ls.
- the loudspeaker signal output device 22 can thus effect a direct passage of an input-side channel or can map the surrounding channels and the direct channels, for example by an in-band concept or a direct / ambient concept, such that a distribution of the channels takes place on the individual speakers and ultimately, in order to produce the actual speaker signal, an accumulation of the shares can be made from the individual channels.
- Fig. 2 shows a time / frequency split of an analysis signal in the upper portion and an ambient channel or input signal in a lower portion.
- the time is plotted along the horizontal axis and the frequency is plotted along the vertical axis.
- the signal modifier 20 z.
- the speech detector 18 in section 22 detects a speech signal, somehow processes the portion of the surround channel / input signal, such as attenuates, completely eliminates, or substitutes a synthesis signal that has no speech property.
- the division need not be as selective as shown in FIG Fig. 2 is shown. Instead, even a temporal detection can already provide a satisfactory effect, in which case a specific time segment of the analysis signal, for example from second 2 to second 2.1 is detected as containing speech signal, and then the section of the ambient channel or the input signal also between second 2 and 2.1 to achieve speech suppression.
- an orthogonal decomposition can be performed, for. B. by means of a principal component analysis, in which case the same component decomposition is used both in the environment channel or input signal and in the analysis signal. Then, certain components that have been detected as speech components in the analysis signal are attenuated or completely suppressed or eliminated in the ambient channel or input signal. Thus, depending on the implementation, a section is detected in the analysis signal, in which case this section is not necessarily processed in the analysis signal, but possibly also in another signal.
- Fig. 3 shows an implementation of a speech detector in cooperation with an environment channel modifier, wherein the speech detector provides only time information, that is, when Fig. 2 only broadband identifies the first, second, third, fourth or fifth time period and this information is communicated to the surround channel modifier 20 via a control line 18d (FIG. Fig. 1 ) communicates.
- the speech detector 18 and the environmental channel modifier 20, operating synchronously or buffered, together achieve that in the signal to be modified, which may be, for example, the signal 12 or the signal 16, the speech signal is attenuated while ensuring in that such attenuation of the corresponding section in the direct channel does not occur or only occurs to a lesser extent.
- this may be achieved by the high mixer 14 operating without regard to speech components, such as in a matrix method or other method that does not perform special speech processing.
- the direct signal thus obtained is then supplied to the output device 22 without further processing, while the surrounding signal is processed for speech suppression.
- the up-mixer 14 may, so to speak, operate twice to extract the direct channel component based on the original input signal, but to extract the modified surround channel 16 'based on the modified input signal 20b.
- the same high-mix algorithm would run twice, but using a different input signal, in which one input signal the speech component is attenuated and in the other input signal the speech component is not attenuated.
- the environment channel modifier has broadband attenuation functionality or high pass filtering functionality, as set forth below.
- the environmental signal a is extracted from the input signal x, this extraction being part of the functionality of the upmixing 14.
- the occurrence of speech is detected in the surround signal a.
- the detection result d is used in the environment channel modifier 20, which computes the modified surround signal 21 in which speech components are suppressed.
- Fig. 6b shows one to Fig. 6a
- the input signal and not the surrounding signal is supplied to the speech detector 18 as the analysis signal 18a.
- the modified surround channel signal a s becomes similar to the configuration of FIG Fig. 6a calculated, but the language is detected in the input signal. This is motivated by the fact that the speech components are generally more pronounced in the input signal x can be found as in the ambient signal a.
- the in Fig. 6b configuration achieved higher reliability.
- the speech-modified surround signal a s is extracted from a version x s of the input signal that has already undergone speech signal suppression. Since the speech components in x typically emerge more prominently than in an extracted environmental signal, their suppression is safer and more sustainable than in Fig. 6a , Disadvantage of in Fig. 6c shown configuration compared to the configuration in Fig. 6a is that possible artifacts of the speech suppression and the environmental extraction process could still be increased depending on the type of extraction process. However, in Fig. 6c the functionality of the surround channel extractor 14 is only used to extract the environment channel from the modified audio signal. However, the direct channel is not extracted from the modified audio signal x s (20b), but on the basis of the original input signal x (12).
- the surround signal a is extracted from the input signal x by the high mixer.
- the occurrence of speech is detected in the input signal x.
- additional page information e which additionally controls the functionality of the environment channel modifier 20, is calculated by a speech analyzer 30. This page information is calculated directly from the input signal and can determine the position of speech components in a time / frequency representation, for example in the form of a spectrogram of Fig. 2 or may be additional information, which will be discussed in more detail below.
- the functionality of the speech detector 18 will be discussed in greater detail below.
- the task of speech detection is to add a mix of audio signals analyze to estimate a likelihood that language is present.
- the input signal may be a signal that may be composed of a variety of different types of audio signals, such as a music signal, noise, or special sound effects, as known from movies.
- One way to detect speech is to use a pattern recognition system. Pattern recognition is understood to mean analyzing raw data and performing special processing based on a category of a pattern discovered in the raw data. In particular, the term "pattern" or "pattern” describes an underlying similarity that can be found between the measurements of objects of the same categories (classes).
- the basic operations of a pattern recognition system consist in capturing, that is, recording the data using a converter, preprocessing, feature extraction, and classification, which basic operations can be performed in the order given.
- microphones are used as sensors for a speech capture system.
- a preparation may include A / D conversion, resampling, or noise reduction.
- the feature extraction is the calculation of characteristic features for each object from the measurements. The features are chosen to be similar among objects of the same class, so that good intra-class compactness is achieved and that they are different for objects of different classes, so that inter-class separability is achieved.
- a third requirement is that the features should be robust in terms of noise, environmental conditions, and transformations of the input signal that are irrelevant to human perception.
- Feature extraction can be split into two separate stages. The first level is the feature calculation and the second level is feature projection or transformation to a generally orthogonal basis to minimize correlation between feature vectors and to reduce the dimensionality of the features by not using low energy elements.
- a set of training vectors ⁇ xy is defined, where feature vectors are denoted by x i and the set of classes by Y.
- Y has two values, ⁇ speech, non-speech ⁇ .
- the features x i are calculated from designated data, ie from audio signals, in which it is known to which class y they belong.
- the classifier After completing the training, the classifier has learned the characteristics of all classes.
- the features are computed from the unknown data as in the training phase and projected and classified by the classifier on the basis of the knowledge gained in training about the characteristics of the classes.
- voice amplification and noise reduction approaches exist which attenuate or enhance the coefficients of a time / frequency representation according to an estimate of the degree of noise contained in such a time / frequency coefficient.
- a time / frequency plot is obtained from a noisy measurement using, for example, special minimal statistics techniques.
- a noise suppression rule calculates a damping factor using the noise estimate. This principle is known as short-term spectral attenuation or spectral weighting, as it is known, for example, in G. Schmid, "Single-channel noise suppression based on spectral weighting", Eurasip Newsletter 2004.
- STSA short-term spectral attenuation
- speech enhancement techniques and noise reduction techniques introduce audible artifacts into the output signal.
- An example of such artifact is known as music noise or musical tones and results from an erroneous estimation of noise floors and fluctuating subband attenuation factors.
- blind source separation techniques may be used to separate the speech signal components from the surround signal and then separately manipulate both.
- One method is broadband attenuation, as described in US Pat Fig. 3 indicated at 20.
- the audio signal is attenuated at the intervals where speech is present.
- Special amplification factors range between -12 dB and -3 dB, with a preferred attenuation of 6 dB. Since other signal components / components are equally suppressed, one might think that the total loss of audio signal energy is clearly perceived.
- An alternative method also in Fig. 3 at 20 is indicated, consists in a high-pass filtering.
- the audio signal is high-pass filtered where speech is present, with a cutoff frequency in the range between 600 Hz and 3000 Hz.
- the adjustment of the cutoff frequency results from the signal characteristic of speech with respect to the present invention.
- the long-term power spectrum of a speech signal focuses on an area below 2.5 kHz.
- the preferred range of the fundamental frequency of voiced speech is in the range between 75 Hz and 330 Hz.
- a range between 60 Hz and 250 Hz results for male adults.
- Mean values are 120 Hz for male speakers and 215 Hz for female speakers. Due to the resonances in the vocal tract certain signal frequencies are amplified.
- corresponding peaks in the spectrum are also referred to as formant frequencies or simply as formants.
- formant frequencies typically, there are about three significant formants below 3,500 Hz.
- speech exhibits a 1 / F nature, ie the spectral energy decreases with increasing frequency. Therefore, speech components for purposes of the present invention may well by high pass filtering be filtered with the specified cutoff frequency range.
- a first step 40 the fundamental wave of a speech is detected, which detection may take place in the speech detector 18 or, as shown in Fig. 6e, in the speech analyzer 30.
- a step 41 an examination is made to find the harmonics belonging to the fundamental wave.
- This functionality can be performed in the speech detector / speech analyzer or even already in the environment signal modifier.
- a spectrogram is calculated for the ambient signal based on a block-by-block Hin transformation as set forth at 42.
- the actual speech suppression is performed in a step 43, in which the fundamental wave and the harmonics are attenuated in the spectrogram.
- the modified surround signal in which the fundamental and harmonics are attenuated or eliminated, is then re-transformed to achieve the modified surround signal or the modified input signal.
- This sinusoidal signal modeling is often used for tone synthesis, audio coding, source separation, tone manipulation, and noise suppression.
- a signal is represented as a composition of sine waves with time-varying amplitudes and frequencies.
- Tonal speech signal components are manipulated by dividing the partial tones, i. H. the fundamental and its harmonics (harmonics) are identified and modified.
- the partial tones are identified by means of a partial tone finder, as shown at 41.
- Partial tone finding is performed in the time / frequency domain.
- a spectrogram is performed by means of a short-time Fourier transform, as indicated at 42. Local maxima in each spectrum of the spectrogram are detected and trajectories determined by local maxima of neighboring spectra.
- An estimate of the fundamental frequency may support the peak picking process, where this estimate of the fundamental frequency is performed at 40.
- a sine signal representation is then obtained from the trajectories. It should be noted that the order between step 40, 41 and step 42 can also be varied so that an outward transformation 42, which is executed in speech analyzer 30 of FIG Fig. 6d he follows.
- an improved speech signal is obtained by amplifying the sine component.
- the speech suppression according to the invention wants to achieve exactly the opposite, namely to suppress the partial tones, the partial tones comprising the fundamental wave and its harmonics, for a speech segment with tonal speech.
- the high energy speech components are tonal.
- speech is spoken at a level of 60-75 dB for vowels and about 20-30 dB lower for consonants.
- the excitation is a periodic pulse-like signal.
- the excitation signal is filtered by the vocal tract. Consequently, almost all of the energy is one tonal speech segment in the fundamental and its harmonics.
- FIGS. 7 and 8 explain the basic principle of short-term spectral attenuation or spectral weighting.
- the illustrated method estimates the amount of speech contained in a time / frequency tile using so-called low-level features that provide a measure of the "language-like" nature of a signal in a particular frequency span.
- Low level features are low level features in terms of interpretation of their meaning and cost of their computation.
- the audio signal is decomposed into a number of frequency bands by means of a filter bank or a short-time Fourier transform, which in Fig. 7 at 70.
- temporally varying gains for all subbands are computed from such low-level features to attenuate subband signals in proportion to the amount of speech they contain.
- Suitable low-level features are the spectral flatness measure (SFM) and the 4 Hz modulation energy (4HzME).
- SFM measures the degree of tonality of an audio signal and, for a band, results from the quotient of the geometric mean of all spectral values in a band and the arithmetic mean of the spectral components in the band.
- the 4HzME is motivated by the fact that speech has a characteristic energy modulation peak at about 4 Hz, which corresponds to the average syllable rate of a speaker.
- FIG. 12 shows a more detailed illustration of the gain calculation block 71a and 71b of FIG Fig. 7 .
- a multiplicity of different low-level features ie LLF1,..., LLFn are calculated. These features are then combined in a combiner 80 to arrive at a gain g i for a subband.
- the method according to the invention can be implemented in hardware or in software.
- the implementation may be on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which may interact with a programmable computer system such that the method is performed.
- the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention, when the computer program product runs on a computer.
- the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Stereophonic System (AREA)
- Stereo-Broadcasting Methods (AREA)
- Time-Division Multiplex Systems (AREA)
- Dot-Matrix Printers And Others (AREA)
- Color Television Systems (AREA)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PL08802737T PL2206113T3 (pl) | 2007-10-12 | 2008-10-01 | Urządzenie i sposób wytwarzania sygnału wielokanałowego z przetwarzaniem sygnałów mowy |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102007048973A DE102007048973B4 (de) | 2007-10-12 | 2007-10-12 | Vorrichtung und Verfahren zum Erzeugen eines Multikanalsignals mit einer Sprachsignalverarbeitung |
PCT/EP2008/008324 WO2009049773A1 (de) | 2007-10-12 | 2008-10-01 | Vorrichtung und verfahren zum erzeugen eines multikanalsignals mit einer sprachsignalverarbeitung |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2206113A1 EP2206113A1 (de) | 2010-07-14 |
EP2206113B1 true EP2206113B1 (de) | 2011-04-27 |
Family
ID=40032822
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP08802737A Active EP2206113B1 (de) | 2007-10-12 | 2008-10-01 | Vorrichtung und verfahren zum erzeugen eines multikanalsignals mit einer sprachsignalverarbeitung |
Country Status (16)
Country | Link |
---|---|
US (1) | US8731209B2 (ja) |
EP (1) | EP2206113B1 (ja) |
JP (1) | JP5149968B2 (ja) |
KR (1) | KR101100610B1 (ja) |
CN (1) | CN101842834B (ja) |
AT (1) | ATE507555T1 (ja) |
AU (1) | AU2008314183B2 (ja) |
BR (1) | BRPI0816638B1 (ja) |
CA (1) | CA2700911C (ja) |
DE (2) | DE102007048973B4 (ja) |
ES (1) | ES2364888T3 (ja) |
HK (1) | HK1146424A1 (ja) |
MX (1) | MX2010003854A (ja) |
PL (1) | PL2206113T3 (ja) |
RU (1) | RU2461144C2 (ja) |
WO (1) | WO2009049773A1 (ja) |
Families Citing this family (41)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5577787B2 (ja) | 2009-05-14 | 2014-08-27 | ヤマハ株式会社 | 信号処理装置 |
US20110078224A1 (en) * | 2009-09-30 | 2011-03-31 | Wilson Kevin W | Nonlinear Dimensionality Reduction of Spectrograms |
TWI459828B (zh) | 2010-03-08 | 2014-11-01 | Dolby Lab Licensing Corp | 在多頻道音訊中決定語音相關頻道的音量降低比例的方法及系統 |
JP5299327B2 (ja) * | 2010-03-17 | 2013-09-25 | ソニー株式会社 | 音声処理装置、音声処理方法、およびプログラム |
JP5598536B2 (ja) * | 2010-03-31 | 2014-10-01 | 富士通株式会社 | 帯域拡張装置および帯域拡張方法 |
JP5711733B2 (ja) | 2010-06-11 | 2015-05-07 | パナソニック インテレクチュアル プロパティ コーポレーション オブアメリカPanasonic Intellectual Property Corporation of America | 復号装置、符号化装置及びこれらの方法 |
WO2012093290A1 (en) * | 2011-01-05 | 2012-07-12 | Nokia Corporation | Multi-channel encoding and/or decoding |
EP2523473A1 (en) * | 2011-05-11 | 2012-11-14 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for generating an output signal employing a decomposer |
JP5057535B1 (ja) * | 2011-08-31 | 2012-10-24 | 国立大学法人電気通信大学 | ミキシング装置、ミキシング信号処理装置、ミキシングプログラム及びミキシング方法 |
KR101803293B1 (ko) | 2011-09-09 | 2017-12-01 | 삼성전자주식회사 | 입체 음향 효과를 제공하는 신호 처리 장치 및 신호 처리 방법 |
US9280984B2 (en) | 2012-05-14 | 2016-03-08 | Htc Corporation | Noise cancellation method |
TWI545562B (zh) * | 2012-09-12 | 2016-08-11 | 弗勞恩霍夫爾協會 | 用於提升3d音訊被導引降混性能之裝置、系統及方法 |
JP6054142B2 (ja) * | 2012-10-31 | 2016-12-27 | 株式会社東芝 | 信号処理装置、方法およびプログラム |
WO2014112792A1 (ko) * | 2013-01-15 | 2014-07-24 | 한국전자통신연구원 | 사운드 바를 위한 오디오 신호 처리 장치 및 방법 |
BR112015021520B1 (pt) * | 2013-03-05 | 2021-07-13 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V | Aparelho e método para criar um ou mais sinais do canal de saída de áudio dependendo de dois ou mais sinais do canal de entrada de áudio |
EP2830064A1 (en) | 2013-07-22 | 2015-01-28 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for decoding and encoding an audio signal using adaptive spectral tile selection |
RU2639952C2 (ru) | 2013-08-28 | 2017-12-25 | Долби Лабораторис Лайсэнзин Корпорейшн | Гибридное усиление речи с кодированием формы сигнала и параметрическим кодированием |
EP2866227A1 (en) | 2013-10-22 | 2015-04-29 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Method for decoding and encoding a downmix matrix, method for presenting audio content, encoder and decoder for a downmix matrix, audio encoder and audio decoder |
US10176818B2 (en) * | 2013-11-15 | 2019-01-08 | Adobe Inc. | Sound processing using a product-of-filters model |
KR101808810B1 (ko) * | 2013-11-27 | 2017-12-14 | 한국전자통신연구원 | 음성/무음성 구간 검출 방법 및 장치 |
CN104683933A (zh) | 2013-11-29 | 2015-06-03 | 杜比实验室特许公司 | 音频对象提取 |
EP3095117B1 (en) * | 2014-01-13 | 2018-08-22 | Nokia Technologies Oy | Multi-channel audio signal classifier |
JP6274872B2 (ja) * | 2014-01-21 | 2018-02-07 | キヤノン株式会社 | 音処理装置、音処理方法 |
US10362422B2 (en) | 2014-08-01 | 2019-07-23 | Steven Jay Borne | Audio device |
US20160071524A1 (en) * | 2014-09-09 | 2016-03-10 | Nokia Corporation | Audio Modification for Multimedia Reversal |
CN104409080B (zh) * | 2014-12-15 | 2018-09-18 | 北京国双科技有限公司 | 语音端点检测方法和装置 |
WO2016156237A1 (en) * | 2015-03-27 | 2016-10-06 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for processing stereo signals for reproduction in cars to achieve individual three-dimensional sound by frontal loudspeakers |
CN106205628B (zh) | 2015-05-06 | 2018-11-02 | 小米科技有限责任公司 | 声音信号优化方法及装置 |
US10038967B2 (en) * | 2016-02-02 | 2018-07-31 | Dts, Inc. | Augmented reality headphone environment rendering |
WO2017202680A1 (en) * | 2016-05-26 | 2017-11-30 | Telefonaktiebolaget Lm Ericsson (Publ) | Method and apparatus for voice or sound activity detection for spatial audio |
CN109416912B (zh) * | 2016-06-30 | 2023-04-11 | 杜塞尔多夫华为技术有限公司 | 一种对多声道音频信号进行编码和解码的装置和方法 |
CN106412792B (zh) * | 2016-09-05 | 2018-10-30 | 上海艺瓣文化传播有限公司 | 对原立体声文件重新进行空间化处理并合成的***及方法 |
WO2018053518A1 (en) * | 2016-09-19 | 2018-03-22 | Pindrop Security, Inc. | Channel-compensated low-level features for speaker recognition |
EP3382703A1 (en) * | 2017-03-31 | 2018-10-03 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and methods for processing an audio signal |
US9820073B1 (en) | 2017-05-10 | 2017-11-14 | Tls Corp. | Extracting a common signal from multiple audio signals |
CN114125691A (zh) | 2017-12-18 | 2022-03-01 | 杜比国际公司 | 用于在虚拟现实环境中呈现音频信号的方法和*** |
US11019201B2 (en) | 2019-02-06 | 2021-05-25 | Pindrop Security, Inc. | Systems and methods of gateway detection in a telephone network |
US12015637B2 (en) | 2019-04-08 | 2024-06-18 | Pindrop Security, Inc. | Systems and methods for end-to-end architectures for voice spoofing detection |
KR102164306B1 (ko) * | 2019-12-31 | 2020-10-12 | 브레인소프트주식회사 | 디제이변환에 기초한 기본주파수 추출 방법 |
CN111654745B (zh) * | 2020-06-08 | 2022-10-14 | 海信视像科技股份有限公司 | 多声道的信号处理方法及显示设备 |
CN114630057B (zh) * | 2022-03-11 | 2024-01-30 | 北京字跳网络技术有限公司 | 确定特效视频的方法、装置、电子设备及存储介质 |
Family Cites Families (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH03236691A (ja) * | 1990-02-14 | 1991-10-22 | Hitachi Ltd | テレビジョン受信機用音声回路 |
JPH07110696A (ja) * | 1993-10-12 | 1995-04-25 | Mitsubishi Electric Corp | 音声再生装置 |
JP3412209B2 (ja) * | 1993-10-22 | 2003-06-03 | 日本ビクター株式会社 | 音響信号処理装置 |
EP1076928B1 (en) | 1998-04-14 | 2010-06-23 | Hearing Enhancement Company, Llc. | User adjustable volume control that accommodates hearing |
US6928169B1 (en) * | 1998-12-24 | 2005-08-09 | Bose Corporation | Audio signal processing |
JP2001069597A (ja) * | 1999-06-22 | 2001-03-16 | Yamaha Corp | 音声処理方法及び装置 |
FR2797343B1 (fr) | 1999-08-04 | 2001-10-05 | Matra Nortel Communications | Procede et dispositif de detection d'activite vocale |
JP4463905B2 (ja) * | 1999-09-28 | 2010-05-19 | 隆行 荒井 | 音声処理方法、装置及び拡声システム |
US6351733B1 (en) * | 2000-03-02 | 2002-02-26 | Hearing Enhancement Company, Llc | Method and apparatus for accommodating primary content audio and secondary content remaining audio capability in the digital audio production process |
US7177808B2 (en) * | 2000-11-29 | 2007-02-13 | The United States Of America As Represented By The Secretary Of The Air Force | Method for improving speaker identification by determining usable speech |
US20040086130A1 (en) * | 2002-05-03 | 2004-05-06 | Eid Bradley F. | Multi-channel sound processing systems |
US7257231B1 (en) * | 2002-06-04 | 2007-08-14 | Creative Technology Ltd. | Stream segregation for stereo signals |
US7567845B1 (en) * | 2002-06-04 | 2009-07-28 | Creative Technology Ltd | Ambience generation for stereo signals |
CN1774957A (zh) * | 2003-04-17 | 2006-05-17 | 皇家飞利浦电子股份有限公司 | 音频信号生成 |
JP4597967B2 (ja) * | 2003-04-17 | 2010-12-15 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | オーディオ信号生成 |
SE0400997D0 (sv) * | 2004-04-16 | 2004-04-16 | Cooding Technologies Sweden Ab | Efficient coding of multi-channel audio |
SE0400998D0 (sv) | 2004-04-16 | 2004-04-16 | Cooding Technologies Sweden Ab | Method for representing multi-channel audio signals |
SE0402652D0 (sv) * | 2004-11-02 | 2004-11-02 | Coding Tech Ab | Methods for improved performance of prediction based multi- channel reconstruction |
JP2007028065A (ja) * | 2005-07-14 | 2007-02-01 | Victor Co Of Japan Ltd | サラウンド再生装置 |
WO2007034806A1 (ja) * | 2005-09-22 | 2007-03-29 | Pioneer Corporation | 信号処理装置、信号処理方法、信号処理プログラムおよびコンピュータに読み取り可能な記録媒体 |
JP4940671B2 (ja) * | 2006-01-26 | 2012-05-30 | ソニー株式会社 | オーディオ信号処理装置、オーディオ信号処理方法及びオーディオ信号処理プログラム |
WO2007096792A1 (en) * | 2006-02-22 | 2007-08-30 | Koninklijke Philips Electronics N.V. | Device for and a method of processing audio data |
KR100773560B1 (ko) * | 2006-03-06 | 2007-11-05 | 삼성전자주식회사 | 스테레오 신호 생성 방법 및 장치 |
DE102006017280A1 (de) * | 2006-04-12 | 2007-10-18 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Vorrichtung und Verfahren zum Erzeugen eines Umgebungssignals |
-
2007
- 2007-10-12 DE DE102007048973A patent/DE102007048973B4/de active Active
-
2008
- 2008-10-01 CN CN2008801112350A patent/CN101842834B/zh active Active
- 2008-10-01 KR KR1020107007771A patent/KR101100610B1/ko active IP Right Grant
- 2008-10-01 JP JP2010528297A patent/JP5149968B2/ja active Active
- 2008-10-01 US US12/681,809 patent/US8731209B2/en active Active
- 2008-10-01 AU AU2008314183A patent/AU2008314183B2/en active Active
- 2008-10-01 EP EP08802737A patent/EP2206113B1/de active Active
- 2008-10-01 AT AT08802737T patent/ATE507555T1/de active
- 2008-10-01 WO PCT/EP2008/008324 patent/WO2009049773A1/de active Application Filing
- 2008-10-01 MX MX2010003854A patent/MX2010003854A/es active IP Right Grant
- 2008-10-01 DE DE502008003378T patent/DE502008003378D1/de active Active
- 2008-10-01 ES ES08802737T patent/ES2364888T3/es active Active
- 2008-10-01 PL PL08802737T patent/PL2206113T3/pl unknown
- 2008-10-01 CA CA2700911A patent/CA2700911C/en active Active
- 2008-10-01 BR BRPI0816638-2A patent/BRPI0816638B1/pt active IP Right Grant
- 2008-10-01 RU RU2010112890/08A patent/RU2461144C2/ru active
-
2011
- 2011-01-12 HK HK11100278.0A patent/HK1146424A1/xx unknown
Also Published As
Publication number | Publication date |
---|---|
KR101100610B1 (ko) | 2011-12-29 |
MX2010003854A (es) | 2010-04-27 |
JP5149968B2 (ja) | 2013-02-20 |
KR20100065372A (ko) | 2010-06-16 |
BRPI0816638A2 (pt) | 2015-03-10 |
US20100232619A1 (en) | 2010-09-16 |
RU2461144C2 (ru) | 2012-09-10 |
RU2010112890A (ru) | 2011-11-20 |
WO2009049773A1 (de) | 2009-04-23 |
US8731209B2 (en) | 2014-05-20 |
DE102007048973A1 (de) | 2009-04-16 |
DE502008003378D1 (de) | 2011-06-09 |
CA2700911A1 (en) | 2009-04-23 |
AU2008314183B2 (en) | 2011-03-31 |
ES2364888T3 (es) | 2011-09-16 |
EP2206113A1 (de) | 2010-07-14 |
BRPI0816638B1 (pt) | 2020-03-10 |
AU2008314183A1 (en) | 2009-04-23 |
CA2700911C (en) | 2014-08-26 |
JP2011501486A (ja) | 2011-01-06 |
DE102007048973B4 (de) | 2010-11-18 |
ATE507555T1 (de) | 2011-05-15 |
CN101842834A (zh) | 2010-09-22 |
HK1146424A1 (en) | 2011-06-03 |
PL2206113T3 (pl) | 2011-09-30 |
CN101842834B (zh) | 2012-08-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2206113B1 (de) | Vorrichtung und verfahren zum erzeugen eines multikanalsignals mit einer sprachsignalverarbeitung | |
DE102006050068B4 (de) | Vorrichtung und Verfahren zum Erzeugen eines Umgebungssignals aus einem Audiosignal, Vorrichtung und Verfahren zum Ableiten eines Mehrkanal-Audiosignals aus einem Audiosignal und Computerprogramm | |
EP2402943B1 (de) | Vorrichtung und Verfahren zum Erzeugen eines Umgebungssignals | |
DE602005005186T2 (de) | Verfahren und system zur schallquellen-trennung | |
CN103402169B (zh) | 用于提取和改变音频输入信号的混响内容的方法和装置 | |
EP1854334B1 (de) | Vorrichtung und verfahren zum erzeugen eines codierten stereo-signals eines audiostücks oder audiodatenstroms | |
DE69827775T2 (de) | Tonkanalsmischung | |
EP2730102B1 (en) | Method and apparatus for decomposing a stereo recording using frequency-domain processing employing a spectral weights generator | |
EP2891334B1 (de) | Erzeugung von mehrkanalton aus stereo-audiosignalen | |
RU2663345C2 (ru) | Устройство и способ масштабирования центрального сигнала и улучшения стереофонии на основе отношения сигнал-понижающее микширование | |
DE10148351B4 (de) | Verfahren und Vorrichtung zur Auswahl eines Klangalgorithmus | |
Lopatka et al. | Improving listeners' experience for movie playback through enhancing dialogue clarity in soundtracks | |
WO2015128379A1 (de) | Kodierung und dekodierung eines niederfrequenten kanals in einem audiomultikanalsignal |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20100407 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA MK RS |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
DAX | Request for extension of the european patent (deleted) | ||
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1146424 Country of ref document: HK |
|
REF | Corresponds to: |
Ref document number: 502008003378 Country of ref document: DE Date of ref document: 20110609 Kind code of ref document: P |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502008003378 Country of ref document: DE Effective date: 20110609 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: T3 |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2364888 Country of ref document: ES Kind code of ref document: T3 Effective date: 20110916 |
|
LTIE | Lt: invalidation of european patent or patent extension |
Effective date: 20110427 |
|
REG | Reference to a national code |
Ref country code: PL Ref legal event code: T3 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110727 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110829 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FD4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110827 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110728 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 Ref country code: IE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 |
|
REG | Reference to a national code |
Ref country code: HK Ref legal event code: GR Ref document number: 1146424 Country of ref document: HK |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20120130 |
|
BERE | Be: lapsed |
Owner name: FRAUNHOFER-GESELLSCHAFT ZUR FORDERUNG DER ANGEWAN Effective date: 20111031 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502008003378 Country of ref document: DE Effective date: 20120130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111031 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111031 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111001 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110727 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20121031 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20121031 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110427 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 507555 Country of ref document: AT Kind code of ref document: T Effective date: 20131001 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20131001 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 8 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 9 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 10 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 11 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230512 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: TR Payment date: 20230919 Year of fee payment: 16 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: PL Payment date: 20230919 Year of fee payment: 16 Ref country code: NL Payment date: 20231023 Year of fee payment: 16 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20231025 Year of fee payment: 16 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20231117 Year of fee payment: 16 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20231031 Year of fee payment: 16 Ref country code: FR Payment date: 20231023 Year of fee payment: 16 Ref country code: DE Payment date: 20231018 Year of fee payment: 16 |