EP2610867B1 - Dispositif de reproduction audio et méthode de reproduction audio - Google Patents

Dispositif de reproduction audio et méthode de reproduction audio Download PDF

Info

Publication number
EP2610867B1
EP2610867B1 EP13161700.3A EP13161700A EP2610867B1 EP 2610867 B1 EP2610867 B1 EP 2610867B1 EP 13161700 A EP13161700 A EP 13161700A EP 2610867 B1 EP2610867 B1 EP 2610867B1
Authority
EP
European Patent Office
Prior art keywords
information
audio signal
basic
stereo
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP13161700.3A
Other languages
German (de)
English (en)
Other versions
EP2610867A1 (fr
Inventor
Takashi Yokoyama
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Socionext Inc
Original Assignee
Panasonic Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Corp filed Critical Panasonic Corp
Publication of EP2610867A1 publication Critical patent/EP2610867A1/fr
Application granted granted Critical
Publication of EP2610867B1 publication Critical patent/EP2610867B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes
    • G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing

Definitions

  • the present invention relates to an audio reproducing device which decodes an encoded audio signal and reproduces the decoded audio signal.
  • SBR spectral band replication
  • HQ-SBR high-quality SBR
  • LP-SBR low-power SBR
  • HQ-SBR performs complex arithmetic for overall processing of sub-band analysis, high-band generation, and sub-band synthesis.
  • HQ-SBR is suitable for enhancing sound quality, but requires a large amount of computation.
  • LP-SBR performs real number operations instead of the complex arithmetic of HQ-SBR.
  • LP-SBR is designed to reduce aliasing distortion generated by the real number operation.
  • LP-SBR is capable of significantly reducing the amount of computation, and achieving, at low bit rates, the sound quality equivalent to that of HQ-SBR. It is known that LP-SBR requires only approximately half the amount of processing that is required in HQ-SBR (See Non-Patent Literature (NPL) 1).
  • AAC Advanced Audio Coding
  • HE-AAC High-Efficiency AAC
  • AAC+LP-SBR requires only approximately 70 % of the processing amount that is required in AAC+HQ-SBR (see NPL 1).
  • PS Parametric Stereo
  • SBR Complex Quadrature Mirror Filter
  • PS is used in combination with AAC and SBR, and the combined configuration is referred to as HE-AACv2 profile.
  • PS needs to be used in combination with HQ-SBR which uses the complex QMF (see Non-Patent Literatures 2 and 3).
  • AAC may be used in combination with either HQ-SBR or LP-SBR.
  • the HE-AAC profile and HE-AACv2 profile have a concept of levels. The higher the level is, the more variety of types of signals can be decoded. Examples of the types here include maximum sampling frequency or maximum number of channels of an encoded input audio signal, and maximum sampling frequency of a decoded output audio signal (see NPL 3).
  • TILMAN LIEBCHEN "Proposed 2nd Edition of ISO/IEC 14496-3:2005/Amd.2, Audio Lossless Coding (ALS), new audio profiles and BSAC extensions", 79.
  • MPEG MEETING 15-01-2007 - 19-01-2007; MARRAKECH; (MOTION PICTUREEXPERT GROUP OR ISO/IEC JTC1/SC29/WG11), no. M14193, 10 January 2007 (2007-01-10 )
  • NPL 3 in the case where the technique of decoding encoded audio signals complies with HE-AACv2 profile and there is PS data, processing needs to be performed in combination with HQ-SBR. However, in the case where there is no PS data, processing may be performed in combination with either HQ-SBR or LP-SBR.
  • such a method for preventing the increase in the computation amount, such a method can be considered which switches SBR depending on the state of the stream to be decoded. More specifically, when HQ-SBR needs to be used, that is, when there is PS data, HQ-SBR is used. In other cases, that is, when there is no PS data, LP-SBR is used for reducing the increase in the computation amount.
  • HQ-SBR is switched to LP-SBR.
  • LP-SBR is switched to HQ-SBR when the state where a stream includes PS data without any missing, but SBR and stereo processing cannot be executed because SBR header is not yet obtained, is changed to the state where the SBR header is obtained.
  • HQ-SBR performs complex arithmetic for QMF filtering
  • LP-SBR performs real number operations for QMF filtering.
  • HQ-SBR and LP-SBR have different formats of delay information, which does not allow HQ-SBR and LP-SBR to share the delay information of the QMF filtering.
  • delay information of the QMF filtering becomes discontinuous at the time of switching of SBR, thereby generating abnormal sounds.
  • FIG. 7 shows an output audio signal for a single channel in the case where SBR is switched at times t0 and t2. It is shown that abnormal sounds are generated during the periods between t0 and t1 and between t2 and t3 because delay information cannot be used due to the switching of SBR (in FIG. 7 , (b) shows a normal audio signal). In such a manner, attempts to prevent the increase in computation amount by switching SBR results in generating abnormal sounds at the time of switching of SBR.
  • the present invention has been conceived in order to solve the problem, and has an object to provide an audio reproducing device and an audio reproducing method which prevent occurrence of abnormal sounds without significantly increasing the computation amount even when an encoded input audio signal is a multi-channel signal.
  • two separate processing having different processing amount are switched based on the analysis information indicating the type of basic codec.
  • more appropriate processing can be selected.
  • processing is switched based on the analysis information; and thus, processing is not switched while the type of the basic codec is the same. As a result, it is possible to prevent abnormal sounds which may occur at the time of switching of processing.
  • the stream separating unit separates, on the frame basis, the stream into the basic codec, the bandwidth extension information, and stereo extension information that is used for performing stereo processing on the basic codec
  • the audio reproducing device further includes: a stereo extension processing unit which performs, by using the stereo extension information, stereo processing on the decoded basic codec signal having the frequency band extended by the second bandwidth extension processing unit.
  • the basic codec information analyzing unit analyzes the basic codec separated by the stream separating unit, to generate analysis information including at least one of channel information and sampling frequency information, the channel information indicating the number of channels of the basic codec, the sampling frequency information indicating a sampling frequency of the basic codec, and that the switching unit determines at least one of (i) whether the number of channels indicated by the channel information is greater than a predetermined first threshold and (ii) whether the sampling frequency indicated by the sampling frequency information is greater than a predetermined second threshold, and select the first bandwidth extension processing unit when at least one of the following is determined: (i) the number of channels is greater than the predetermined first threshold and (ii) the sampling frequency is greater than the predetermined second threshold.
  • a first processing is selected which requires less processing amount but produces lower accuracy.
  • the first processing is also selected which requires less processing amount but produces lower accuracy.
  • the audio reproducing device further includes a buffer which stores stereo extension information of a first frame, wherein the stereo extension processing unit performs stereo processing on a decoded basic codec signal of a second frame by using the stereo extension information stored in the buffer, the second frame being a frame after the first frame and being a frame in which the stereo extension information is missing.
  • stereo extension information used for stereo processing is stored in a buffer, and the stereo extension information stored in the buffer is used when stereo extension information cannot be obtained.
  • stereo processing can be properly performed on the frame.
  • the second bandwidth extension processing unit generates a high-frequency component signal from the decoded basic codec signal by using the bandwidth extension information
  • the stereo extension processing unit performs, by using the stereo extension information, stereo processing on the decoded basic codec signal and the high-frequency component signal generated by the second bandwidth extension processing unit, to generate a decoded basic codec signal and a high-frequency component signal for a first channel and a decoded basic codec signal and a high-frequency component signal for a second channel
  • the second bandwidth extension processing unit further includes a band synthesis filter for synthesizing the high-frequency component signal and the decoded basic codec signal that have been generated, and synthesizes bands of the second channel by using delay information that is stored in the band synthesis filter of the first channel, as delay information stored in the band synthesis filter of the second channel, when the stereo extension information is missing.
  • the obtained delay information is used as delay information for the other channel.
  • bands of the respective signals of two channels can be properly synthesized.
  • the basic codec is an audio signal encoded according to Advanced Audio Coding (AAC) scheme
  • the bandwidth extension information is Spectral Band Replication (SBR) information generated according to SBR scheme
  • the stereo extension information is Parametric Stereo (PS) information generated according to PS scheme
  • the first bandwidth extension processing unit extends a frequency band of the decoded basic codec signal according to Low Power-SBR (LP-SBR) scheme
  • the second bandwidth extension processing unit extends a frequency band of the decoded basic codec signal according to High Quality-SBR (HQ-SBR) scheme.
  • AAC Advanced Audio Coding
  • SBR Spectral Band Replication
  • PS Parametric Stereo
  • the present invention may be implemented not only as an audio reproducing device, but also as an audio reproducing method which includes processing units of the audio reproducing device as steps.
  • the present invention may be also implemented as a program causing a computer to execute these steps.
  • the present invention may be implemented as a computer-readable recording medium, such as a Compact Disc-Read Only Memory (CD-ROM), which records the program therein, and as information, data, or signals indicating the program.
  • CD-ROM Compact Disc-Read Only Memory
  • Such program, information, data, and signals may be distributed over a communication network such as the Internet.
  • each audio reproducing device above may be in a form of a single system large scale integration (LSI).
  • the system LSI is a ultra-multifunctional LSI which is produced by integrating a plurality of constitutional units on a single chip. More specifically, the system LSI is a computer system including, for example, a microprocessor, a ROM, and a Random Access Memory (RAM).
  • LSI system large scale integration
  • An audio reproducing device is characterized in switching between two bandwidth extension processing having different characteristics based on an analysis result of basic codec, regardless of validity of stereo extension information used for performing stereo processing on a monaural audio signal.
  • the two bandwidth extension processing are: processing which requires larger processing amount but produces higher accuracy, that is, processing for outputting an audio signal with excellent sound quality; and processing which requires less processing amount but produces lower accuracy.
  • FIG. 1 is a block diagram illustrating an example of a structure of an audio reproducing device 100 according to Embodiment 1.
  • the audio reproducing device 100 in FIG. 1 includes: a stream separating unit 101; a basic codec analyzing unit 102; a basic codec decoding unit 103; a bandwidth extension data analyzing unit 104; a stereo extension data analyzing unit 105; a first bandwidth extension processing unit 106; a second bandwidth extension processing unit 107; a stereo extension processing unit 108; and a switching unit 109.
  • the stream separating unit 101 separates an input stream into basic codec, bandwidth extension data, and stereo extension data. When an input stream includes no stereo extension data, the stream separating unit 101 separates the stream into basic codec and bandwidth extension data. The stream separating unit 101 then transmits the separated basic codec to the basic codec analyzing unit 102, transmits the bandwidth extension data to the bandwidth extension data analyzing unit 104, and transmits the stereo extension data to the stereo extension data analyzing unit 105.
  • the stream input to the audio reproducing device 100 is, for example, a stream having HE-AACv2 profile.
  • the basic codec is an encoded audio signal, and is, for example, an audio signal encoded in accordance with AAC scheme.
  • the bandwidth extension data is data used for extending bandwidth of the basic codec, and is, for example, SBR data.
  • the stereo extension data is data used for performing stereo processing on a monaural audio signal, and is, for example, PS data.
  • the basic codec analyzing unit 102 generates basic codec analysis information by analyzing the basic codec transmitted from the stream separating unit 101.
  • the basic codec analysis information includes, for example, channel information representing the number of channels (CH) of the basic codec, and sampling frequency information representing the sampling frequency (FS) of the basic codec.
  • the basic codec analyzing unit 102 transmits the generated basic codec analysis information to the basic codec decoding unit 103.
  • the basic codec analyzing unit 102 also transmits the channel information and the sampling frequency information to the switching unit 109.
  • the basic codec decoding unit 103 decodes the basic codec by using the basic codec analysis information transmitted from the basic codec analyzing unit 102, and generates a decoded basic codec signal. The basic codec decoding unit 103 then transmits the decoded basic codec signal to the switching unit 109.
  • the bandwidth extension data analyzing unit 104 analyzes the bandwidth extension data transmitted from the stream separating unit 101 to generate bandwidth extension information, and transmits the generated bandwidth extension information to the switching unit 109.
  • the bandwidth extension information includes, for example, side information used for prediction for reconstruction of high band of the decoded basic codec signal using the SBR technique.
  • the stereo extension data analyzing unit 105 analyzes the stereo extension data transmitted from the stream separating unit 101 to generate stereo extension information, and transmits the generated stereo extension information to the stereo extension processing unit 108.
  • the stereo extension information is, for example, information used for performing stereo extension processing (also referred to as stereo processing) on a monaural audio signal using the PS technique.
  • the first bandwidth extension processing unit 106 extends the frequency band of the decoded basic codec signal by using the bandwidth extension information transmitted from the switching unit 109 to output an audio signal. More specifically, the first bandwidth extension processing unit 106 predicts and generates high frequency components by using the bandwidth extension information, and synthesizes the bands of the generated high frequency component signal and the decoded basic codec signal to output an audio signal.
  • the first bandwidth extension processing unit 106 has an advantage over the second bandwidth extension processing unit 107 in that the first bandwidth extension processing unit 106 requires less processing amount for processing a same signal. However, the sound quality of the audio signal output by the first bandwidth extension processing unit 106 is lower than that of the audio signal output by the second bandwidth extension processing unit 107.
  • the first bandwidth extension processing unit 106 performs, for example, bandwidth extension based the LP-SBR scheme.
  • the second bandwidth extension processing unit 107 extends the frequency band of the decoded basic codec signal by using the bandwidth extension information transmitted from the switching unit 109 to output an audio signal. More specifically, the second bandwidth extension processing unit 107 predicts and generates high frequency components by using the bandwidth extension information, and synthesizes the bands of the generated high frequency component signal and the decoded basic codec signal to output an audio signal.
  • the sound quality of the audio signal output by the second bandwidth extension processing unit 107 is higher than that of the audio signal output by the first bandwidth extension processing unit 106.
  • the second bandwidth extension processing unit 107 requires processing amount larger than that of the first bandwidth extension processing unit 106.
  • the second bandwidth extension processing unit 107 performs, for example, bandwidth extension based the HQ-SBR scheme.
  • the decoded basic codec signal is an audio signal mainly including low frequency components.
  • the bandwidth extension performed by the first bandwidth extension processing unit 160 and the second bandwidth extension processing unit 107 are processing in which the removed high frequency components are predicted and generated by using bandwidth extension information.
  • the first bandwidth extension processing unit 106 and the second bandwidth extension processing unit 107 each includes a band synthesis filter.
  • the first and second bandwidth extension processing units 106 and 107 reconstruct an output audio signal that is close to an original sound by synthesizing the bands of the decoded basic codec signal generated by the basic codec decoding unit 103 and the high frequency component signal reconstructed based on the decoded basic codec signal by using the bandwidth extension information.
  • the stereo extension processing unit 108 uses stereo extension information transmitted from the stereo extension data analyzing unit 105 to perform stereo processing on the monaural audio signal having a frequency band extended by the second bandwidth extension processing unit 107. More specifically, the stereo extension processing unit 108 performs, by using the stereo extension information, stereo processing on the decoded basic codec signal that is a monaural audio signal and the high frequency component signal generated by the second bandwidth extension processing unit 107, to generate a decoded basic codec signal and a high frequency component signal for left (L) channel, and a decoded basic codec signal and a high frequency component signal for right (R) channel.
  • the stereo extension processing unit 108 performs, for example, stereo processing based on the PS scheme.
  • the stereo extension processing unit 108 has to be used in combination with the second bandwidth extension processing unit 107. In other words, the stereo extension processing unit 108 shares the complex QMF with the second bandwidth extension processing unit 107.
  • the second bandwidth extension processing unit 107 synthesizes the bands of the generated L-channel signals and the bands of the generated R-channel signals.
  • the delay information of the L channel is copied to the delay information of the R channel.
  • band synthesis of R channel is performed using the delay information of the L channel copied for a previous frame, as the delay information of R channel.
  • the delay information of the L channel is information held over frames in the band synthesis filter of the band synthesis processing.
  • the switching unit 109 determines whether the outputs of the basic codec decoding unit 103 and the bandwidth extension data analyzing unit 104 are connected to terminal A or terminal B, based on the number of channels CH and the sampling frequency FS transmitted form the basic codec analyzing unit 102. The determination procedure will be specifically described later with reference to FIG. 3 .
  • the switching unit 109 transmits the decoded basic codec signal transmitted form the basic codec decoding unit 103 and the bandwidth extension information transmitted from the bandwidth extension data analyzing unit 104, to the first bandwidth extension processing unit 106 or the second bandwidth extension processing unit 107 depending on the determination result.
  • the audio reproducing device 100 includes the switching unit 109 which selects one of the two bandwidth extension processing having different characteristics, based on the analysis result of the basic codec.
  • the two bandwidth extension processing are: a first processing which requires less processing amount but produces lower sound quality; and a second processing which requires larger processing amount but produces higher sound quality.
  • FIG. 2 is a flowchart of the operations of the audio reproducing device 100 according to Embodiment 1. The following operations are performed on a frame basis.
  • the stream separating unit 101 separates an input stream into basic codec, bandwidth extension data, and stereo extension data (S101).
  • the basic codec is transmitted to the basic codec analyzing unit 102.
  • the bandwidth extension data is transmitted to the bandwidth extension data analyzing unit 104.
  • the stereo extension data is transmitted to the stereo extension data analyzing unit 105.
  • each of separated data is analyzed (S102). More specifically, the basic codec analyzing unit 102 analyzes the basic codec to generate basic codec analysis information.
  • the bandwidth extension data analyzing unit 104 analyzes the bandwidth extension data to generate bandwidth extension information.
  • the stereo extension data analyzing unit 105 analyzes the stereo extension data to generate stereo extension information. In the case where stereo extension information cannot be generated, such as the case where stereo extension data is missing, the stereo extension data analyzing unit 105 transmits, to the stereo extension processing unit 108, information indicating that there is no stereo extension information.
  • the basic codec decoding unit 103 decodes the basic codec in accordance with the basic codec analysis information (S103).
  • the decoded basic codec signal is transmitted to the switching unit 109.
  • the switching unit 109 determines the connection destination of the transmission path of the decoded basic codec signal based on the basic codec analysis information, and switches between the terminal A and the terminal B based on the determination result (S104). For example, the switching unit 109 refers to the channel information included in the basic codec analysis information, and selects the terminal A when the number of channels CH of the basic codec is greater than a predetermined threshold. Alternatively, the switching unit 109 refers to the sampling frequency information included in the basic codec analysis information, and selects the terminal A when the sampling frequency FS of the basic codec is equal to or greater than a predetermined threshold. In other cases, the switching unit 109 selects the terminal B.
  • the decoded basic codec signal and the bandwidth extension information are transmitted to the first bandwidth extension processing unit 106.
  • the first bandwidth extension processing unit 106 extends the frequency band of the decoded basic codec signal to generate an output audio signal (S106).
  • the first bandwidth extension processing unit 106 executes processing in accordance with the LP-SBR scheme or the like which requires less processing amount but generates an audio signal with lower sound quality.
  • the decoded basic codec signal and the bandwidth extension information are transmitted to the second bandwidth extension processing unit 107.
  • the second bandwidth extension processing unit 107 extends the frequency band of the decoded basic codec signal to generate an output audio signal (S107).
  • the second bandwidth extension processing unit 107 executes processing in accordance with the HQ-SBR scheme or the like which requires larger processing amount but generates an audio signal with higher sound quality.
  • the stereo extension processing unit 108 performs stereo processing on the decoded basic codec signal (monaural audio signal) having the frequency band extended by the second bandwidth extension processing unit 107.
  • the audio signal generated by the first bandwidth extension processing unit 106 or the second bandwidth extension processing unit 107 is output (S108).
  • processing is selected based on the basic codec analysis information representing the type of the basic codec. Accordingly, for example, in the case where processing amount is increased due to multi-channel or higher sampling frequency, it is possible to prevent an increase in the processing amount by selecting the first bandwidth extension processing unit 106 which requires less processing amount.
  • connection destination S104
  • FIG. 3 is a flowchart of a specific example of the operations of the switching unit 109 according to Embodiment 1.
  • the transmission path is connected to the terminal A, and the input bandwidth extension information and the decoded basic codec signal are transmitted to the first bandwidth extension processing unit 106 (S202).
  • the transmission path is connected to the terminal B, and the input bandwidth extension information and the decoded basic codec signal are transmitted to the second bandwidth extension processing unit 107 (S203).
  • FIG. 4 is a diagram illustrating an example of an input stream which includes stereo extension data.
  • the second bandwidth extension processing unit 107 extends the band of the decoded basic codec signal transmitted from the switching unit 109, by using the bandwidth extension information.
  • the stereo extension processing unit 108 performs stereo extension processing by using the stereo extension information, and outputs the stereo audio signal.
  • the number of channels CH is 1.
  • the stereo extension data is information used for performing stereo processing on a monaural audio signal.
  • the number of channels is 1, it represents that the decoded basic codec signal is a monaural audio signal.
  • FIG. 5 is a diagram illustrating an example of an input stream which includes no stereo extension data.
  • the first bandwidth extension processing unit 106 extends the band of the decoded basic codec signal transmitted from the switching unit 109, by using the bandwidth extension information, and outputs an audio signal.
  • the audio reproducing device 100 receives a stream in which stereo extension data is missing in a frame at some point within the stream, and stereo extension data reappears in the subsequent frames.
  • FIG. 6 is a diagram illustrating an example of an input stream including a frame in which stereo extension data is missing.
  • stereo extension data is included in frames 201 and 203, but stereo extension data is missing in a frame 202.
  • the switching unit 109 determines that each frame meets the condition shown in FIG. 3 (Yes in S201), and connects the transmission path to the terminal B (S203).
  • the second bandwidth extension processing unit 107 performs bandwidth extension on each frame.
  • FIG. 7 is a diagram illustrating an example of waveforms of output audio signals.
  • (a) shows a conventional waveform of an output audio signal of the case where HQ-SBR is switched to LP-SBR at time t0 and LP-SBR is switched to HQ-SBR at time t2 due to the missing PS data of the frame 202.
  • Conventionally, such switching of the processing causes abnormal sounds because delay information is not available during the periods between times t0 and t1 and between times t2 and t3.
  • the audio reproducing device 100 determines the first bandwidth extension processing unit 106 or the second bandwidth extension processing unit 107 for performing processing, independently of the existence of the stereo extension data within a stream. More specifically, in the case where the respective frames have the same analysis information of the basic codec, the same processing unit is used for extending the band of the decoded basic codec signal of each frame. Thus, discontinuity of the delay information does not occur, thereby preventing abnormal sounds as shown in FIG. 7(b) .
  • An audio reproducing device includes a buffer for storing stereo extension information. For example, when there is missing stereo extension data under the influences of broadcast receiving, stereo processing is performed by using the stereo extension information stored in the buffer.
  • FIG. 8 is a block diagram illustrating an example of a structure of an audio reproducing device 300 according to Embodiment 2.
  • the audio reproducing device 300 shown in FIG. 8 differs from the audio reproducing device 100 shown in FIG. 1 in that a stereo extension processing unit 308 is included instead of the stereo extension processing unit 108, and that a buffer 310 is further included.
  • a stereo extension processing unit 308 is included instead of the stereo extension processing unit 108
  • a buffer 310 is further included.
  • the stereo extension processing unit 308 stores, in the buffer 310, stereo extension information used for the stereo processing. More specifically, the stereo extension processing unit 308 performs stereo processing on the decoded basic codec signal having the frequency band extended by the second bandwidth extension processing unit 107, by using the stereo extension information transmitted from the stereo extension data analyzing unit 105.
  • the stereo extension information used here is stored in the buffer 310. For example, each time stereo extension information is obtained, the stereo extension processing unit 308 updates the stereo extension information stored in the buffer 310.
  • the stereo extension processing unit 308 reads stereo extension information from the buffer 310, and performs stereo processing on the decoded basic codec signal (monaural audio signal) of the frame by using the read stereo extension information.
  • the buffer 310 stores the stereo extension information transmitted from the stereo extension data analyzing unit 105.
  • the buffer 310 not only stores newest stereo extension information, but also may store a plurality pieces of stereo extension information.
  • the stereo extension processing unit 308, for example refers to the basic codec extension information and uses the stereo extension information used for the stereo processing of a previous decoded basic codec signal similar to the current decoded basic codec signal.
  • the audio reproducing device 300 includes the buffer 310 for storing stereo extension information. In the case where there is no stereo extension information, the audio reproducing device 300 performs stereo processing on the decoded basic codec signal by using the stereo extension information stored in the buffer 310.
  • the audio reproducing device 300 decodes input streams in accordance with the flowcharts shown in FIG. 2 and FIG. 3 .
  • the stereo extension processing unit 308 according to Embodiment 2 performs processing when the second bandwidth extension processing unit 107 performs bandwidth extension (S107).
  • FIG. 9 is a flowchart of the operations of the stereo extension processing unit 308 according to Embodiment 2.
  • the stereo extension processing unit 308 determines whether or not a stream includes stereo extension data, that is, whether or not stereo extension information is transmitted from the stereo extension data analyzing unit 105 (S301). In the case where the stereo extension information is transmitted (Yes in S301), stereo extension processing is performed by using the stereo extension information (S302). The stereo extension processing unit 308 further stores the stereo extension information used here (S303).
  • stereo extension processing is performed by using the stereo extension information stored when decoding previous frames (S305). In the case where no stereo extension processing has been performed (No in S304), the processing ends here.
  • the stereo extension processing unit 308 stores, in the buffer 310, the stereo extension information used for decoding previous frames.
  • stereo processing is performed on the decoded basic codec signal by using the stereo extension information stored in the buffer 310.
  • the switching unit 109 connects the transmission path to the terminal B because all of the frames 201 to 203 has a single channel and sampling frequency FS that is equal to or higher than 24 kHz.
  • the decoded basic codec signal and bandwidth extension information are transmitted to the second bandwidth extension processing unit 107.
  • the bandwidth extension processing on all of the frames 201 to 203 is performed by the second bandwidth extension processing unit 107, which allows continuity of delay information.
  • FIG. 10 is a diagram illustrating an example of waveforms of output stereo audio signals.
  • stereo extension processing is not performed during a period of the frame in which stereo extension data is missing(period between t4 and t5).
  • R-channel audio signal is not output, which gives a listener a feeling of strangeness.
  • the stereo extension processing unit 308 performs the following operations.
  • the stereo extension processing unit 308 performs stereo extension processing (S302), and stores the stereo extension information used here (S303).
  • the frame 202 in which stereo extension data is missing is input. Since stereo extension data is missing in the frame 202 (No in S301) and the stereo extension processing is performed at the time of decoding of the frame 201 (Yes in S304), the stereo extension processing unit 308 performs stereo extension processing on the frame 202 by using the stereo extension information of the frame 201.
  • the stereo extension processing unit 308 performs stereo extension processing on the frame 203 by using the stereo extension information extracted from the frame 203 (S302).
  • the audio reproducing device 300 according to Embodiment 2 is capable of keeping continuity of an output sound, and also performing stereo extension even on a frame in which stereo extension data is missing.
  • FIG. 11 is an external view of an example of an audio reproducing apparatus incorporating an audio reproducing device according to the present invention.
  • FIG. 11 illustrates a recording medium 401, an audio reproducing apparatus 402, and earphones 403.
  • the recording medium 401 is a recording medium which is capable of recording compressed audio streams.
  • FIG. 11 shows the recording medium 401 as a medium, such as a secure digital (SD) card, removable from an apparatus; however, the recording medium 401 may also be implemented as an optical disk, a hard disk drive (HDD) incorporated in the apparatus, or the like.
  • SD secure digital
  • HDD hard disk drive
  • the audio reproducing apparatus 402 is an apparatus which reproduces compressed audio streams, and includes at least one of the audio reproducing devices 100 and 300 according to Embodiments 1 and 2.
  • the earphones 403 are loud speaker apparatus which output audio signals output from the audio reproducing apparatus 402 to outside.
  • FIG. 11 illustrates earphones which are inserted into the ears of a user; however, the earphones may be headphones which are put on the head of the user, or desktop loudspeakers.
  • the audio reproducing apparatus 402 it is possible to obtain an output audio signal without causing abnormal sounds even when a stream includes a frame in which stereo extension data is missing.
  • the switching unit 109 makes the determination based on the determination condition that the number of channels is 1 and the sampling frequency is 24 kHz or lower; however, the determination condition is not limitative. For example, it may be that the switching unit 109 determines to use the second bandwidth extension processing unit 107 (connect to the terminal B) only when the number of channels is two or less. In this case, when a stream having the basic codec with 1 or 2 channels is input, bandwidth extension is performed by the second bandwidth extension processing unit 107 which generates higher sound quality but requires larger processing amount.
  • the present invention may be implemented not only as an audio reproducing device and an audio reproducing method as described above, but also as a program causing a computer to execute an audio reproducing method according to the embodiments.
  • the present invention may also be implemented as a recording medium, such as a computer readable CD-ROM, which stores the program.
  • the present invention may be implemented as information, data, or a signal indicating the program. Such program, information, data, and signal may be distributed over a communication network such as the Internet.
  • portion or all of the constituent elements of the audio reproducing device according to the present invention may be structured as a single system LSI.
  • the system LSI is a super multi-functional LSI manufactured by integrating a plurality of structural units onto a single chip. Specifically, it is a computer system including a microprocessor, a ROM, a RAM, and the like.
  • the present invention prevents a significant increase in processing amount, and also prevents occurrence of abnormal sounds.
  • the present invention may be used for, for example, an audio reproducing device.
  • the present invention may be used for an audio reproducing apparatus, such as a portable music player, which has limited processor capability and limited memory resources.

Landscapes

  • Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Stereophonic System (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Claims (8)

  1. Dispositif de reproduction audio configuré pour reproduire un flux comportant un signal audio codé de codec de base, ledit dispositif de reproduction audio comprenant
    une unité de séparation de flux (101) configurée pour séparer, sur une base de trames, le flux en signal audio codé de base et des informations d'extension de bande passante qui sont utilisées pour étendre une bande du signal audio codé de base ;
    une unité d'analyse d'informations de codec de base (102) configurée pour analyser le signal audio codé de base séparé par ladite unité de séparation de flux, pour générer des informations d'analyse indiquant un type du signal audio codé de base ;
    une unité de décodage de codec de base (103) configurée pour décoder le signal audio codé de base en fonction des informations d'analyse générées par ladite unité d'analyse d'informations de codec de base, pour générer un signal audio de base décodé ;
    une première unité de traitement d'extension de bande passante (106) configurée pour exécuter un premier traitement qui étend, en utilisant les informations d'extension de bande passante, une bande de fréquences du signal audio de base décodé généré par ladite unité de décodage de codec de base ;
    une deuxième unité de traitement d'extension de bande passante (107) configurée pour exécuter un deuxième traitement qui étend, en utilisant les informations d'extension de bande passante, la bande de fréquences du signal audio de base décodé généré par ladite unité de décodage de codec de base, le deuxième traitement étant exécuté avec une précision plus élevée qu'une précision du premier traitement ;
    une unité de commutation configurée pour commuter entre ladite première unité de traitement d'extension de bande passante et ladite deuxième unité de traitement d'extension de bande passante sur la base des informations d'analyse,
    dans lequel les informations d'extension de bande passante sont des informations de Reproduction de Bande Spectrale (SBR) générées selon un schéma SBR,
    ladite première unité de traitement d'extension de bande passante est configurée pour exécuter le premier traitement en utilisant un schéma SBR de Faible Puissance,
    ladite deuxième unité de traitement d'extension de bande passante est configurée pour exécuter le deuxième traitement en utilisant un schéma SBR de Haute Qualité, et
    dans lequel ladite unité d'analyse d'informations de codec de base est configurée pour analyser le signal audio codé de base séparé par ladite unité de séparation de flux, pour générer des informations d'analyse comportant au moins l'une parmi des informations de canal et des informations de fréquence d'échantillonnage, les informations de canal indiquant le nombre de canaux du signal audio codé de base, les informations de fréquence d'échantillonnage indiquant une fréquence d'échantillonnage du signal audio codé de base, et
    ladite unité de commutation est configurée pour déterminer au moins l'une des conditions : (i) si le nombre de canaux indiqué par les informations de canal est supérieur à un premier seuil prédéterminé et (ii) si la fréquence d'échantillonnage indiquée par les informations de fréquence d'échantillonnage est supérieure à un deuxième seuil prédéterminé, et pour choisir ladite première unité de traitement d'extension de bande passante lorsqu'au moins l'une des conditions suivantes est déterminée : (i) le nombre de canaux est supérieur au premier seuil prédéterminé et (ii) la fréquence d'échantillonnage est supérieure au deuxième seuil prédéterminé.
  2. Dispositif de reproduction audio selon la revendication 1,
    dans lequel ladite unité de séparation de flux (101) est configurée pour séparer, sur une base de trames, le flux en signal audio codé de base, informations d'extension de bande passante, et informations d'extension stéréo qui sont utilisées pour effectuer un traitement stéréo sur la base du signal audio codé de base,
    ledit dispositif reproduction audio (100) comprend en outre :
    une unité de traitement d'extension stéréo (108) configurée pour effectuer, en utilisant les informations d'extension stéréo, un traitement stéréo sur le signal audio de base codé ayant la bande de fréquences étendue par ladite deuxième unité de traitement d'extension de bande passante.
  3. Dispositif de reproduction audio selon la revendication 1 ou 2, comprenant en outre
    une mémoire tampon qui stocke des informations d'extension stéréo d'une première trame,
    dans lequel ladite unité de traitement d'extension stéréo est configurée pour effectuer un traitement stéréo sur un signal audio de base décodé d'une deuxième trame en utilisant les informations d'extension stéréo stockées dans ladite mémoire tampon, la deuxième trame étant une trame après la première trame et étant une trame dans laquelle les informations d'extension stéréo sont manquantes.
  4. Dispositif de reproduction audio selon la revendication 1 ou 2,
    dans lequel ladite deuxième unité de traitement d'extension de bande passante est configurée pour générer un signal à composantes de haute fréquence à partir du signal audio de base décodé en utilisant les informations d'extension de bande passante,
    ladite unité de traitement d'extension stéréo est configurée pour effectuer, en utilisant les informations d'extension stéréo, un traitement stéréo sur le signal audio de base décodé et sur le signal à composantes de haute fréquence généré par ladite deuxième unité de traitement d'extension de bande passante, pour générer un signal audio de base décodé et un signal à composantes de haute fréquence pour un premier canal et un signal audio de base décodé et un signal à composantes de haute fréquence pour un deuxième canal, et
    ladite deuxième unité de traitement d'extension de bande passante comporte en outre un filtre de synthèse de bande pour synthétiser le signal à composantes de haute fréquence et le signal audio de base décodé qui ont été générés, et est configuré pour synthétiser des bandes du deuxième canal en utilisant des informations de retard qui sont stockées dans ledit filtre de synthèse de bande du premier canal, en tant qu'informations de retard stockées dans ledit filtre de synthèse de bande du deuxième canal, lorsque les informations d'extension stéréo sont manquantes.
  5. Appareil de reproduction audio comprenant le dispositif de reproduction audio selon l'une des revendication 1 à 4.
  6. Procédé de reproduction audio permettant de reproduire un flux comportant un signal audio codé de base, ledit procédé de reproduction audio comprenant :
    la séparation, sur une base de trames, du flux en signal audio codé de base et des informations d'extension de bande passante qui sont utilisées pour étendre une bande du signal audio codé de base ;
    l'analyse du signal audio codé de base séparé dans ladite séparation, pour générer des informations d'analyse indiquant un type du signal audio codé de base ;
    le décodage du signal audio codé de base en fonction des informations d'analyse générées dans ladite analyse, pour générer un signal audio de base décodé ;
    la commutation entre un premier traitement et un deuxième traitement sur la base des informations d'analyse, du deuxième traitement étant exécuté avec une précision plus élevée qu'une précision du premier traitement ;
    l'exécution, lorsque le premier traitement est choisi dans ladite commutation, du premier traitement qui étend, en utilisant les informations d'extension de bande passante, une bande de fréquences du signal de codec de base décodé généré dans ledit décodage ; et
    l'exécution, lorsque le deuxième traitement est choisi dans ladite commutation, du deuxième traitement qui étend, en utilisant les informations d'extension de bande passante, la bande de fréquences du signal audio de base décodé généré dans ledit décodage,
    dans lequel les informations d'extension de bande passante sont des informations de Reproduction de Bande Spectrale (SBR) générées selon un schéma SBR,
    un nombre réel d'opérations sont effectuées dans le premier traitement en utilisant un schéma SBR de Faible Puissance, et
    une arithmétique complexe est effectuée dans le deuxième traitement en utilisant un schéma SBR de Haute Qualité, et
    dans lequel ladite étape d'analyse consiste à analyser le signal audio codé de base séparé dans ladite étape de séparation de flux, pour générer des informations d'analyse comportant au moins l'une parmi des informations de canal et des informations de fréquence d'échantillonnage, les informations de canal indiquant le nombre de canaux du signal audio codé de base, les informations de fréquence d'échantillonnage indiquant une fréquence d'échantillonnage du signal audio codé de base, et
    ladite étape de commutation détermine au moins l'une des conditions : (i) si le nombre de canaux indiqué par les informations de canal est supérieur à un premier seuil prédéterminé et (ii) si la fréquence d'échantillonnage indiquée par les informations de fréquence d'échantillonnage est supérieure à un deuxième seuil prédéterminé, et choisit ladite première unité de traitement d'extension de bande passante lorsqu'au moins l'une des conditions suivantes est déterminée : (i) le nombre de canaux est supérieur au premier seuil prédéterminé et (ii) la fréquence d'échantillonnage est supérieure au deuxième seuil prédéterminé.
  7. Circuit intégré configuré pour reproduire un flux comportant un signal audio codé de base, ledit circuit intégré comprenant :
    un dispositif de reproduction audio selon la revendication 1.
  8. Programme informatique pour reproduire un flux comportant un signal audio codé de base, ledit programme informatique comprenant : des instructions qui, lorsqu'elles sont exécutées sur un ordinateur, amènent l'ordinateur à mettre en oeuvre le procédé tel que défini dans la revendication 6.
EP13161700.3A 2008-11-21 2009-10-13 Dispositif de reproduction audio et méthode de reproduction audio Active EP2610867B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2008298809A JP5629429B2 (ja) 2008-11-21 2008-11-21 オーディオ再生装置及びオーディオ再生方法
EP09827300.6A EP2360684B1 (fr) 2008-11-21 2009-10-13 Dispositif et procédé de lecture audio

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
EP09827300.6A Division EP2360684B1 (fr) 2008-11-21 2009-10-13 Dispositif et procédé de lecture audio
EP09827300.6 Division 2009-10-13

Publications (2)

Publication Number Publication Date
EP2610867A1 EP2610867A1 (fr) 2013-07-03
EP2610867B1 true EP2610867B1 (fr) 2015-03-11

Family

ID=42197962

Family Applications (2)

Application Number Title Priority Date Filing Date
EP09827300.6A Active EP2360684B1 (fr) 2008-11-21 2009-10-13 Dispositif et procédé de lecture audio
EP13161700.3A Active EP2610867B1 (fr) 2008-11-21 2009-10-13 Dispositif de reproduction audio et méthode de reproduction audio

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP09827300.6A Active EP2360684B1 (fr) 2008-11-21 2009-10-13 Dispositif et procédé de lecture audio

Country Status (4)

Country Link
EP (2) EP2360684B1 (fr)
JP (1) JP5629429B2 (fr)
BR (1) BRPI0921067B1 (fr)
WO (1) WO2010058518A1 (fr)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5692255B2 (ja) * 2010-12-03 2015-04-01 ヤマハ株式会社 コンテンツ再生装置およびコンテンツ処理方法
AU2012279357B2 (en) 2011-07-01 2016-01-14 Dolby Laboratories Licensing Corporation System and method for adaptive audio signal generation, coding and rendering
KR101798126B1 (ko) 2013-01-29 2017-11-16 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. 주파수 향상 오디오 신호를 생성하는 디코더, 디코딩 방법, 인코딩된 신호를 생성하는 인코더, 및 컴팩트 선택 사이드 정보를 이용한 인코딩 방법
US20230410832A1 (en) * 2020-11-05 2023-12-21 Nippon Telegraph And Telephone Corporation Sound signal high frequency compensation method, sound signal post processing method, sound signal decode method, apparatus thereof, program, and storage medium
US20230395092A1 (en) * 2020-11-05 2023-12-07 Nippon Telegraph And Telephone Corporation Sound signal high frequency compensation method, sound signal post processing method, sound signal decode method, apparatus thereof, program, and storage medium
JP7517461B2 (ja) 2020-11-05 2024-07-17 日本電信電話株式会社 音信号高域補償方法、音信号後処理方法、音信号復号方法、これらの装置、プログラム、および記録媒体

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3957589B2 (ja) * 2001-08-23 2007-08-15 松下電器産業株式会社 音声処理装置
JP4281349B2 (ja) * 2001-12-25 2009-06-17 パナソニック株式会社 電話装置
KR100917464B1 (ko) * 2003-03-07 2009-09-14 삼성전자주식회사 대역 확장 기법을 이용한 디지털 데이터의 부호화 방법,그 장치, 복호화 방법 및 그 장치
JP2005114813A (ja) * 2003-10-03 2005-04-28 Matsushita Electric Ind Co Ltd オーディオ信号再生装置及び再生方法
BRPI0418839A (pt) * 2004-05-17 2007-11-13 Nokia Corp método para suportar e dispositivo eletrÈnico suportando uma codificação de um sinal de áudio, sistema de codificação de áudio, e, produto de programa de software
JP2006065002A (ja) * 2004-08-26 2006-03-09 Kenwood Corp コンテンツ再生装置及び方法
JP4567412B2 (ja) * 2004-10-25 2010-10-20 アルパイン株式会社 音声再生機および音声再生方法
CN101288117B (zh) * 2005-10-12 2014-07-16 三星电子株式会社 对音频数据和扩展数据进行编码/解码的方法和设备

Also Published As

Publication number Publication date
JP2010122640A (ja) 2010-06-03
EP2360684B1 (fr) 2013-05-29
BRPI0921067A2 (pt) 2015-12-15
EP2360684A1 (fr) 2011-08-24
EP2610867A1 (fr) 2013-07-03
EP2360684A4 (fr) 2012-09-12
BRPI0921067B1 (pt) 2020-02-18
WO2010058518A1 (fr) 2010-05-27
JP5629429B2 (ja) 2014-11-19

Similar Documents

Publication Publication Date Title
JP4939933B2 (ja) オーディオ信号符号化装置及びオーディオ信号復号化装置
KR101228630B1 (ko) 에너지 정형 장치 및 에너지 정형 방법
JP4685925B2 (ja) 適応残差オーディオ符号化
JP4601669B2 (ja) マルチチャネル信号またはパラメータデータセットを生成する装置および方法
EP3745397B1 (fr) Dispositif de décodage et procédé de décodage et programme
JP4794448B2 (ja) オーディオエンコーダ
JP6155274B2 (ja) 過剰サンプリングされたｓｂｒを使ったアップサンプリング
US20110044458A1 (en) Slot position coding of residual signals of spatial audio coding application
US20080013614A1 (en) Device and method for generating a data stream and for generating a multi-channel representation
EP2610867B1 (fr) Dispositif de reproduction audio et méthode de reproduction audio
JP2010204533A (ja) オーディオ復号装置及びオーディオ復号方法
JP5468020B2 (ja) 音響信号復号装置及びバランス調整方法
KR20130007439A (ko) 신호 처리 장치, 신호 처리 방법 및 프로그램
KR100636145B1 (ko) 확장된 고해상도 오디오 신호 부호화 및 복호화 장치
JPWO2008132826A1 (ja) ステレオ音声符号化装置およびステレオ音声符号化方法
JP2006337767A (ja) 低演算量パラメトリックマルチチャンネル復号装置および方法
WO2021139757A1 (fr) Procédé et dispositif de codage audio, et procédé et dispositif de décodage audio
US10553230B2 (en) Decoding apparatus, decoding method, and program
JP5943982B2 (ja) オーディオ再生装置及びオーディオ再生方法
JP2005114813A (ja) オーディオ信号再生装置及び再生方法
WO2006011367A1 (fr) Codeur et décodeur de signal audio
JP2005181672A (ja) オーディオ再生装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AC Divisional application: reference to earlier application

Ref document number: 2360684

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

17P Request for examination filed

Effective date: 20131021

RBV Designated contracting states (corrected)

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

17Q First examination report despatched

Effective date: 20140304

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20141105

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 19/005 20130101ALN20141024BHEP

Ipc: G10L 19/008 20130101ALN20141024BHEP

Ipc: G10L 21/038 20130101ALI20141024BHEP

Ipc: G10L 19/24 20130101AFI20141024BHEP

INTG Intention to grant announced

Effective date: 20141120

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AC Divisional application: reference to earlier application

Ref document number: 2360684

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 715707

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150415

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602009029994

Country of ref document: DE

Effective date: 20150423

RAP2 Party data changed (patent owner data changed or rights of a patent transferred)

Owner name: SOCIONEXT INC.

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20150311

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20150311

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150611

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 715707

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150311

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150612

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602009029994

Country of ref document: DE

Representative=s name: NOVAGRAAF, FR

Ref country code: DE

Ref legal event code: R081

Ref document number: 602009029994

Country of ref document: DE

Owner name: SOCIONEXT INC., YOKOHAMA-SHI, JP

Free format text: FORMER OWNER: PANASONIC CORP., KADOMA-SHI, OSAKA, JP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150713

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150711

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602009029994

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

26N No opposition filed

Effective date: 20151214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20160218 AND 20160224

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151013

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151031

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151013

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20091013

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150311

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231020

Year of fee payment: 15

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20231026

Year of fee payment: 15

Ref country code: FR

Payment date: 20231025

Year of fee payment: 15

Ref country code: DE

Payment date: 20231020

Year of fee payment: 15