EP1286332A1 - Sound processing method and device for modifying a sound characteristic, such as an impression of age associated to a voice - Google Patents
Sound processing method and device for modifying a sound characteristic, such as an impression of age associated to a voice Download PDFInfo
- Publication number
- EP1286332A1 EP1286332A1 EP01402177A EP01402177A EP1286332A1 EP 1286332 A1 EP1286332 A1 EP 1286332A1 EP 01402177 A EP01402177 A EP 01402177A EP 01402177 A EP01402177 A EP 01402177A EP 1286332 A1 EP1286332 A1 EP 1286332A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- frequency
- sound data
- sound
- sampling
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000003672 processing method Methods 0.000 title 1
- 238000005070 sampling Methods 0.000 claims abstract description 90
- 230000001747 exhibiting effect Effects 0.000 claims abstract description 5
- 230000004048 modification Effects 0.000 claims abstract description 5
- 238000012986 modification Methods 0.000 claims abstract description 5
- 238000000034 method Methods 0.000 claims description 31
- 230000008569 process Effects 0.000 claims description 9
- 230000032683 aging Effects 0.000 claims description 6
- 230000003247 decreasing effect Effects 0.000 claims description 4
- 230000005236 sound signal Effects 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 230000001427 coherent effect Effects 0.000 description 2
- 230000003716 rejuvenation Effects 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 230000002996 emotional effect Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/02—Methods for producing synthetic speech; Speech synthesisers
- G10L13/033—Voice editing, e.g. manipulating the voice of the synthesiser
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/04—Time compression or expansion
Definitions
- the invention relates to a method and device for processing a sound such as a voice delivering a natural or synthesised utterance, so as to controllably modify a characteristic feature thereof, and notably its apparent age.
- a sound such as a voice delivering a natural or synthesised utterance
- the sound characteristics of the human voice evolves significantly with age, from infancy through childhood, youth, adulthood, and advancing years. Similar evolutions in the sounding of utterances can be observed with animals. This makes it possible to associate an age to a voice, simply on the basis of everyday experience.
- an utterance - be it natural or synthesised - conveys an impression of age, hereafter referred to generically as the "age" of the utterance.
- robotic pets, humanoids and the communicating objects can be expected to simulate a lifespan, e.g. starting from infancy at an initial starting phase.
- Other applications are, inter alia, in voice synthesis where it is often desirable to control the age of a voice among other parameters such as male/female, speed of delivery, emotion portrayed, etc.
- Figure 1 illustrates what would be a standard approach for providing a control in the age of an utterance where the latter is produced from samples of a human voice.
- the samples are obtained from a number N of different human speakers S1, S2, S3, ..., SN, each having a respective age A1, A2, A3, ..., AN corresponding to a given age group, e.g. from a child to an elderly person.
- the voices V1, V2, V3, ..., VN obtained from each of the speakers are processed independently of each other.
- the analog voice output is fed to a digitizer 2 where it is converted to successive intensity or amplitude values.
- the digitizer 2 operates according to a given sampling frequency F1 (Hz) provided by a signal from a sampling clock 4 set for that frequency.
- F1 sampling frequency
- the sampling frequency corresponds to the number (F1) of intensity or amplitude values that are sampled i.e. measured, per second on the voice signal. Each sampled value is converted into a digital word of a given number of bits, depending on the required dynamic range.
- the successive digital words corresponding to respective successive samples are compiled into a sound file 6 for reading by a digital sound player, usually a synthesiser which produces a digital-to-analog conversion on the sound data.
- Sound files 6 for respective speakers S1-SN are collected and stored in a sound file memory 8.
- a sound file may contain complete words, or it may result from intermediate processing steps in order to correspond to a collection of basic sound elements extracted from the voice information, allowing words to be formed by so-called concatenative techniques.
- the digital data of a sound file 6 is reproduced by a digital sound player 10, which converts the digitally-expressed intensity or amplitude values as it receives them into a corresponding time-varying audio signal suitably amplified to be played on a speaker 12 or other transducer.
- a chosen sound file is selected from the memory 8 by a sound file reader 14, which serves to determine and organise the digital data to be sent to the digital sound player 10.
- the sound file reader 14 can read selected portions of a sound file 6 to obtain a required concatenation of syllables for producing an utterance.
- the sampled data contained in the sound files are passed onto the digital sound player at the frequency F1 at which they were sampled, so that the delivery rate of the initially-recorded voice is respected; i.e. the playback speed is the same as the recording speed, as per a normal recording and playback system, the speed being in this case set by the sampling frequency F1 which fixes the delivery rate.
- the data supplied to the digital sound player 10 will come from a sound file - or possibly a set of sound files - originating specifically from a voice Vi of a speaker Si having that prescribed age.
- the sound file reader 14 has to access one or more corresponding different sound files from the sound file memory 8.
- the invention proposes a method of controllably modifying a value of a characteristic associated to a sound contained in sound data to which is associated a sampling frequency, the sound exhibiting a given value of the characteristic for a first sampling frequency F1, characterised in that it comprises the steps of:
- the above process is particularly advantageous for controllably modifying an age associated to a voice, the sound data corresponding to at least one utterance, the value of the characteristic being an age associated to the utterance.
- the time-adjusting step is preferably performed by means of a Pitch-Synchronous Overlap and Add (PSOLA) algorithm or the like.
- PSOLA Pitch-Synchronous Overlap and Add
- the sound data can be obtained by sampling an analog sound signal at the first sampling frequency F1.
- All the sound data can obtained by sampling a voice from a same speaker.
- the sound data may also be obtained by generating synthetically values of sound data such as to exhibit the given value of the characteristic for the first sampling frequency F1.
- the time adjusting step causes the sampling duration of the time-adjusted form of the sound data portion sampled at the second frequency F2 to be substantially the same as the first sampling duration R1.
- the time-adjusted sound data can be played substantially at the second frequency F2.
- the second frequency F2 can be substantially continuously variable, e.g. over a range covering frequency values above and below the first frequency F1.
- the latter When applied to controlling an age of a voice, the latter can be controllably increased by correspondingly decreasing the value of the second frequency F2.
- the invention relates to the implementation of the above method in a device which simulates an ageing process, such as a robotic pet having a life-cycle, wherein the second frequency F2 is caused to decrease progressively with time.
- the invention provides a device for controllably modifying a value of a characteristic associated to a sound contained in sound data to which is associated a sampling frequency, the sound exhibiting a given value of the characteristic for a first sampling frequency F1, characterised in that it comprises:
- the invention relates to a device which simulates an ageing process, such as a robotic pet having a life-cycle, characterised in that it comprises a device according to the third aspect and means for causing the second frequency F2 to decrease progressively with time.
- the invention relates to a computer program providing computer executable instructions, which when loaded onto a data processor causes the data processor to operate the above method.
- the computer program can embodied in a recording medium of any suitable form.
- FIG. 2 forms a system 20 which exploits the possibility of using a same source of sound, whether it be of natural origin as for a human speaker S or a source of virtual voice data 22, from which to produce utterances having different possible ages.
- the original voice signal 24 e.g. as picked up by a microphone is supplied to a standard digitizer 26 which is run by a sampling clock 28 for a first sampling frequency F1 (Hz).
- F1 a first sampling frequency
- the first sampling frequency can be typically 16 000 or 32 000 Hz, or any other usual sampling frequency.
- the digitizer 26 thereby produces a number F1 of sampled amplitude values of the voice signal per second, each sampled value being expressed as a number by a binary word of a given bit length.
- the digitizer For an inputted voice signal 24 lasting for a time interval t, the digitizer produces a time-ordered sequence of t.F1 amplitude values, each giving numerically the instantaneous amplitude of the signal 24 at the moment when it was sampled, and referred to hereafter as a digital sample value.
- the digital sample values produced by the digitizer 26 are compiled into one or several sound files 30, each containing a digital recording of the voice signal V.
- the speaker has an age A, whereupon the voice recorded in the sound file(s) will have an ascribed age A.
- the sound file(s) 30 is/are stored in a sound file memory 32 from which their data can be selectively accessed for reading out by a player.
- the sound file memory 32 is accessed by a sound data processor 34 which uses the voice sample data to generate utterance data UD, from which a desired utterance is subsequently expressed audibly.
- the utterance data UD may correspond to a complete pronounced word or a sequence of words already contained in a sound file 30, whereupon the processor 34 has the task of identifying the required word(s) and reading the corresponding digital samples.
- the processor 34 may also operate in a concatenative mode, in which it generates words or phrases by selecting and concatenating sound elements such as syllables or phonemes contained in a sound file. In this case, it selectively reads out the corresponding groups of digital samples to form the concatenation required for the utterance data UD to be reproduced audibly.
- the sound data processor 34 generates the utterance data UD in response to commands specifying the utterance to be produced, the latter coming e.g. from the central control system of a robotic pet or the like, as a function of its operating context.
- the description of the operation that follows is based on an example of a real-time access from the sound file memory 32 to produce the audible age-modified voice on the fly.
- the operation is given with reference to a portion of utterance data UD having a sampling duration R1 at a sampling frequency equal to F1.
- the sampling duration R1 would simply be equal to t.
- an utterance data portion UD undergoes a time adjustment with reference to a second frequency F2 (Hz).
- the time adjustment is such that the sampling duration of the time-adjusted form of utterance data portion, hereafter designated UDta, at that second frequency F2 is substantially the same as the sampling duration R1 of the original utterance data portion UD at the sampling frequency F1.
- this expedient has the remarkable effect of changing the age A of the voice as a function of this second frequency F2 when playing the time-adjusted form of utterance data UDta at a frequency at or in the region of the second frequency F2.
- the second frequency F2 is produced by a continuously variable frequency generator 36 which can be electronically controlled to generate a range of frequency values for F2 above and below the initial frequency F1, respectively to rejuvenate or age the voice conveyed by the utterance data.
- the variable frequency generator delivers a sampling clock signal at frequency F2 to the processor 34 for managing the data flow rate accordingly, and to a sound player 40 to provide the play sampling frequency F2.
- the time-adjustment is produced by a pitch-synchronous overlap and add - generally referred to in the literature by the acronym "PSOLA" - algorithm unit 38.
- PSOLA pitch-synchronous overlap and add - generally referred to in the literature by the acronym "PSOLA" - algorithm unit 38.
- PSOLA algorithm unit 38 is operative to perform on the incoming utterance data portion UD a PSOLA synthesis in which e.g. the following steps are performed:
- PSOLA More information on PSOLA algorithms can be found in the following web page: http://www.fon.let.uva.nl/praat/manual/PSOLA.html (copyright ppgb, March 30 th , 2001).
- the PSOLA algorithm unit 38 thus judiciously produces an appropriate number of samples for the time adjusted utterance data UDta so that a time-adjusted utterance data portion corresponding to the voice signal 24, for instance, will have the same duration t when played at the sampling frequency of F2 (age modified signal 24' at the bottom of figure 2).
- the PSOLA algorithm unit 38 receives as input parameters the values of F1 and F2 or information expressing the ratio of these two values in order to calculate the required adjustment.
- a PSOLA algorithm unit is - to the contrary - to modify the sampling duration of a given sound sequence so that it becomes contracted or expanded when played at the same frequency as its digitisation sampling frequency. In other words, it serves to accelerate or slow down the rate of playing through a digital sound sequence. The steps performed to provide this adjustment allows the changed playing rate to maintain nevertheless the same pitch characteristics as the original sound.
- a typical application in the art of a PSOLA algorithm is in the field of voice synthesis, where it serves to modify the delivery rate of speech (slow /fast talking speed) without for so much altering the pitch of the voice.
- the PSOLA algorithm unit 38 is used instead to maintain an initial duration of a given voice portion despite a change in the play sampling frequency F2 relative to the sampling frequency F1 for which the sound data was initially prepared.
- the sampling duration of the utterance data UD at the second frequency F2 would be t.F1/F2 without the above time-adjustment.
- the second frequency F2 is less than F1 (F2 ⁇ F1)
- the sampling duration of the utterance data UD would be expanded, while it will be contracted if the frequency F2 were greater than F1 (F2>F1).
- the time-adjusted form UDta of the sound data portion illustrated just below produced the PSOLA algorithm unit 38 has the same sampling duration t for the second frequency F2.
- the time-adjusted utterance data UDta outputted by the PSOLA algorithm unit 38 is fed to a digital sound data player 40 where it is digital-to-analog converted into an audible form through a loudspeaker 42.
- the selection of the apparent age to give to a voice can thus be achieved by correspondingly adjusting the frequency F2.
- this adjustment is effected through a user-accessible selector 44 which presents a visible scale 46 along which can be displaced a moveable cursor 48.
- the scale 46 spans an age range from very young to very old respectively at upper and lower limits of the frequency F2 for a range which contains the first frequency value F1.
- the position of the cursor 48 is arranged to select a corresponding value of the frequency F2.
- the thus-designated value of F2 is expressed as appropriate frequency-setting data for the frequency selection input of the variable frequency generator 36.
- This frequency setting data is also sent to the PSOLA algorithm unit 38 so that it can take that parameter into account accordingly.
- the scale and cursor 46, 48 can be material devices, such as a potentiometer or similar variable control device with graduations, or it can be a virtual device presented on a display screen, the cursor being displaceable by a mouse, trackball, or designated by a pushbutton etc.
- the frequency-setting data can also be generated without user intervention in the course of the execution of a general management program. For instance, if applied to a robotic pet having a pre-ordained life cycle, the part of its program that governs its ageing process will regularly update the F2 frequency-setting data by gradually decreasing its value to make the voice correlate with the simulated age.
- the implementation of the system 20 can also allow the sound processing to be performed in advance, rather than on the fly as described above.
- the sound processor 34 can be set to produce utterance data (UD) for later use, the data being stored in a memory which is read out and fed to the PSOLA algorithm unit 38 as and when required to be rendered audible.
- the outputted time-adjusted utterance data UDta from the PSOLA algorithm unit 38 can be stored in a memory and supplied to the digital sound player as and when required.
- the operation of the system 20 when handling virtual voice data is analogous in all respects.
- the virtual voice data are produced entirely by a voice generating algorithm according to a programmed parameterisation (which may include the possibility of adding an emotional content to the voice).
- the voice can again be made to sound human or animal-like, or may be invented sounds e.g. to correspond to the "personality" of a robotic pet.
- Such an algorithm typically has a parameter for establishing a prescribed playing frequency, i.e. the frequency at which the digital sound elements are to be fed through a digital sound player to sound as intended.
- the "intended" sound is the voice of a prescribed age (e.g. A as above) and the prescribed frequency shall also be designated F1.
- This frequency is supplied as data to the sound data processor 34 and to the PSOLA algorithm unit 38 for scaling as mentioned above.
- Figure 3 expresses the main operating steps of the system 20 for producing a voice with a variable age A.
- the process starts with a step E2 of preparing the voice data of a given age A for a sampling frequency F1, corresponding to a delivery rate R1.
- This data can be produced from the virtual voice data source 22 or from the digitiser 26 sampling a human voice or other analog sound source.
- the age of the voice to be produced from this voice data is selected at a step E4, the age being specified relative to the initial age A through a user input (cf. pointer 48 and scale 46), or from a program output.
- the voice data is processed at step E8 to produce utterance data UD.
- the utterance data UD is submitted to a PSOLA algorithm or the like at step E10 so as to produce time-adjusted utterance data UDta, which yields the sampling duration R1 when played at a play sampling frequency of F2.
- This time-adjusted utterance data are then played e.g. at the frequency F2 to produce a voice of variable age, as follows : F2 ⁇ F1 gives an age > A, and F2>F1 gives and age ⁇ A.
- the variability in the age of the voice exhibits a remarkably smooth and accurate transition with a corresponding change in the frequency F2.
- the latter can thus advantageously be made substantially continuously variable to give a similarly continuous change in the age of the voice, the rate of change being determined by the application.
- the system 20 can for instance be incorporated in a device which simulates a life cycle, in which case the second frequency F2 is made to decrease with time over the life cycle.
- the voice can be made to sound younger than the age A of the speaker, while it can be made to sound older than that age A towards the end of the life cycle, by allowing the second frequency to take values in a range which contains the initial sampling frequency F1.
- the invention finds many different technical applications in robotic pets, age simulation systems for allowing operators and users of talking computerised systems to choose an appropriate age of voice, for creating animated characters of different ages in studio productions, etc., for educational training, for computerised reading of texts etc.
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Toys (AREA)
Abstract
- producing from the sound data at least one sound data portion (UD) containing a sound portion to be reproduced with a modified value of the characteristic, the sound data portion having a first sampling duration R1 at the first sampling frequency F1,
- establishing (36, 44-48) a second frequency F2 selected as a function of a desired modification in the value of the characteristic,
- time adjusting (38) the sound data portion (UD) by elaborating a time-adjusted form (UDta) thereof on the basis of a sampling frequency equal to the second frequency F2, the time-adjusting step at least partially tending to cause the sampling duration of the time-adjusted form of sound data portion at the second frequency F2 to match the first sampling duration R1, and
- playing the time adjusted sound data portion (UDta).
Description
- The invention relates to a method and device for processing a sound such as a voice delivering a natural or synthesised utterance, so as to controllably modify a characteristic feature thereof, and notably its apparent age. As is well known, the sound characteristics of the human voice, for instance, evolves significantly with age, from infancy through childhood, youth, adulthood, and advancing years. Similar evolutions in the sounding of utterances can be observed with animals. This makes it possible to associate an age to a voice, simply on the basis of everyday experience. In other words, an utterance - be it natural or synthesised - conveys an impression of age, hereafter referred to generically as the "age" of the utterance.
- There is a growing need for technical means that can produce or reproduce utterances with a controllable age. For instance, robotic pets, humanoids and the communicating objects can be expected to simulate a lifespan, e.g. starting from infancy at an initial starting phase. In this case, it is desirable to make the age of their utterances seem coherent with their simulated age as they evolve through their lifespan. Other applications are, inter alia, in voice synthesis where it is often desirable to control the age of a voice among other parameters such as male/female, speed of delivery, emotion portrayed, etc.
- Figure 1 illustrates what would be a standard approach for providing a control in the age of an utterance where the latter is produced from samples of a human voice. The samples are obtained from a number N of different human speakers S1, S2, S3, ..., SN, each having a respective age A1, A2, A3, ..., AN corresponding to a given age group, e.g. from a child to an elderly person. The voices V1, V2, V3, ..., VN obtained from each of the speakers are processed independently of each other. For a particular speaker, the analog voice output is fed to a
digitizer 2 where it is converted to successive intensity or amplitude values. Thedigitizer 2 operates according to a given sampling frequency F1 (Hz) provided by a signal from asampling clock 4 set for that frequency. The sampling frequency corresponds to the number (F1) of intensity or amplitude values that are sampled i.e. measured, per second on the voice signal. Each sampled value is converted into a digital word of a given number of bits, depending on the required dynamic range. The successive digital words corresponding to respective successive samples are compiled into asound file 6 for reading by a digital sound player, usually a synthesiser which produces a digital-to-analog conversion on the sound data.Sound files 6 for respective speakers S1-SN are collected and stored in asound file memory 8. A sound file may contain complete words, or it may result from intermediate processing steps in order to correspond to a collection of basic sound elements extracted from the voice information, allowing words to be formed by so-called concatenative techniques. - Possible sampling frequency values for F1 are generally standardised. A low sampling frequency is more economical on memory requirements while a higher sampling frequency gives better performance. As an indication, a typical sampling frequency value for the
digitizer 2 is F1 = 16 000 Hz or F1 = 32 000 Hz. The sampling frequency for a givensound file 6 is indicated at a particular part of the file contents, or known implicitly from the protocol used. - The digital data of a
sound file 6 is reproduced by adigital sound player 10, which converts the digitally-expressed intensity or amplitude values as it receives them into a corresponding time-varying audio signal suitably amplified to be played on aspeaker 12 or other transducer. In the example, a chosen sound file is selected from thememory 8 by asound file reader 14, which serves to determine and organise the digital data to be sent to thedigital sound player 10. For instance, thesound file reader 14 can read selected portions of asound file 6 to obtain a required concatenation of syllables for producing an utterance. - The sampled data contained in the sound files are passed onto the digital sound player at the frequency F1 at which they were sampled, so that the delivery rate of the initially-recorded voice is respected; i.e. the playback speed is the same as the recording speed, as per a normal recording and playback system, the speed being in this case set by the sampling frequency F1 which fixes the delivery rate.
- To output an utterance having a prescribed age, the data supplied to the
digital sound player 10 will come from a sound file - or possibly a set of sound files - originating specifically from a voice Vi of a speaker Si having that prescribed age. - Thus, to change the age of the utterance, the
sound file reader 14 has to access one or more corresponding different sound files from thesound file memory 8. - Such an approach to the problem of providing a range of ages to utterances suffers from serious drawbacks, among which are :
- it is not practical to have the services of speakers of different age groups producing voice samples that are coherent in content and delivery;
- each age group to be covered by the range requires its own
specific sound file or set of sound files, which imposes a burden on the
sound file memory 8 and management units, such as thesound file reader 14; - it does not allow for a continuous variation or choice in the age of the utterance, i.e. it will produce unwanted jumps when going from one age group to the next;
- similarly, it does not allow to use a same speaker for the source of reproduced utterances, which also produces the undesirable impression that the utterances do not belong to the same vocalising object when going from one age group to the next.
- In view of the foregoing, the invention proposes a method of controllably modifying a value of a characteristic associated to a sound contained in sound data to which is associated a sampling frequency, the sound exhibiting a given value of the characteristic for a first sampling frequency F1, characterised in that it comprises the steps of:
- producing from the sound data at least one sound data portion containing a sound portion to be reproduced with a modified value of the characteristic, the sound data portion having a first sampling duration R1 at the first sampling frequency F1,
- establishing a second frequency F2 selected as a function of a desired modification in the value of the characteristic,
- time adjusting the sound data portion by elaborating a time-adjusted form thereof on the basis of a sampling frequency equal to the second frequency F2, the time-adjusting step at least partially tending to cause the sampling duration of the time-adjusted form of sound data portion at the second frequency F2 to match the first sampling duration R1, and
- playing the time adjusted sound data portion.
- The above process is particularly advantageous for controllably modifying an age associated to a voice, the sound data corresponding to at least one utterance, the value of the characteristic being an age associated to the utterance.
- The time-adjusting step is preferably performed by means of a Pitch-Synchronous Overlap and Add (PSOLA) algorithm or the like.
- The sound data can be obtained by sampling an analog sound signal at the first sampling frequency F1.
- All the sound data can obtained by sampling a voice from a same speaker.
- The sound data may also be obtained by generating synthetically values of sound data such as to exhibit the given value of the characteristic for the first sampling frequency F1.
- In the preferred embodiment, the time adjusting step causes the sampling duration of the time-adjusted form of the sound data portion sampled at the second frequency F2 to be substantially the same as the first sampling duration R1.
- The time-adjusted sound data can be played substantially at the second frequency F2.
- The second frequency F2 can be substantially continuously variable, e.g. over a range covering frequency values above and below the first frequency F1.
- When applied to controlling an age of a voice, the latter can be controllably increased by correspondingly decreasing the value of the second frequency F2.
- According to a second aspect, the invention relates to the implementation of the above method in a device which simulates an ageing process, such as a robotic pet having a life-cycle, wherein the second frequency F2 is caused to decrease progressively with time.
- According to a third aspect, the invention provides a device for controllably modifying a value of a characteristic associated to a sound contained in sound data to which is associated a sampling frequency, the sound exhibiting a given value of the characteristic for a first sampling frequency F1, characterised in that it comprises:
- means for producing from the sound data at least one sound data portion containing a sound portion to be reproduced with a modified value of the characteristic, the sound data portion having a first sampling duration R1 at the first sampling frequency F1,
- means for establishing a second frequency F2 selected as a function of a desired modification in the value of the characteristic,
- means for time adjusting the sound data portion by elaborating a time-adjusted form thereof on the basis of a sampling frequency equal to the second frequency F2, the time adjusting means at least partially tending to cause the sampling duration of the time-adjusted form of sound data portion at the second frequency F2 to match the first sampling duration R1, and
- means for playing the time adjusted sound data portion.
- The optional aspects of the invention presented in the context of the above method (first aspect) can apply mutatis mutandis to the apparatus according to the third aspect, and shall not be repeated for conciseness.
- According to a fourth aspect, the invention relates to a device which simulates an ageing process, such as a robotic pet having a life-cycle, characterised in that it comprises a device according to the third aspect and means for causing the second frequency F2 to decrease progressively with time.
- According to a fifth aspect, the invention relates to a computer program providing computer executable instructions, which when loaded onto a data processor causes the data processor to operate the above method. The computer program can embodied in a recording medium of any suitable form.
- The invention and its advantages shall become more apparent from reading the following description of the preferred embodiments, given purely as non-limiting examples with reference to the appended figures in which:
- figure 1, already described, is a block diagram showing a possible approach for producing utterances of different ages, relying on standard sound sampling and sound playing procedures and the use of several speakers of respective age groups;
- figure 2 is a schematic representation of the preferred embodiment of the invention and of signals it processes at different stages, applied for controllably modifying an age associated to a voice; and
- figure 3 is a flow chart indicating process steps performed by the apparatus of figure 2.
- The embodiment of the invention illustrated in figure 2 forms a
system 20 which exploits the possibility of using a same source of sound, whether it be of natural origin as for a human speaker S or a source ofvirtual voice data 22, from which to produce utterances having different possible ages. - If the sounds at the origin come from a human speaker S, the
original voice signal 24 e.g. as picked up by a microphone is supplied to astandard digitizer 26 which is run by asampling clock 28 for a first sampling frequency F1 (Hz). As in the case of figure 1, the first sampling frequency can be typically 16 000 or 32 000 Hz, or any other usual sampling frequency. Thedigitizer 26 thereby produces a number F1 of sampled amplitude values of the voice signal per second, each sampled value being expressed as a number by a binary word of a given bit length. Thus, for an inputtedvoice signal 24 lasting for a time interval t, the digitizer produces a time-ordered sequence of t.F1 amplitude values, each giving numerically the instantaneous amplitude of thesignal 24 at the moment when it was sampled, and referred to hereafter as a digital sample value. - The digital sample values produced by the
digitizer 26 are compiled into one or several sound files 30, each containing a digital recording of the voice signal V. In the example, the speaker has an age A, whereupon the voice recorded in the sound file(s) will have an ascribed age A. - The sound file(s) 30 is/are stored in a
sound file memory 32 from which their data can be selectively accessed for reading out by a player. - In the example, the
sound file memory 32 is accessed by asound data processor 34 which uses the voice sample data to generate utterance data UD, from which a desired utterance is subsequently expressed audibly. The utterance data UD may correspond to a complete pronounced word or a sequence of words already contained in asound file 30, whereupon theprocessor 34 has the task of identifying the required word(s) and reading the corresponding digital samples. Theprocessor 34 may also operate in a concatenative mode, in which it generates words or phrases by selecting and concatenating sound elements such as syllables or phonemes contained in a sound file. In this case, it selectively reads out the corresponding groups of digital samples to form the concatenation required for the utterance data UD to be reproduced audibly. - Other forms of processing can be envisaged to generate the required utterance data UD.
- In the embodiment, the
sound data processor 34 generates the utterance data UD in response to commands specifying the utterance to be produced, the latter coming e.g. from the central control system of a robotic pet or the like, as a function of its operating context. - For ease of understanding, the description of the operation that follows is based on an example of a real-time access from the
sound file memory 32 to produce the audible age-modified voice on the fly. The operation is given with reference to a portion of utterance data UD having a sampling duration R1 at a sampling frequency equal to F1. Thus, in the case of the utterance data portion UD corresponding to thevoice signal 24 of duration t, the sampling duration R1 would simply be equal to t. - Before being played, an utterance data portion UD undergoes a time adjustment with reference to a second frequency F2 (Hz). The time adjustment is such that the sampling duration of the time-adjusted form of utterance data portion, hereafter designated UDta, at that second frequency F2 is substantially the same as the sampling duration R1 of the original utterance data portion UD at the sampling frequency F1.
- As discovered by the Applicant, this expedient has the remarkable effect of changing the age A of the voice as a function of this second frequency F2 when playing the time-adjusted form of utterance data UDta at a frequency at or in the region of the second frequency F2. The second frequency F2 is produced by a continuously
variable frequency generator 36 which can be electronically controlled to generate a range of frequency values for F2 above and below the initial frequency F1, respectively to rejuvenate or age the voice conveyed by the utterance data. The variable frequency generator delivers a sampling clock signal at frequency F2 to theprocessor 34 for managing the data flow rate accordingly, and to asound player 40 to provide the play sampling frequency F2. - The time-adjustment is produced by a pitch-synchronous overlap and add - generally referred to in the literature by the acronym "PSOLA" -
algorithm unit 38. Such algorithms are in themselves well known in the art. Basically, thePSOLA algorithm unit 38 is operative to perform on the incoming utterance data portion UD a PSOLA synthesis in which e.g. the following steps are performed: - i) from the pitch tier, new points are generated along the entire time domain, with the method of pitch tier: to point process;
- ii) the period information in the pulses is used to remove from the new pulses all points that lie within the voiceless intervals (i.e. places where the distance between adjacent points in the original pulses is greater than a specified limit, e.g. 20 ms;
- iii) the voiceless parts are copied from the source sound to the target sound, re-using some parts if the local duration is greater than 1;
- iv) for each target point, look up the nearest source point. A piece of source sound, centred around the source point, is copied to the target sound at a location determined by the target point, using a bell-shaped window whose left-hand half length is the minimum of the left-hand periods adjacent to the source and target points (and analogously for the right-hand half-length.
-
- More information on PSOLA algorithms can be found in the following web page: http://www.fon.let.uva.nl/praat/manual/PSOLA.html (copyright ppgb, March 30th, 2001).
- The
PSOLA algorithm unit 38 thus judiciously produces an appropriate number of samples for the time adjusted utterance data UDta so that a time-adjusted utterance data portion corresponding to thevoice signal 24, for instance, will have the same duration t when played at the sampling frequency of F2 (age modified signal 24' at the bottom of figure 2). - The
PSOLA algorithm unit 38 receives as input parameters the values of F1 and F2 or information expressing the ratio of these two values in order to calculate the required adjustment. - It is recalled that the normal function of a PSOLA algorithm unit is - to the contrary - to modify the sampling duration of a given sound sequence so that it becomes contracted or expanded when played at the same frequency as its digitisation sampling frequency. In other words, it serves to accelerate or slow down the rate of playing through a digital sound sequence. The steps performed to provide this adjustment allows the changed playing rate to maintain nevertheless the same pitch characteristics as the original sound. A typical application in the art of a PSOLA algorithm is in the field of voice synthesis, where it serves to modify the delivery rate of speech (slow /fast talking speed) without for so much altering the pitch of the voice.
- In the embodiment, the
PSOLA algorithm unit 38 is used instead to maintain an initial duration of a given voice portion despite a change in the play sampling frequency F2 relative to the sampling frequency F1 for which the sound data was initially prepared. - To illustrate this concept, the right-hand part of figure 2 illustrates the case of an item of utterance data UD composed of N = t.F1 digital samples, e.g. corresponding to the
voice signal 24 whose original duration is t. The sampling duration of the utterance data UD at the second frequency F2 would be t.F1/F2 without the above time-adjustment. Thus, if the second frequency F2 is less than F1 (F2<F1), the sampling duration of the utterance data UD would be expanded, while it will be contracted if the frequency F2 were greater than F1 (F2>F1). The time-adjusted form UDta of the sound data portion illustrated just below produced thePSOLA algorithm unit 38 has the same sampling duration t for the second frequency F2. - The time-adjusted utterance data UDta outputted by the
PSOLA algorithm unit 38 is fed to a digitalsound data player 40 where it is digital-to-analog converted into an audible form through aloudspeaker 42. - Experimentation by the Applicant shows that:
- the voice is aged by making the second frequency F2 smaller than the digitisation frequency F1, the amount of ageing increasing continuously as the second frequency is decreased, and
- the voice is rejuvenated (made younger sounding) by making the second frequency F2 greater than the digitisation frequency F1, the amount of rejuvenation increasing continuously as the second frequency is increased.
- The selection of the apparent age to give to a voice can thus be achieved by correspondingly adjusting the frequency F2. In the embodiment, this adjustment is effected through a user-
accessible selector 44 which presents avisible scale 46 along which can be displaced amoveable cursor 48. Thescale 46 spans an age range from very young to very old respectively at upper and lower limits of the frequency F2 for a range which contains the first frequency value F1. The position of thecursor 48 is arranged to select a corresponding value of the frequency F2. The thus-designated value of F2 is expressed as appropriate frequency-setting data for the frequency selection input of thevariable frequency generator 36. This frequency setting data is also sent to thePSOLA algorithm unit 38 so that it can take that parameter into account accordingly. - The scale and
cursor - The frequency-setting data can also be generated without user intervention in the course of the execution of a general management program. For instance, if applied to a robotic pet having a pre-ordained life cycle, the part of its program that governs its ageing process will regularly update the F2 frequency-setting data by gradually decreasing its value to make the voice correlate with the simulated age.
- It can be appreciated that the implementation of the
system 20 can also allow the sound processing to be performed in advance, rather than on the fly as described above. In this case, thesound processor 34 can be set to produce utterance data (UD) for later use, the data being stored in a memory which is read out and fed to thePSOLA algorithm unit 38 as and when required to be rendered audible. Similarly, the outputted time-adjusted utterance data UDta from thePSOLA algorithm unit 38 can be stored in a memory and supplied to the digital sound player as and when required. - The operation of the
system 20 when handling virtual voice data (source 22) is analogous in all respects. In this case, the virtual voice data are produced entirely by a voice generating algorithm according to a programmed parameterisation (which may include the possibility of adding an emotional content to the voice). The voice can again be made to sound human or animal-like, or may be invented sounds e.g. to correspond to the "personality" of a robotic pet. Such an algorithm typically has a parameter for establishing a prescribed playing frequency, i.e. the frequency at which the digital sound elements are to be fed through a digital sound player to sound as intended. Here, the "intended" sound is the voice of a prescribed age (e.g. A as above) and the prescribed frequency shall also be designated F1. This frequency is supplied as data to thesound data processor 34 and to thePSOLA algorithm unit 38 for scaling as mentioned above. - Thus, it is always possible to identify an original (first) sampling frequency F1 for producing a voice of a given age, and from which a different play sampling frequency can be used by the
system 20 to modify that age. - It can be noted that there is some latitude in the play sampling frequency actually used at the level of
sound player 40 itself: while that frequency is simply the frequency F2 used for producing the age variation upstream and gives excellent results, departures from F2 for playing the time-adjusted utterance data can be envisaged e.g. for changing the delivery rate to a certain extent, while still producing a convincingly modified age to the voice. - Figure 3 expresses the main operating steps of the
system 20 for producing a voice with a variable age A. - The process starts with a step E2 of preparing the voice data of a given age A for a sampling frequency F1, corresponding to a delivery rate R1. This data can be produced from the virtual
voice data source 22 or from thedigitiser 26 sampling a human voice or other analog sound source. - The age of the voice to be produced from this voice data is selected at a step E4, the age being specified relative to the initial age A through a user input (cf.
pointer 48 and scale 46), or from a program output. - From this selection is produced at step E6 a corresponding value of the second frequency F2.
- On the basis of the second frequency F2, the voice data is processed at step E8 to produce utterance data UD.
- The utterance data UD is submitted to a PSOLA algorithm or the like at step E10 so as to produce time-adjusted utterance data UDta, which yields the sampling duration R1 when played at a play sampling frequency of F2.
- This time-adjusted utterance data are then played e.g. at the frequency F2 to produce a voice of variable age, as follows : F2<F1 gives an age > A, and F2>F1 gives and age < A.
- The variability in the age of the voice exhibits a remarkably smooth and accurate transition with a corresponding change in the frequency F2. The latter can thus advantageously be made substantially continuously variable to give a similarly continuous change in the age of the voice, the rate of change being determined by the application. The
system 20 can for instance be incorporated in a device which simulates a life cycle, in which case the second frequency F2 is made to decrease with time over the life cycle. At the initial phase, the voice can be made to sound younger than the age A of the speaker, while it can be made to sound older than that age A towards the end of the life cycle, by allowing the second frequency to take values in a range which contains the initial sampling frequency F1. - The invention finds many different technical applications in robotic pets, age simulation systems for allowing operators and users of talking computerised systems to choose an appropriate age of voice, for creating animated characters of different ages in studio productions, etc., for educational training, for computerised reading of texts etc.
Claims (24)
- A method of controllably modifying a value of a characteristic associated to a sound contained in sound data (30) to which is associated a sampling frequency, the sound exhibiting a given value (A) of the characteristic for a first sampling frequency F1, characterised in that it comprises the steps of:producing from the sound data at least one sound data portion (UD) containing a sound portion to be reproduced with a modified value of the characteristic, the sound data portion having a first sampling duration R1 at the first sampling frequency F1,establishing (36, 44-48) a second frequency F2 selected as a function of a desired modification in the value of the characteristic,time adjusting (38) the sound data portion (UD) by elaborating a time-adjusted form (UDta) thereof on the basis of a sampling frequency equal to the second frequency F2, the time-adjusting step at least partially tending to cause the sampling duration of the time-adjusted form of sound data portion at the second frequency F2 to match the first sampling duration R1, andplaying the time adjusted sound data portion (UDta).
- Method according to claim 1, used for controllably modifying an age (A) associated to a voice, the sound data (30) corresponding to at least one utterance, the value of the characteristic being an age associated to the utterance.
- Method according to any one of claims 1 or 2, wherein the time-adjusting step is performed by means of a Pitch-Synchronous Overlap and Add (PSOLA) algorithm (38) or the like.
- Method according to any one of claims 1 to 3, wherein the sound data is obtained by sampling (26) an analog sound signal (24) at the first sampling frequency F1.
- Method according to claim 4, wherein all the sound data are obtained by sampling a voice from a same speaker S.
- Method according to any one of claims 1 to 3, wherein the sound data are obtained by generating synthetically values of sound data such as to exhibit the given value (A) of the characteristic for the first sampling frequency F1.
- Method according to any one of claims 1 to 6, wherein the time adjusting step causes the sampling duration of the time-adjusted form (UDta) of the sound data portion sampled at the second frequency F2 to be substantially the same as the first sampling duration R1.
- Method according to any one of claims 1 to 7, wherein the time-adjusted sound data (UDta) are played substantially at the second frequency F2.
- Method according to any one of claims 1 to 8, wherein the second frequency F2 is substantially continuously variable.
- Method according to any one of claims 1 to 9, wherein the second frequency F2 is variable over a range covering frequency values above and below the first frequency F1.
- Method according to any one of claims 2 to 10, wherein the age of the voice is controllably increased by correspondingly decreasing the value of the second frequency F2.
- Method according to any one of claims 2 to 11, implemented in a device which simulates an ageing process, such as a robotic pet having a life-cycle, wherein the second frequency F2 is caused to decrease progressively with time.
- Device (20) for controllably modifying a value of a characteristic associated to a sound contained in sound data (30) to which is associated a sampling frequency, the sound exhibiting a given value (A) of the characteristic for a first sampling frequency F1, characterised in that it comprises:means (32, 34) for producing from the sound data at least one sound data portion (UD) containing a sound portion to be reproduced with a modified value of the characteristic, the sound data portion having a first sampling duration R1 at the first sampling frequency F1,means (36, 44-48) for establishing a second frequency F2 selected as a function of a desired modification in the value of the characteristic,means (38) for time adjusting the sound data portion (UD) by elaborating a time-adjusted form (UDta) thereof on the basis of a sampling frequency equal to the second frequency F2, the time adjusting means (38) at least partially tending to cause the sampling duration of the time-adjusted form of sound data portion at the second frequency F2 to match the first sampling duration R1, andmeans (40, 42) for playing the time adjusted sound data portion (UDta).
- Device according to claim 13 configured for controllably modifying an age (A) associated to a voice, the sound data (30) corresponding to at least one utterance, the value of the characteristic being an age of associated to the utterance.
- Device according to claim 13 or 14, wherein the time-adjusting means comprises a Pitch-Synchronous Overlap and Add (PSOLA) algorithm unit (38) or the like.
- Device according to any one of claims 13 to 15, comprising means (26) for sampling an analog sound signal (24) at the first sampling frequency F1, thereby to produce sound data (30).
- Device according to claim 16, wherein all the sound data (30) consist of samples of a voice from a same speaker S.
- Device according to any one of claims 13 to 15, comprising means (22) for generating synthetically values forming the sound data (30) such as to exhibit the given value (A) of the characteristic for the first sampling frequency F1.
- Device according to any one of claims 13 to 18, wherein the time adjusting means (38) causes the sampling duration of the time-adjusted form (UDta) of the sound data portion sampled at the second frequency F2 to be substantially the same as the first sampling duration R1.
- Device according to any one of claims 13 to 19, wherein the playing means (40, 42) are set to play the time-adjusted sound data (UDta) substantially at the second frequency F2.
- Device according to any one of claims 13 to 20, wherein the second frequency F2 establishing means (36, 44-48) are controllable to produce substantially continuously variations of the second frequency F2.
- Device according to any one of claims 13 to 21, wherein the second frequency F2 establishing means (36, 44-48) are controllable to produce the second frequency F2 over a range covering frequency values above and below the first frequency F1.
- Device which simulates an ageing process, such as a robotic pet having a life-cycle, characterised in that it comprises a device according to any one of claims 14 to 22 and means for causing the second frequency F2 to decrease progressively with time.
- A computer program providing computer executable instructions, which when loaded onto a data processor causes the data processor to operate the method in accordance with any of claims 1 to 12.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP01402177A EP1286332A1 (en) | 2001-08-14 | 2001-08-14 | Sound processing method and device for modifying a sound characteristic, such as an impression of age associated to a voice |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP01402177A EP1286332A1 (en) | 2001-08-14 | 2001-08-14 | Sound processing method and device for modifying a sound characteristic, such as an impression of age associated to a voice |
Publications (1)
Publication Number | Publication Date |
---|---|
EP1286332A1 true EP1286332A1 (en) | 2003-02-26 |
Family
ID=8182853
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP01402177A Withdrawn EP1286332A1 (en) | 2001-08-14 | 2001-08-14 | Sound processing method and device for modifying a sound characteristic, such as an impression of age associated to a voice |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP1286332A1 (en) |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0688010A1 (en) * | 1994-06-16 | 1995-12-20 | Canon Kabushiki Kaisha | Speech synthesis method and speech synthesizer |
JPH10133852A (en) * | 1996-10-31 | 1998-05-22 | Toshiba Corp | Personal computer, and method for managing voice attribute parameter |
-
2001
- 2001-08-14 EP EP01402177A patent/EP1286332A1/en not_active Withdrawn
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0688010A1 (en) * | 1994-06-16 | 1995-12-20 | Canon Kabushiki Kaisha | Speech synthesis method and speech synthesizer |
JPH10133852A (en) * | 1996-10-31 | 1998-05-22 | Toshiba Corp | Personal computer, and method for managing voice attribute parameter |
Non-Patent Citations (3)
Title |
---|
MOULINES E ET AL: "Non-parametric techniques for pitch-scale and time-scale modification of speech", SPEECH COMMUNICATION, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 16, no. 2, 1 February 1995 (1995-02-01), pages 175 - 205, XP004024959, ISSN: 0167-6393 * |
TITZE I ET AL: "Considerations in voice transformation with physiologic scaling principles", SPEECH COMMUNICATION, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 22, no. 2-3, 1 August 1997 (1997-08-01), pages 113 - 123, XP004100286, ISSN: 0167-6393 * |
VELDHUIS R ET AL: "Time-scale and pitch modifications of speech signals and resynthesis from the discrete short-time Fourier transform", SPEECH COMMUNICATION, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 18, no. 3, 1 May 1996 (1996-05-01), pages 257 - 279, XP004018610, ISSN: 0167-6393 * |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US5890115A (en) | Speech synthesizer utilizing wavetable synthesis | |
JP6791258B2 (en) | Speech synthesis method, speech synthesizer and program | |
CN104412320B (en) | Use the automatic Playing technology of audio waveform data | |
JPH06102877A (en) | Acoustic constituting device | |
JP2008146094A (en) | Audio-intonation calibration method | |
US7432435B2 (en) | Tone synthesis apparatus and method | |
CN107430849A (en) | Sound control apparatus, audio control method and sound control program | |
KR20000005183A (en) | Image synthesizing method and apparatus | |
CN113016028B (en) | Sound processing method and sound processing system | |
US5659664A (en) | Speech synthesis with weighted parameters at phoneme boundaries | |
US7457752B2 (en) | Method and apparatus for controlling the operation of an emotion synthesizing device | |
EP1286332A1 (en) | Sound processing method and device for modifying a sound characteristic, such as an impression of age associated to a voice | |
JP5779838B2 (en) | Sound processing apparatus and program | |
CN115349147A (en) | Sound signal generation method, estimation model training method, sound signal generation system, and program | |
SE516521C2 (en) | Device and method of speech synthesis | |
JPH1115489A (en) | Singing sound synthesizing device | |
JP3233036B2 (en) | Singing sound synthesizer | |
WO2004027758A1 (en) | Method for controlling duration in speech synthesis | |
JPH09179576A (en) | Voice synthesizing method | |
JP6036903B2 (en) | Display control apparatus and display control method | |
JPH06250695A (en) | Method and device for pitch control | |
JP5471138B2 (en) | Phoneme code converter and speech synthesizer | |
JP5552797B2 (en) | Speech synthesis apparatus and speech synthesis method | |
JP6787491B2 (en) | Sound generator and method | |
JP3284634B2 (en) | Rule speech synthesizer |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK RO SI |
|
17P | Request for examination filed |
Effective date: 20030826 |
|
AKX | Designation fees paid |
Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR |
|
17Q | First examination report despatched |
Effective date: 20040203 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20041201 |