CN109309607A - Household appliance operation execution method and device, household appliance and readable storage medium - Google Patents
Household appliance operation execution method and device, household appliance and readable storage medium Download PDFInfo
- Publication number
- CN109309607A CN109309607A CN201811195101.3A CN201811195101A CN109309607A CN 109309607 A CN109309607 A CN 109309607A CN 201811195101 A CN201811195101 A CN 201811195101A CN 109309607 A CN109309607 A CN 109309607A
- Authority
- CN
- China
- Prior art keywords
- voice signal
- reference speech
- speech signals
- household appliance
- voice
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000009434 installation Methods 0.000 claims description 28
- 230000005236 sound signal Effects 0.000 claims description 20
- 238000004891 communication Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 11
- 238000010030 laminating Methods 0.000 claims description 9
- 230000001755 vocal effect Effects 0.000 claims 1
- 230000008569 process Effects 0.000 description 15
- 230000006870 function Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 238000004378 air conditioning Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000002452 interceptive effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 235000015170 shellfish Nutrition 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2803—Home automation networks
- H04L12/2816—Controlling appliance services of a home automation network by calling their functionalities
- H04L12/282—Controlling appliance services of a home automation network by calling their functionalities based on user interaction within the home
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
- G01S5/18—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using ultrasonic, sonic, or infrasonic waves
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
- G10L17/04—Training, enrolment or model building
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Acoustics & Sound (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Radar, Positioning & Navigation (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Automation & Control Theory (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Selective Calling Equipment (AREA)
Abstract
The invention discloses a method and a device for executing operation of household electrical appliance, the household electrical appliance and a readable storage medium, which are used for solving the problem that the household electrical appliance cannot accurately execute corresponding operation due to low accuracy of voice signal identification in the prior art. The method comprises the following steps: the household appliance collects voice signals through a plurality of microphones respectively, wherein each microphone is installed at different positions of the household appliance, and the number of the microphones is not less than three; determining the voice signal with the maximum decibel value as a reference voice signal according to the decibel value of the voice signal acquired by each microphone; and executing corresponding operation according to the reference voice signal. The voice signals are respectively collected through the plurality of microphones installed at different positions, the reference voice signal is selected based on the decibel value of the collected voice signal, and the voice signals which are relatively accurate can be collected no matter where the user is located, so that the accuracy of corresponding operation executed by the household appliance is ensured.
Description
Technical field
The present invention relates to voice processing technology fields more particularly to a kind of operation of household appliance to execute method, apparatus, household electrical appliances
Equipment and readable storage medium storing program for executing.
Background technique
With the continuous development of voice technology, more and more interactive voice equipment are come into being.By to household appliance
Voice control is carried out, household appliance can be allowed to complete the function such as playing music, play weather.The hair of following household appliance
Exhibition trend is artificial intelligence, and household appliance can be listened, can say, is the important development direction of the following human-computer interaction, wherein voice at
For the man-machine interaction mode being most expected, and voice is more more advantageous than other interactive modes.
Mostly the household appliance based on interactive voice is mostly near field voice identification now, and microphone is usually one
It is a, there may come a time when that two can be configured, the installation site of microphone is fixed, if on the corresponding direction of microphone installation site to
Household appliance sends voice signal, and household appliance is generally potentially based on the voice signal and is normally controlled, but user exists
Position when sending from voice signal to household appliance is uncontrollable, if microphone installation site is deviateed in the position where user
Corresponding direction, then the collected voice signal of microphone may be very weak, in some instances it may even be possible to the voice signal that can input user
As noise processed, cause voice signal recognition efficiency lower, to household appliance cannot be made accurately to execute corresponding operating.
Summary of the invention
It is operated the embodiment of the invention provides a kind of household appliance and executes method, apparatus, household appliance and readable storage medium
Matter, it is not high to solve voice signal recognition accuracy in the prior art, cause household appliance that can not accurately execute corresponding behaviour
The problem of making.
The embodiment of the invention provides a kind of household appliances to operate execution method, which comprises
Household appliance acquires voice signal by multiple microphones respectively, sets wherein each microphone is mounted on the household electrical appliances
Standby different location, and the quantity of the microphone is no less than three；
According to the decibel value of the voice signal of each microphone acquisition, on the basis of the maximum voice signal of decibel value is determined
Voice signal；
Respective operations are executed according to the reference speech signals.
Further, the method also includes:
According to the installation site for the microphone for acquiring the reference speech signals, the orientation of sound source is determined.
Further, the spacing of installation site of any two neighboring microphones in the household appliance is equal.
Further, before the execution respective operations according to the reference speech signals, the method also includes:
For each voice signal in addition to the reference speech signals, identify in the voice signal with the benchmark language
The matched targeted voice signal of sound signal, the targeted voice signal is superimposed upon in the reference speech signals.
Further, in described identification voice signal with the matched targeted voice signal packet of the reference speech signals
It includes:
The voice signal is handled, the characteristic of the voice signal is generated；
The reference speech signals are matched with the characteristic, corresponding to the characteristic by successful match
Sub- voice signal is determined as targeted voice signal.
Further, the characteristic comprises at least one of the following:
Voice voiceprint feature model data, voiced keyword data.
Further, described to match the reference speech signals with the characteristic, by the spy of successful match
Sub- voice signal corresponding to sign data is determined as targeted voice signal and includes:
If this feature data include voice voiceprint feature model data, by the reference speech signals and the voice sound
Line characteristic model data are matched, will be corresponding to the voice voiceprint feature model data in the voice signal with successful match
Sub- voice signal is determined as targeted voice signal；
If this feature data include voiced keyword data, by the reference speech signals and the voiced keyword number
According to being matched, sub- voice signal corresponding to the voiced keyword data in the voice signal with successful match is determined as mesh
Poster sound signal.
The embodiment of the invention provides a kind of household appliances to operate executive device, and described device includes:
Receiving module, for acquiring voice signal respectively by multiple microphones, wherein each microphone be mounted on it is described
The different location of household appliance, and the quantity of the microphone is no less than three；
First determining module, the decibel value of the voice signal for being acquired according to each microphone are maximum by decibel value
Voice signal is determined as reference speech signals；
Execution module, for executing respective operations according to the reference speech signals.
Further, described device further include:
Second determining module determines sound source for the installation site according to the microphone for acquiring the reference speech signals
Orientation.
Further, described device further include:
Laminating module, for successively for each voice signal in addition to the reference speech signals, identifying the voice
With the matched targeted voice signal of the reference speech signals in signal, the targeted voice signal is superimposed upon the benchmark language
In sound signal.
Further, the laminating module generates the spy of the voice signal specifically for handling the voice signal
Levy data；The reference speech signals are matched with the characteristic, corresponding to the characteristic by successful match
Sub- voice signal is determined as targeted voice signal.
Further, the laminating module, if including voice voiceprint feature model data specifically for this feature data,
The reference speech signals are matched with the voice voiceprint feature model data, by the voice signal with successful match
Voice voiceprint feature model data corresponding to sub- voice signal be determined as targeted voice signal；If this feature data include
Voiced keyword data match the reference speech signals with the voiced keyword data, will be in the voice signal
Sub- voice signal corresponding to voiced keyword data with successful match is determined as targeted voice signal.
The embodiment of the invention provides a kind of household appliances, comprising: processor, communication interface, memory and communication bus,
Wherein, processor, communication interface, memory complete mutual communication by communication bus；
It is stored with computer program in the memory, when described program is executed by the processor, so that the place
Manage the step of device executes any of the above-described the method.
The embodiment of the invention provides a kind of computer readable storage medium, it is stored with the meter that can be executed by household appliance
Calculation machine program, when described program is run in the household appliance, so that the household appliance executes described in any of the above-described
The step of method.
It is operated the embodiment of the invention provides a kind of household appliance and executes method, apparatus, household appliance and readable storage medium
Matter, this method comprises: household appliance acquires voice signal by multiple microphones respectively, wherein each microphone be mounted on it is described
The different location of household appliance, and the quantity of the microphone is no less than three；The voice signal acquired according to each microphone
Decibel value, the maximum voice signal of decibel value is determined as reference speech signals；According to reference speech signals execution pair
It should operate.Due in embodiments of the present invention, acquiring voice signal respectively by multiple microphones for being mounted on different location, and
Based on the decibel value of collected voice signal, select reference speech signals, it is ensured that no matter user where, can
Relatively accurate voice signal is collected, to ensure that household appliance executes the accuracy of corresponding operating.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly introduced, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill in field, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 is the flow diagram that a kind of household appliance that the embodiment of the present invention 1 provides operates execution method；
Fig. 2A is a kind of schematic view of the mounting position of the multi-microphone that provides of the embodiment of the present invention 2 in home appliance；
Fig. 2 B is that multiple microphones that the embodiment of the present invention 2 provides acquire voice signal and determine that the process of effective sound source is shown
It is intended to；
Fig. 3 is the structure chart that a kind of household appliance that the embodiment of the present invention 5 provides operates executive device；
Fig. 4 is a kind of structural schematic diagram for household appliance that the embodiment of the present invention 6 provides.
Specific embodiment
The present invention will be describe below in further detail with reference to the accompanying drawings, it is clear that described embodiment is only this
Invention a part of the embodiment, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art exist
All other embodiment obtained under the premise of creative work is not made, shall fall within the protection scope of the present invention.
Embodiment 1:
Fig. 1 is the flow diagram that a kind of household appliance provided in an embodiment of the present invention operates execution method, the process packet
Include following steps:
S101: household appliance acquires voice signal by multiple microphones respectively, wherein each microphone be mounted on it is described
The different location of household appliance, and the quantity of the microphone is no less than three.
The household appliance has the function of interactive voice, and the voice signal that can be specifically inputted according to user carries out corresponding
Operation, to realize corresponding function, such as ON/OFF household appliance, play the functions such as weather.Specifically, household appliance can
To be air-conditioning, TV, refrigerator, speaker etc..Such as the household appliance is air-conditioning, user can control sky by input speech signal
The functions such as the temperature of tune and switch air-conditioning.
Specifically, can execute user input voice signal household appliance in be provided with speech recognition module, the language
Sound identification module has speech signal collection and processing function.And the speech recognition module includes multiple microphones, Mei Gemai
The installation site of gram wind in household appliance is different, preferably, multiple microphone is mounted on each orientation of household appliance.Peace
Microphone mounted in each position acquires the voice signal of user's sending respectively, and the speech recognition module can be to collecting
Voice signal carry out certain processing, such as voice signal can be amplified, the processing such as filtering.
Specifically, the process of multiple microphones acquisition voice signal in speech recognition module belongs to the prior art, at this
In inventive embodiments, which is repeated no more.
S102: according to the decibel value of the voice signal of each microphone acquisition, the maximum voice signal of decibel value is determined
For benchmark voice signal.
Since the installation site of multiple microphones in home appliance is different, then multiple microphones are apart from the far and near of sound source
Difference, so that the decibel value of the collected voice signal of each microphone is different, and the size of decibel value can be from certain journey
Show distance of the microphone apart from sound source on degree.Therefore the maximum voice signal of decibel value may be considered nearest apart from sound source
, and the maximum voice signal of the decibel value may be considered collected clearest, the acoustic information for including in other words is most quasi-
True voice signal.
Specifically, handling first the collected voice signal of each microphone, point of each voice signal is obtained
Shellfish value；The decibel value of each voice signal is compared, the maximum voice signal of decibel value is obtained, by the maximum language of decibel value
Sound signal is determined as reference speech signals.The maximum reference speech signals of decibel value are what the microphone nearest apart from sound source acquired
Voice signal, household appliance are based on the reference speech signals and carry out respective operations, and the accuracy of operation may be implemented.
S103: respective operations are executed according to the reference speech signals.
By comparing the decibel value of the voice signal of each microphone acquisition, using the maximum voice signal of decibel value as base
Quasi- voice signal, the reference speech signals can consider the voice signal for being closest to the microphone acquisition of sound source, household appliance
Executing respective operations based on the reference speech signals may be implemented the accuracy of operation.Wherein, household appliance judges whether to need
The process that respective operations are executed according to the reference speech signals is the prior art, no longer superfluous to the process in embodiments of the present invention
It states.
Specifically, household appliance according to the reference speech signals execute respective operations include, to the reference speech signals into
Row analysis, specific includes being matched the reference speech signals with sound bank using big data, extracts effective voice
Signal executes corresponding operation for effective voice signal again, so that household appliance is realized corresponding function, wherein language
Sound library is usually to be learnt by big data, carries out the sound bank that voice training preserves to household appliance.
Wherein, household appliance is the prior art according to the process that the reference speech signals execute respective operations, in the present invention
To the process without repeating in embodiment.
Due in embodiments of the present invention, acquiring voice signal respectively by multiple microphones for being mounted on different location,
And the decibel value based on collected voice signal, select reference speech signals, it is ensured that no matter user where, all
Relatively accurate voice signal can be collected, to ensure that household appliance executes the accuracy of corresponding operating.
Embodiment 2:
In order to accurately determine the orientation where sound source, sound source is handled so as to subsequent, in the base of above-described embodiment
On plinth, in embodiments of the present invention, the method also includes:
According to the installation site for the microphone for acquiring the reference speech signals, the orientation of sound source is determined.
For the ease of carrying out subsequent processing to sound source, such as sound source can be tracked, so as to it is subsequent more accurately
Respective operations are executed, therefore the orientation of sound source can be determined according to the position of the microphone corresponding to reference speech signals.Work as language
After sound identification module collects voice signal by multiple microphones, because the microphone for acquiring each voice signal is known
, therefore each voice signal can be marked, the corresponding identification information of each voice signal is determined, wherein each voice
The identification information of signal can be the identification information of its collected microphone.When the decibel by comparing each voice signal
Value, it is determined that when reference speech signals, can be determined with sound source distance most according to the corresponding identification information of the reference speech signals
Close microphone.Further, according to the installation site of the microphone, the orientation of sound source is determined.
The installation site of plurality of microphone in home appliance can be any, a kind of possible embodiment, multiple
Microphone is transversely arranged in home appliance；Alternatively possible embodiment, multiple microphones are in home appliance with rectangle
Form arrangement.But microphone is preferably mounted on to the different direction of household appliance, for example, if microphone is three, then
Installation site of three microphones on air-conditioning may be constructed an equilateral triangle, if microphone is four, four Mikes
Installation site of the wind on air-conditioning may be constructed a square, if microphone is five, five microphones are on air-conditioning
Installation site may be constructed pentalpha etc..
In embodiments of the present invention, for the voice signal of multi-faceted accurate acquisition user input, any two are adjacent
The spacing of installation site of the microphone in the household appliance is equal.
For the comprehensive voice signal for collecting sound source sending, it is provided with multiple microphones in home appliance, and
And multiple microphones are installed in home appliance according to certain rules, are in specifically, multiple microphones are installed at equal intervals
In electric equipment, wherein the quantity of microphone can be 5,6 and 7 etc., and the quantity of specific microphone is according to the household appliance
The control precision to be realized is related.
Preferably, six microphones, specifically by taking six microphones as an example, six Mikes can be installed in household appliance
Installation site is as shown in Figure 2 A in home appliance for wind, wherein there are six microphones altogether in figure, are microphone 01, Mike respectively
Wind 02, microphone 03, microphone 04, microphone 05 and microphone 06.The installation site of six microphones is rounded, every two
For the clipping room of microphone away from equal, the clipping room is equal away from the equal spacing for referring to installation site at the same distance of the center of circle,
Installation site and the center of circle in specific six microphones per two adjacent microphones constitute 60 degree of angles.If the sound in figure
Source issues voice signal, and microphone 01 and sound source distance are nearest at this time, the decibel of the collected voice signal of microphone 01
Value is maximum, then the voice signal that microphone 01 acquires is reference speech signals.
When the same sound source of six microphone detections shown in Fig. 2A, different microphones is due to different with a distance from sound source
And cause the decibel value of collected voice signal different.If there is multi-acoustical carries out the input of voice signal simultaneously, then lead to
The mode for crossing multiple microphone acquisition voice signals can more embody the advantage of the embodiment of the present invention.Specifically, if there is more
A sound source carries out the input of voice signal simultaneously, then household appliance is handled just for the maximum voice signal of decibel value, and
Determine whether to carry out corresponding operating based on the maximum voice signal of the decibel value, that is, has reached the effect for carrying out noise reduction to voice signal
Fruit, and realize the accuracy that operation executes.
Fig. 2 B is that multiple microphones provided in an embodiment of the present invention acquire voice signal and determine that the process of effective sound source is shown
It is intended to, detailed process includes:
S201: all microphones acquire the voice signal of sound source sending simultaneously.
S202: the decibel value of the collected voice signal of each microphone is compared.
S203: using the input of the corresponding microphone of the maximum voice signal of decibel value as reference speech signals, and root
Effective sound source is determined according to the installation site of the microphone, and wherein effective sound source is to issue the sound source of the reference speech signals.
Embodiment 3:
More accurate reference speech signals in order to obtain accurately control household appliance with realizing, in above-mentioned each reality
On the basis of applying example, in embodiments of the present invention, before the execution respective operations according to the reference speech signals, the side
Method further include:
For each voice signal in addition to the reference speech signals, identify in the voice signal with the benchmark language
The matched targeted voice signal of sound signal, the targeted voice signal is superimposed upon in the reference speech signals.
Due to including identical part, the voice that only each microphone acquires in the voice signal of multiple microphones acquisition
The decibel value of signal is different, therefore can make full use of the collected voice letter of remaining microphone in addition to reference speech signals
Number, to reference speech signals carry out respective handling, obtain more accurate reference speech signals, according to the reference speech signals into
Row respective operations may be implemented to accurately control household appliance.
Before executing respective operations to reference speech signals obtained above, it can also be acquired according to other microphones
Voice signal carries out supplementary explanation to reference speech signals.Specifically, if other microphones acquisition part of speech signal with
Reference speech signals are similar, for example, the sound spectrum of the voice signal and the sound spectrum of reference speech signals it is similar, it is determined that the part language
Sound signal is targeted voice signal, then the targeted voice signal is superimposed upon in reference speech signals, particularly by target language
Sound signal is added to the matched part of the reference speech signals, then the reference speech signals generated are more clear, according to
The reference speech signals can guarantee that household appliance executes the accuracy of respective operations.
It wherein, is the prior art by the process that targeted voice signal is superimposed upon reference speech signals, in the embodiment of the present invention
In the process is not repeated.
Embodiment 4:
In order to obtain with the matched targeted voice signal of reference speech signals, realizes and household appliance is accurately controlled,
On the basis of the various embodiments described above, in embodiments of the present invention, in described identification voice signal with the reference speech signals
Matched targeted voice signal includes:
The voice signal is handled, the characteristic of the voice signal is generated；
The reference speech signals are matched with the characteristic, corresponding to the characteristic by successful match
Sub- voice signal is determined as targeted voice signal.
Household appliance directly can execute corresponding operation according to the maximum reference speech signals of decibel value, can also basis
Remaining voice signal in addition to reference speech signals is overlapped that treated to reference speech signals, and reference speech signals are held
The corresponding operation of row.Before being overlapped processing to reference speech signals according to remaining voice signal, need to generate remaining first
The characteristic of voice signal, so as to be matched according to characteristic with reference speech signals.Wherein, characteristic refers to structure
At the characteristic information of the voice signal, such as voice voiceprint feature model data are a characteristic of the voice signal.
Specifically, in embodiments of the present invention, the characteristic comprises at least one of the following:
Voice voiceprint feature model data, voiced keyword data.
Wherein, voice signal being handled, the process for generating the characteristic of the voice signal belongs to the prior art,
The process is not repeated in the embodiment of the present invention.
According to the above-mentioned processing to voice signal, after obtaining characteristic, by reference speech signals and characteristic into
Row matching, using sub- voice signal corresponding to the characteristic of successful match as targeted voice signal, by the target language message
It number is superimposed upon in reference speech signals, the reference speech signals after being overlapped operation are more clear, according to superimposed benchmark
Voice signal executes corresponding operation, to ensure that household appliance executes the accuracy of corresponding operating.
In embodiments of the present invention, described by the benchmark in order to further obtain more accurate reference speech signals
Voice signal is matched with the characteristic, and sub- voice signal corresponding to the characteristic by successful match is determined as mesh
Poster sound signal includes:
If this feature data include voice voiceprint feature model data, by the reference speech signals and the voice sound
Line characteristic model data are matched, will be corresponding to the voice voiceprint feature model data in the voice signal with successful match
Sub- voice signal is determined as targeted voice signal；
If this feature data include voiced keyword data, by the reference speech signals and the voiced keyword number
According to being matched, sub- voice signal corresponding to the voiced keyword data in the voice signal with successful match is determined as mesh
Poster sound signal.
Since the characteristic of voice signal comprises at least one of the following: voice voiceprint feature model data, voice are crucial
Digital data.Specifically, by reference speech signals and being somebody's turn to do if the characteristic generated includes voice voiceprint feature model data
Voice voiceprint feature model data are matched, if successful match, by the voice voiceprint feature model data of successful match
Corresponding sub- voice signal is determined as targeted voice signal, and the targeted voice signal is superimposed upon the reference speech signals
In, executing corresponding operation for superimposed reference speech signals can guarantee that household appliance executes the accurate of corresponding operating
Property；If matching is unsuccessful, it is not necessary that voice signal and reference speech signals are overlapped operation, household appliance can direct root
Respective operations are executed according to reference speech signals.
If the characteristic generated includes voiced keyword data, by reference speech signals and the voiced keyword number
According to being matched, if successful match, sub- voice signal corresponding to the voiced keyword data by successful match is determined as
Targeted voice signal, and the targeted voice signal is superimposed upon in the reference speech signals, believe for superimposed reference speech
Number executing corresponding operation can guarantee that household appliance executes the accuracy of corresponding operating；If matching is unsuccessful, it is not necessarily to language
Sound signal and reference speech signals are overlapped operation, and household appliance directly can execute corresponding behaviour according to reference speech signals
Make.
Embodiment 5:
Fig. 3 is the structure chart that a kind of household appliance provided in an embodiment of the present invention operates executive device, which includes:
Receiving module 301, for acquiring voice signal respectively by multiple microphones, wherein each microphone is mounted on institute
The different location of household appliance is stated, and the quantity of the microphone is no less than three；
First determining module 302, the decibel value of the voice signal for being acquired according to each microphone, by decibel value maximum
Voice signal be determined as reference speech signals；
Execution module 303, for executing respective operations according to the reference speech signals.
Described device further include:
Second determining module 304 determines sound for the installation site according to the microphone for acquiring the reference speech signals
The orientation in source.
Described device further include:
Laminating module 305, for successively for each voice signal in addition to the reference speech signals, identifying the language
With the matched targeted voice signal of the reference speech signals in sound signal, the targeted voice signal is superimposed upon the benchmark
In voice signal.
The laminating module 305 generates the characteristic of the voice signal specifically for handling the voice signal
According to；The reference speech signals are matched with the characteristic, sub- language corresponding to the characteristic by successful match
Sound signal is determined as targeted voice signal.
The laminating module 305 will be described if including voice voiceprint feature model data specifically for this feature data
Reference speech signals are matched with the voice voiceprint feature model data, by the voice in the voice signal with successful match
Sub- voice signal corresponding to voiceprint feature model data is determined as targeted voice signal；If this feature data include that voice closes
Key digital data matches the reference speech signals with the voiced keyword data, by the voice signal with matching
Sub- voice signal corresponding to successful voiced keyword data is determined as targeted voice signal.
Due in embodiments of the present invention, acquiring voice signal respectively by multiple microphones for being mounted on different location,
And the decibel value based on collected voice signal, select reference speech signals, it is ensured that no matter user where, all
Relatively accurate voice signal can be collected, to ensure that household appliance executes the accuracy of corresponding operating.
Embodiment 6:
On the basis of the various embodiments described above, the embodiment of the invention also provides a kind of household appliances 400, as described in Figure 4,
It include: processor 401, communication interface 402, memory 403 and communication bus 404, wherein processor 401, communication interface 402,
Memory 403 completes mutual communication by communication bus 404；
It is stored with computer program in the memory 403, when described program is executed by the processor 401, so that
The processor 401 executes following steps:
Household appliance acquires voice signal by multiple microphones respectively, sets wherein each microphone is mounted on the household electrical appliances
Standby different location, and the quantity of the microphone is no less than three；
According to the decibel value of the voice signal of each microphone acquisition, on the basis of the maximum voice signal of decibel value is determined
Voice signal；
Respective operations are executed according to the reference speech signals.
The communication bus that above-mentioned household appliance is mentioned can be Peripheral Component Interconnect standard (Peripheral Component
Interconnect, PCI) bus or expanding the industrial standard structure (Extended Industry Standard
Architecture, EISA) bus etc..The communication bus can be divided into address bus, data/address bus, control bus etc..For just
It is only indicated with a thick line in expression, figure, it is not intended that an only bus or a type of bus.
Communication interface 402 is for the communication between above-mentioned household appliance and other equipment.
The processor 401 also executes the following steps:
According to the installation site for the microphone for acquiring the reference speech signals, the orientation of sound source is determined.
The spacing of installation site of any two neighboring microphones in the household appliance is equal.
The processor 401 is also used to be executed according to the reference speech signals before respective operations:
For each voice signal in addition to the reference speech signals, identify in the voice signal with the benchmark language
The matched targeted voice signal of sound signal, the targeted voice signal is superimposed upon in the reference speech signals.
The processor 401 is also used to identify in the voice signal and the matched target language message of the reference speech signals
Number include:
The voice signal is handled, the characteristic of the voice signal is generated；
The reference speech signals are matched with the characteristic, corresponding to the characteristic by successful match
Sub- voice signal is determined as targeted voice signal.
The characteristic comprises at least one of the following:
Voice voiceprint feature model data, voiced keyword data.
The processor 401 matches the reference speech signals with the characteristic, by the spy of successful match
Sub- voice signal corresponding to sign data is determined as targeted voice signal and includes:
If this feature data include voice voiceprint feature model data, by the reference speech signals and the voice sound
Line characteristic model data are matched, will be corresponding to the voice voiceprint feature model data in the voice signal with successful match
Sub- voice signal is determined as targeted voice signal；
If this feature data include voiced keyword data, by the reference speech signals and the voiced keyword number
According to being matched, sub- voice signal corresponding to the voiced keyword data in the voice signal with successful match is determined as mesh
Poster sound signal.
Memory may include random access memory (Random Access Memory, RAM), also may include non-easy
The property lost memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory may be used also
To be storage device that at least one is located remotely from aforementioned processor.
Above-mentioned processor can be general processor, including central processing unit, network processing unit (Network
Processor, NP) etc.；It can also be digital command processor (Digital Signal Processing, DSP), dedicated collection
At circuit, field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hard
Part component etc..
In embodiments of the present invention, it when processor executes the program stored on memory, realizes through multiple installations
Voice signal, and the decibel value based on collected voice signal are acquired respectively in the microphone of different location, select benchmark language
Sound signal, it is ensured that no matter user where, relatively accurate voice signal can be collected, to ensure that household electrical appliances
The accuracy of equipment execution corresponding operating.
Embodiment 7:
On the basis of the various embodiments described above, the embodiment of the invention also provides a kind of computers to store readable storage medium
Matter is stored with the computer program that can be executed by household appliance in the computer readable storage medium, when described program is in institute
It states when being run in household appliance, so that the household appliance realizes following steps when executing:
Household appliance acquires voice signal by multiple microphones respectively, sets wherein each microphone is mounted on the household electrical appliances
Standby different location, and the quantity of the microphone is no less than three；
According to the decibel value of the voice signal of each microphone acquisition, on the basis of the maximum voice signal of decibel value is determined
Voice signal；
Respective operations are executed according to the reference speech signals.
When described program is run in the household appliance, further includes:
According to the installation site for the microphone for acquiring the reference speech signals, the orientation of sound source is determined.
The spacing of installation site of any two neighboring microphones in the household appliance is equal.
Before the execution respective operations according to the reference speech signals, the method also includes:
For each voice signal in addition to the reference speech signals, identify in the voice signal with the benchmark language
The matched targeted voice signal of sound signal, the targeted voice signal is superimposed upon in the reference speech signals.
Include: with the matched targeted voice signal of the reference speech signals in described identification voice signal
The voice signal is handled, the characteristic of the voice signal is generated；
The reference speech signals are matched with the characteristic, corresponding to the characteristic by successful match
Sub- voice signal is determined as targeted voice signal.
The characteristic comprises at least one of the following:
Voice voiceprint feature model data, voiced keyword data.
It is described to match the reference speech signals with the characteristic, the characteristic institute of successful match is right
The sub- voice signal answered is determined as targeted voice signal and includes:
If this feature data include voice voiceprint feature model data, by the reference speech signals and the voice sound
Line characteristic model data are matched, will be corresponding to the voice voiceprint feature model data in the voice signal with successful match
Sub- voice signal is determined as targeted voice signal；
If this feature data include voiced keyword data, by the reference speech signals and the voiced keyword number
According to being matched, sub- voice signal corresponding to the voiced keyword data in the voice signal with successful match is determined as mesh
Poster sound signal.
Above-mentioned computer readable storage medium can be any usable medium that the processor in household appliance can access
Or data storage device, including but not limited to magnetic storage such as floppy disk, hard disk, tape, magneto-optic disk (MO) etc., optical memory
Such as CD, DVD, BD, HVD and semiconductor memory such as ROM, EPROM, EEPROM, nonvolatile memory (NAND
FLASH), solid state hard disk (SSD) etc..
Computer program, computer program are provided in the computer readable storage medium provided in embodiments of the present invention
It when being executed by processor, realizes and acquires voice signal respectively by multiple microphones for being mounted on different location, and be based on adopting
The decibel value of the voice signal collected, select reference speech signals, it is ensured that no matter user where, can collect
Relatively accurate voice signal, to ensure that household appliance executes the accuracy of corresponding operating.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (14)
1. a kind of household appliance operates execution method, which is characterized in that the described method includes:
Household appliance acquires voice signal by multiple microphones respectively, wherein each microphone is mounted on the household appliance
Different location, and the quantity of the microphone is no less than three；
According to the decibel value of the voice signal of each microphone acquisition, the maximum voice signal of decibel value is determined as reference speech
Signal；
Respective operations are executed according to the reference speech signals.
2. the method as described in claim 1, which is characterized in that the method also includes:
According to the installation site for the microphone for acquiring the reference speech signals, the orientation of sound source is determined.
3. the method as described in claim 1, which is characterized in that peace of any two neighboring microphones in the household appliance
The spacing of holding position is equal.
4. the method as described in claim 1, which is characterized in that it is described according to the reference speech signals execute respective operations it
Before, the method also includes:
For each voice signal in addition to the reference speech signals, identifies in the voice signal and believe with the reference speech
Number matched targeted voice signal, the targeted voice signal is superimposed upon in the reference speech signals.
5. method as claimed in claim 4, which is characterized in that in described identification voice signal with the reference speech signals
Matched targeted voice signal includes:
The voice signal is handled, the characteristic of the voice signal is generated；
The reference speech signals are matched with the characteristic, sub- language corresponding to the characteristic by successful match
Sound signal is determined as targeted voice signal.
6. method as claimed in claim 5, which is characterized in that the characteristic comprises at least one of the following:
Voice voiceprint feature model data, voiced keyword data.
7. method as claimed in claim 5, which is characterized in that it is described by the reference speech signals and the characteristic into
Row matches, and sub- voice signal corresponding to the characteristic by successful match is determined as targeted voice signal and includes:
It is if this feature data include voice voiceprint feature model data, the reference speech signals and the voice vocal print are special
Sign model data is matched, by sub- language corresponding to the voice voiceprint feature model data in the voice signal with successful match
Sound signal is determined as targeted voice signal；
If this feature data include voiced keyword data, by the reference speech signals and the voiced keyword data into
Row matching, is determined as target language for sub- voice signal corresponding to the voiced keyword data in the voice signal with successful match
Sound signal.
8. a kind of household appliance operates executive device, which is characterized in that described device includes:
Receiving module, for acquiring voice signal respectively by multiple microphones, wherein each microphone is mounted on the household electrical appliances
The different location of equipment, and the quantity of the microphone is no less than three；
First determining module, the decibel value of the voice signal for being acquired according to each microphone, by the maximum voice of decibel value
Signal is determined as reference speech signals；
Execution module, for executing respective operations according to the reference speech signals.
9. device as claimed in claim 8, which is characterized in that described device further include:
Second determining module determines the side of sound source for the installation site according to the microphone for acquiring the reference speech signals
Position.
10. device as claimed in claim 8, which is characterized in that described device further include:
Laminating module, for successively for each voice signal in addition to the reference speech signals, identifying the voice signal
In with the matched targeted voice signal of the reference speech signals, the targeted voice signal is superimposed upon the reference speech and is believed
In number.
11. device as claimed in claim 10, which is characterized in that the laminating module, be specifically used for the voice signal into
Row processing, generates the characteristic of the voice signal；The reference speech signals are matched with the characteristic, general
It is determined as targeted voice signal with sub- voice signal corresponding to successful characteristic.
12. device as claimed in claim 11, which is characterized in that the laminating module, if being specifically used for this feature data
Including voice voiceprint feature model data, by the reference speech signals and voice voiceprint feature model data progress
Match, sub- voice signal corresponding to the voice voiceprint feature model data in the voice signal with successful match is determined as target
Voice signal；If this feature data include voiced keyword data, by the reference speech signals and the voiced keyword
Data are matched, and sub- voice signal corresponding to the voiced keyword data in the voice signal with successful match is determined as
Targeted voice signal.
13. a kind of household appliance characterized by comprising processor, communication interface, memory and communication bus, wherein place
Device, communication interface are managed, memory completes mutual communication by communication bus；
It is stored with computer program in the memory, when described program is executed by the processor, so that the processor
Perform claim requires the step of any one of 1~7 the method.
14. a kind of computer readable storage medium, which is characterized in that it is stored with the computer journey that can be executed by household appliance
Sequence, when described program is run in the household appliance, so that the household appliance perform claim requires any one of 1~7 institute
The step of stating method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811195101.3A CN109309607A (en) | 2018-10-15 | 2018-10-15 | Household appliance operation execution method and device, household appliance and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811195101.3A CN109309607A (en) | 2018-10-15 | 2018-10-15 | Household appliance operation execution method and device, household appliance and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109309607A true CN109309607A (en) | 2019-02-05 |
Family
ID=65224473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811195101.3A Pending CN109309607A (en) | 2018-10-15 | 2018-10-15 | Household appliance operation execution method and device, household appliance and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109309607A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109981616A (en) * | 2019-03-12 | 2019-07-05 | 北京神州绿盟信息安全科技股份有限公司 | Detection method, device and the network equipment of voice attack |
| CN110647045A (en) * | 2019-09-27 | 2020-01-03 | 恒大智慧科技有限公司 | Intelligent household control method and device and computer readable storage medium |
| CN110942773A (en) * | 2019-12-10 | 2020-03-31 | 上海雷盎云智能技术有限公司 | Method and device for controlling intelligent household equipment through voice |
| CN113014460A (en) * | 2019-12-20 | 2021-06-22 | 佛山市云米电器科技有限公司 | Voice processing method, home master control device, voice system and storage medium |
| CN114954004A (en) * | 2022-06-22 | 2022-08-30 | 润芯微科技(江苏)有限公司 | Car machine interaction system based on sound source identification |
| CN115273850A (en) * | 2022-09-28 | 2022-11-01 | 科大讯飞股份有限公司 | Autonomous mobile equipment voice control method and system |
| WO2023013022A1 (en) * | 2021-08-06 | 2023-02-09 | 三菱電機ビルソリューションズ株式会社 | Placement position specification system and placement position specification method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012211768A (en) * | 2011-03-30 | 2012-11-01 | Advanced Telecommunication Research Institute International | Sound source positioning apparatus |
| CN104702787A (en) * | 2015-03-12 | 2015-06-10 | 深圳市欧珀通信软件有限公司 | Sound acquisition method applied to MT (Mobile Terminal) and MT |
| CN104935819A (en) * | 2015-06-11 | 2015-09-23 | 广东欧珀移动通信有限公司 | Method for controlling camera to shoot and terminal |
| CN107123421A (en) * | 2017-04-11 | 2017-09-01 | 广东美的制冷设备有限公司 | Sound control method, device and home appliance |
| CN108445451A (en) * | 2018-05-11 | 2018-08-24 | 四川斐讯信息技术有限公司 | A kind of intelligent sound box and its sound localization method |
-
2018
- 2018-10-15 CN CN201811195101.3A patent/CN109309607A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012211768A (en) * | 2011-03-30 | 2012-11-01 | Advanced Telecommunication Research Institute International | Sound source positioning apparatus |
| CN104702787A (en) * | 2015-03-12 | 2015-06-10 | 深圳市欧珀通信软件有限公司 | Sound acquisition method applied to MT (Mobile Terminal) and MT |
| CN104935819A (en) * | 2015-06-11 | 2015-09-23 | 广东欧珀移动通信有限公司 | Method for controlling camera to shoot and terminal |
| CN107123421A (en) * | 2017-04-11 | 2017-09-01 | 广东美的制冷设备有限公司 | Sound control method, device and home appliance |
| CN108445451A (en) * | 2018-05-11 | 2018-08-24 | 四川斐讯信息技术有限公司 | A kind of intelligent sound box and its sound localization method |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109981616A (en) * | 2019-03-12 | 2019-07-05 | 北京神州绿盟信息安全科技股份有限公司 | Detection method, device and the network equipment of voice attack |
| CN109981616B (en) * | 2019-03-12 | 2021-07-13 | 绿盟科技集团股份有限公司 | Voice attack detection method and device and network equipment |
| CN110647045A (en) * | 2019-09-27 | 2020-01-03 | 恒大智慧科技有限公司 | Intelligent household control method and device and computer readable storage medium |
| CN110942773A (en) * | 2019-12-10 | 2020-03-31 | 上海雷盎云智能技术有限公司 | Method and device for controlling intelligent household equipment through voice |
| CN113014460A (en) * | 2019-12-20 | 2021-06-22 | 佛山市云米电器科技有限公司 | Voice processing method, home master control device, voice system and storage medium |
| CN113014460B (en) * | 2019-12-20 | 2023-03-14 | 佛山市云米电器科技有限公司 | Voice processing method, home master control device, voice system and storage medium |
| WO2023013022A1 (en) * | 2021-08-06 | 2023-02-09 | 三菱電機ビルソリューションズ株式会社 | Placement position specification system and placement position specification method |
| CN114954004A (en) * | 2022-06-22 | 2022-08-30 | 润芯微科技(江苏)有限公司 | Car machine interaction system based on sound source identification |
| CN115273850A (en) * | 2022-09-28 | 2022-11-01 | 科大讯飞股份有限公司 | Autonomous mobile equipment voice control method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109309607A (en) | Household appliance operation execution method and device, household appliance and readable storage medium | |
| CN109389976A (en) | Intelligent household electrical appliance control method and device, intelligent household electrical appliance and storage medium | |
| TWI665584B (en) | A voice controlling system and method | |
| CN111063341B (en) | Method and system for segmenting and clustering multi-person voice in complex environment | |
| CN106328127A (en) | Speech recognition apparatus, speech recognition method, and electronic device | |
| CN105308679A (en) | Method and system for identifying location associated with voice command to control home appliance | |
| CN105551498A (en) | Voice recognition method and device | |
| CN107591155A (en) | Audio recognition method and device, terminal and computer-readable recording medium | |
| CN106782529B (en) | Awakening word selection method and device for voice recognition | |
| CN113205803B (en) | Voice recognition method and device with self-adaptive noise reduction capability | |
| Ntalampiras | A novel holistic modeling approach for generalized sound recognition | |
| WO2014172167A1 (en) | Vocal keyword training from text | |
| CN111883122A (en) | Voice recognition method and device, storage medium and electronic equipment | |
| CN109360579A (en) | Charging pile phonetic controller and system | |
| CN109671430B (en) | Voice processing method and device | |
| CN107342097A (en) | Recording method, recording device, intelligent terminal and computer readable storage medium | |
| JP2020009440A (en) | Method and device for generating information | |
| CN109410956A (en) | A kind of object identifying method of audio data, device, equipment and storage medium | |
| CN104992715A (en) | Interface switching method and system of intelligent device | |
| Battaglino et al. | Acoustic context recognition using local binary pattern codebooks | |
| KR20210078133A (en) | Training data generating method for training filled pause detecting model and device therefor | |
| Kim et al. | Efficient implementation of the room simulator for training deep neural network acoustic models | |
| CN113077812B (en) | Voice signal generation model training method, echo cancellation method, device and equipment | |
| Liu et al. | Golden gemini is all you need: Finding the sweet spots for speaker verification | |
| WO2021135454A1 (en) | Method, device, and computer-readable storage medium for recognizing fake speech |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190205 |
|
| RJ01 | Rejection of invention patent application after publication |