WO2023273321A1 - 一种语音控制方法及电子设备 - Google Patents
一种语音控制方法及电子设备 Download PDFInfo
- Publication number
- WO2023273321A1 WO2023273321A1 PCT/CN2022/073135 CN2022073135W WO2023273321A1 WO 2023273321 A1 WO2023273321 A1 WO 2023273321A1 CN 2022073135 W CN2022073135 W CN 2022073135W WO 2023273321 A1 WO2023273321 A1 WO 2023273321A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- electronic device
- voice
- preset
- voice data
- preset keyword
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 89
- 230000003993 interaction Effects 0.000 claims abstract description 67
- 230000006870 function Effects 0.000 claims description 58
- 230000015654 memory Effects 0.000 claims description 35
- 230000004044 response Effects 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 description 29
- 238000012545 processing Methods 0.000 description 25
- 238000004891 communication Methods 0.000 description 18
- 238000010586 diagram Methods 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 9
- 238000007726 management method Methods 0.000 description 8
- 238000010295 mobile communication Methods 0.000 description 8
- 230000005236 sound signal Effects 0.000 description 8
- 235000003140 Panax quinquefolius Nutrition 0.000 description 7
- 240000005373 Panax quinquefolius Species 0.000 description 7
- 238000012360 testing method Methods 0.000 description 6
- 229920001621 AMOLED Polymers 0.000 description 4
- GRRMZXFOOGQMFA-UHFFFAOYSA-J YoYo-1 Chemical compound [I-].[I-].[I-].[I-].C12=CC=CC=C2C(C=C2N(C3=CC=CC=C3O2)C)=CC=[N+]1CCC[N+](C)(C)CCC[N+](C)(C)CCC[N+](C1=CC=CC=C11)=CC=C1C=C1N(C)C2=CC=CC=C2O1 GRRMZXFOOGQMFA-UHFFFAOYSA-J 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000002699 waste material Substances 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013529 biological neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000011269 treatment regimen Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
Definitions
- the present application relates to the field of terminals and artificial intelligence, in particular to a voice control method and electronic equipment.
- voice interaction A typical representative of implementing voice control is voice interaction.
- more and more electronic devices have a voice interaction function, and are equipped with applications capable of voice interaction with users, such as voice assistants. Users can interact with electronic devices through voice assistants to realize functions that previously required multiple manual operations. For example, making calls, playing music, etc. Waste power consumption of electronic equipment.
- the present application provides a voice control method and an electronic device.
- the electronic device does not need to enable the voice interaction function, but can also receive a non-specific voice command input by a user and respond to the non-specific voice command.
- the electronic device does not need to enable the voice interaction function, but can also receive non-specific voice commands input by the user and respond to the non-specific voice commands.
- the power consumption of the electronic device is consumed when voice interaction is turned on, and the power consumption of the electronic device is also consumed in the state of voice interaction.
- the electronic device determines that it is currently before the first application scene, and the method further includes: the electronic device displays a first user interface; the electronic device does not enable the voice interaction function, and responds to the first
- the voice command is to execute the operation corresponding to the first preset keyword, which specifically includes: the electronic device does not turn on the voice interaction function, responds to the first voice command, executes the operation corresponding to the first preset keyword, and displays the second user interface, the second user interface is different from the first user interface.
- the electronic device can change the user interface of the electronic device according to the voice command, so that the electronic device can provide some visualization services for the user. For example, when the electronic device opens a video application, the electronic device can respond to the first voice command to provide the user with functions related to video playback, and the user can control the electronic device without manually touching the display screen of the electronic device.
- the method according to claim 1 is characterized in that determining that the electronic device is currently in the first application scene specifically includes: the electronic device runs the first application program in the foreground, then the The electronic device determines that it is currently in the first application scenario.
- the electronic device determines that it is currently in the first application scenario, specifically including:
- the electronic device determines that it is currently in the first application scenario.
- the electronic device can also determine that it is currently in a specific application scene, and the electronic device can be controlled by non-specific voice commands.
- the electronic device can perform response operations in the background to realize voice control of the electronic device. For example, when the electronic device opens a music application, the first voice command is: "Play louder", and the electronic device can perform this operation in the background, which does not involve changes to the user interface and does not affect the user's use of the electronic device running in the foreground.
- the electronic device determines whether the first voice instruction includes a first preset keyword, specifically including: the electronic device loads all preset keywords corresponding to the first application scene ; The electronic device determines whether the first voice command includes first voice data according to the first voice command and all preset keywords corresponding to the first application scene, and the first voice data includes at least the first preset keyword; in the case of determining that the first voice instruction includes the first voice data, the electronic device determines that the first voice instruction includes the first preset keyword; when determining that the first voice instruction does not include the first In the case of voice data, the electronic device determines that the first voice instruction does not include the first preset keyword.
- the electronic device first determines whether the first voice command includes the first voice data, the algorithm involved in this process does not need to be too complicated, as long as the first voice can be determined, the electronic device can use a digital signal processor Completing the process saves computing resources of the electronic device.
- the method further includes: the electronic device loads all preset keywords corresponding to all specific application scenarios, the All specific application scenarios include the first application scenario, and all preset keywords include the first preset keyword; the electronic device determines the second preset keyword according to the first voice data and all preset keywords corresponding to all specific application scenarios.
- a part of the voice data is used as the second voice data, and the second voice data only includes the first preset keyword; in response to the first voice command, the operation corresponding to the first preset keyword is executed, specifically including: the electronic The device generates an operation corresponding to the first preset keyword according to the first preset keyword in the second voice data; and executes the operation corresponding to the first preset keyword in response to the first voice instruction.
- the electronic device can extract the preset keywords in the first voice instruction through a relatively accurate algorithm, And use the preset keyword in the first voice command to generate a corresponding operation, and the electronic device can respond to the operation to implement voice control of the electronic device.
- the method further includes: the electronic device loads all preset keywords corresponding to all specific application scenarios, the All specific application scenarios include the first application scenario, and all preset keywords include the first preset keyword; the electronic device determines the second preset keyword according to the first voice data and all preset keywords corresponding to all specific application scenarios.
- a part of the voice data is used as the second voice data, and the second voice data only includes the first preset keyword; the electronic device determines whether the voiceprint of the second voice data matches the preset voiceprint, and the preset voiceprint
- the fingerprint is the voiceprint identification of the user's voice data entered by the electronic device, which is used to identify the user's identity; in response to the first voice command, perform the operation corresponding to the first preset keyword, specifically including: after determining the second When the voiceprint of the voice data matches the preset voiceprint, the electronic device generates an operation corresponding to the first preset keyword according to the first preset keyword in the second voice data; responding to the first voice command , executing the operation corresponding to the first preset keyword; if it is determined that the voiceprint of the second voice data does not match the preset voiceprint, not responding to the first voice instruction.
- the electronic device After the electronic device recognizes the preset keywords in the first voice command, it needs to judge whether the first voice command is input by the "owner" of the electronic device.
- the "owner" of the electronic device will input his own biological information, such as a voiceprint, through the electronic device, and the electronic device can match according to the voiceprint of the first voice command and the voiceprint input by the "owner” through the electronic device It is determined whether the voice command is input by the "owner” of the electronic device. If only the first voice command can be responded to, it can prevent anyone from controlling the electronic device by voice, which increases the security of implementing the method.
- all the preset keywords corresponding to the specific application scene are preset and stored in the electronic device, any of the preset keywords corresponding to the specific application scene
- a preset keyword corresponds to an operation with the same meaning as the preset keyword.
- the present application provides an electronic device, which includes: one or more processors and memory; the memory is coupled to the one or more processors, the memory is used to store computer program codes, and the computer
- the program code includes computer instructions
- the one or more processors call the computer instructions to make the electronic device execute: determine that it is currently in the first application scene; detect the first voice instruction; the first voice instruction is not used to open the voice interaction Functional non-specific voice commands; determine whether the first voice command includes a first preset keyword; the first preset keyword is any preset key among all preset keywords corresponding to the first application scene words; when it is determined that the first voice instruction includes the first preset keyword, the voice interaction function is not turned on, and the operation corresponding to the first preset keyword is executed in response to the first voice instruction; When a voice command does not include the first preset keyword, the first voice command is not responded to.
- the electronic device does not need to enable the voice interaction function, but can also receive a non-specific voice command input by the user, and respond to the non-specific voice command.
- the power consumption of the electronic equipment is consumed when voice interaction is turned on, and the power consumption of the electronic equipment is also consumed in the voice interaction state.
- the electronic device can change the user interface of the electronic device according to the voice command, so that the electronic device can provide some visualization services for the user. For example, when the electronic device opens a video application, the electronic device can respond to the first voice command to provide the user with functions related to video playback, and the user can control the electronic device without manually touching the display screen of the electronic device.
- the electronic device when a specific application is running in the foreground, the electronic device may also determine that it is currently in a specific application scene, and non-specific voice commands may be used to control the electronic device.
- the electronic device can perform response operations in the foreground, realizing voice control of the electronic device. For example, when the electronic device opens a video application, the first voice instruction is: "play the next episode", and the electronic device can perform this operation in the foreground to play the next episode.
- the one or more processors are specifically configured to call the computer instruction to make the electronic device execute: run the first application program in the background, then determine that it is currently in the first application scenario.
- the electronic device can also determine that it is currently in a specific application scene, and the electronic device can be controlled by non-specific voice commands.
- the electronic device can perform response operations in the background to realize voice control of the electronic device. For example, when the electronic device opens a music application, the first voice command is: "Play louder", and the electronic device can perform this operation in the background, which does not involve changes to the user interface and does not affect the user's use of the electronic device running in the foreground.
- the one or more processors are specifically configured to call the computer instruction to make the electronic device execute: load all preset keywords corresponding to the first application scene;
- a voice command corresponds to all preset keywords corresponding to the first application scene, and it is determined whether the first voice command includes the first voice data, and the first voice data includes at least the first preset keyword;
- a voice command includes the first voice data, it is determined that the first voice command includes the first preset keyword;
- it is determined that the first voice command does not include the first voice data it is determined that the first The voice instruction does not include the first preset keyword.
- the electronic device first determines whether the first voice command includes the first voice data, the algorithm involved in this process does not need to be too complicated, as long as the first voice can be determined, the electronic device can use a digital signal processor Completing the process saves computing resources of the electronic device.
- the one or more processors are further configured to call the computer instruction to make the electronic device execute: load all preset keywords corresponding to all specific application scenarios, and all specific application scenarios
- the scene includes the first application scene, and all the preset keywords include the first preset keyword; according to all the preset keywords corresponding to the first voice data and all the specific application scenarios, determine a part of the first voice data
- the second voice data only includes the first preset keyword; the one or more processors are specifically used to call the computer instruction to make the electronic device perform: according to the second voice data in the second voice data
- a preset keyword generates an operation corresponding to the first preset keyword; in response to the first voice instruction, the operation corresponding to the first preset keyword is executed.
- the electronic device can extract the preset keywords in the first voice instruction through a relatively accurate algorithm, And use the preset keyword in the first voice command to generate a corresponding operation, and the electronic device can respond to the operation to implement voice control of the electronic device.
- the one or more processors are further configured to call the computer instruction to make the electronic device execute: load all preset keywords corresponding to all specific application scenarios, and all specific application scenarios
- the scene includes the first application scene, and all the preset keywords include the first preset keyword; according to all the preset keywords corresponding to the first voice data and all the specific application scenarios, determine a part of the first voice data
- the second voice data only includes the first preset keyword; determine whether the voiceprint of the second voice data matches the preset voiceprint, the preset voiceprint is the user’s registered voice in the electronic device
- the voiceprint identification of the voice data is used to identify the identity of the user; the one or more processors are specifically used to call the computer instruction to make the electronic device execute: when determining the voiceprint of the second voice data and the preset voiceprint In the case of a match, an operation corresponding to the first preset keyword is generated according to the first preset keyword in the second voice data; in response to the first voice instruction, an operation corresponding to
- the electronic device After the electronic device recognizes the preset keywords in the first voice command, it needs to judge whether the first voice command is input by the "owner" of the electronic device.
- the "owner" of the electronic device will input his own biological information, such as a voiceprint, through the electronic device, and the electronic device can match according to the voiceprint of the first voice command and the voiceprint input by the "owner” through the electronic device It is determined whether the voice command is input by the "owner” of the electronic device. If only the first voice command can be responded to, it can prevent anyone from controlling the electronic device by voice, which increases the security of implementing the method.
- an embodiment of the present application provides an electronic device, where the electronic device includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, The one or more memories are used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the one or more processors are used to invoke the computer instructions so that the The electronic device executes the method described in the first aspect or any implementation manner of the first aspect.
- the electronic device does not need to enable the voice interaction function, but can also receive a non-specific voice command input by the user, and respond to the non-specific voice command.
- the power consumption of the electronic device is consumed when voice interaction is turned on, and the power consumption of the electronic device is also consumed in the state of voice interaction.
- the embodiment of the present application provides a chip system, the chip system includes one or more processors, and the processors are used to call computer instructions to make the electronic device execute any one of the first aspect or the first aspect. The method described in one embodiment.
- the electronic device does not need to enable the voice interaction function, but can also receive a non-specific voice command input by the user, and respond to the non-specific voice command.
- the power consumption of the electronic equipment is consumed when voice interaction is turned on, and the power consumption of the electronic equipment is also consumed in the voice interaction state.
- the embodiment of the present application provides a computer program product containing instructions, and when the computer program product is run on the electronic device, the electronic device is made to execute any one of the first aspect or the first aspect. method described.
- the electronic device does not need to enable the voice interaction function, but can also receive a non-specific voice command input by the user, and respond to the non-specific voice command.
- the power consumption of the electronic device is consumed when voice interaction is turned on, and the power consumption of the electronic device is also consumed in the state of voice interaction.
- the embodiment of the present application provides a computer-readable storage medium, which, when the instruction is run on the electronic device, causes the electronic device to execute the method described in the first aspect or any implementation manner of the first aspect. method.
- the electronic device does not need to enable the voice interaction function, but can also receive a non-specific voice command input by the user, and respond to the non-specific voice command.
- the power consumption of the electronic device is consumed when voice interaction is turned on, and the power consumption of the electronic device is also consumed in the state of voice interaction.
- Figures 1a-1d are a set of user interfaces for a user to control an electronic device through voice in one solution
- Figures 2a-2d are a set of exemplary user interfaces for a user controlling an electronic device through voice provided in the present application;
- FIG. 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
- Fig. 4 is a software structural block diagram of the electronic device provided by the embodiment of the present application.
- FIG. 5 is a schematic diagram of an exemplary information flow of the voice control method involved in the present application.
- FIG. 6 is a schematic flowchart of a voice control method provided by an embodiment of the present application.
- FIG. 7 is a schematic diagram of determining the first voice data in the embodiment of the present application.
- first and second are used for descriptive purposes only, and cannot be understood as implying or implying relative importance or implicitly specifying the quantity of indicated technical features. Therefore, the features defined as “first” and “second” may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present application, unless otherwise specified, the “multiple” The meaning is two or more.
- UI user interface
- the term "user interface (UI)” in the following embodiments of this application is a medium interface for interaction and information exchange between an application program or an operating system and a user, and it realizes the difference between the internal form of information and the form acceptable to the user. conversion between.
- the user interface is the source code written in a specific computer language such as java and extensible markup language (XML).
- the source code of the interface is parsed and rendered on the electronic device, and finally presented as content that can be recognized by the user.
- the commonly used form of user interface is the graphical user interface (graphic user interface, GUI), which refers to the user interface related to computer operation displayed in a graphical way. It may be text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, and other visible interface elements displayed on the display screen of the electronic device.
- the preset keywords are preset and stored in the electronic device. Any preset keyword corresponds to an operation with the same meaning as the preset keyword. For example, when the specific application scene is a music scene, when the preset keyword is "pause”. Then it means that the operation corresponding to the preset keyword is to pause music playback.
- voice control of the electronic device can be implemented.
- the specific application scenario is preset, and the electronic device runs a certain application program, which corresponds to one application scenario.
- a specific application scenario is an application scenario in which voice control can be performed on an electronic device. It can be set that when the electronic device is running (including running in the background and running in the foreground) the first application program, the electronic device will enter the specific application scenario corresponding to the first application program.
- the first application program may be a music application program, a video application program, a navigation application program, a call application program, and the like.
- the electronic device When the electronic device is running a music application program, it is a music scene, and the corresponding preset keywords in the music scene may include: “play the next song”, “play the previous song”, “make the sound louder”, “make the sound softer”, Commonly used keywords such as "pause”.
- the corresponding preset keywords in the video scene may include: “play the next song”, “play the previous song”, “make the sound louder”, “make the sound softer”, Commonly used keywords such as "pause”.
- the corresponding preset keywords in the navigation scene may include: “navigate home”, “navigate to work”, “speak louder”, “speak softer” and other commonly used keywords word.
- the electronic device When the electronic device is running a calling application program, it is a music scene, and the corresponding preset keywords in the calling scene may include commonly used keywords such as "make the sound louder” and “make the sound softer”.
- this application may also include other specific application scenarios, and each specific application scenario may include more or less preset keywords than the above.
- the electronic device must first detect the user's input to enable the voice interaction function before enabling the voice interaction function, so that the user can control the electronic device through voice.
- the input for turning on the voice interaction function may be a preset specific voice command, such as "YOYO, YOYO", or a long press of the power button.
- the specific voice instruction may also be called a wake-up word or a wake-up instruction.
- Figures 1a-1d show a set of user interfaces for the user to control the electronic device through voice in this solution.
- the electronic device may display a user interface 11.
- the user interface 11 may be a music playing interface of the electronic device, and the electronic device is currently playing "first music”.
- the electronic device can detect a specific voice command for the user to turn on the voice interaction function: "YOYO, YOYO".
- the electronic device can display a user interface 12 as shown in FIG. 1b.
- the user interface 12 may include a voice prompt box 121, and a prompt message 121A may be displayed in the voice prompt box 121: "You said, I am listening" and a voice collection logo 121B, the prompt information 121A and the voice prompt
- the collection mark 121B can be used to prompt the user: the current electronic device can receive voice commands, and the user can control the electronic device by voice.
- the user interface 13 can be a user interface when the first voice command input by the user is detected when the electronic device is playing music, the first voice command is a non-specific voice command, and the non-specific voice command refers to a user interface other than Additional voice commands for specific voice commands.
- the electronic device may display the detected first voice instruction in the voice prompt box 131 shown on the user interface 13 .
- the first voice instruction input by the user may be: "play the next song”.
- the electronic device can execute playing the next piece of music, and display the user interface 14 as shown in FIG. 1d.
- the electronic device always turns on the voice interaction function to detect the user's voice command, which will waste the power consumption of the electronic device, so the electronic device is usually set to turn off the voice interaction function after responding to a non-specific voice command. Then the next time the user needs to perform voice interaction with the electronic device, the voice interaction function can only be turned on again if a specific voice command is detected again, so that the user can control the electronic device by voice again.
- the electronic device After the electronic device responds to the first voice command, it will turn off the voice interaction function, and when the user inputs the second voice command: "play the previous song", the electronic device will not respond to the second voice command. Two voice commands, the user interface 14 is still displayed.
- the second voice instruction is a non-specific voice instruction.
- the user needs to turn on the voice interaction function before each input of a non-specific voice command, so that the user can control the electronic device by voice.
- a specific voice command must be input first, so that the electronic device can respond to the first voice command, so that the user can control the electronic device by voice.
- the embodiment of the present application provides a voice control method, implement the voice control method in the embodiment of the present application, in some specific application scenarios, the electronic device does not need to open the voice interaction function, and can also receive A non-specific voice command input by the user and respond to the non-specific voice command.
- the application scenario for running music application programs on the electronic device is music scene
- the application scenario for running video application programs is video scene
- the application scenario for running navigation application programs is navigation scene
- the application scenario for running call application programs is The application scenario.
- Non-specific instructions refer to other voice instructions that are different from specific voice instructions, and are not used to turn on the voice interaction function of electronic devices.
- the electronic device can set some preset keywords for different specific application scenarios, and when the electronic device detects that the preset keyword is included in the voice command input by the user, the electronic device can respond to the voice command and execute the first operation.
- Figures 2a-2d are a set of exemplary user interfaces provided in this application for a user to control an electronic device through voice.
- the preset specific application scenarios include an application scenario in which an electronic device runs a music application.
- the preset keywords may include: “play the previous song” and “play the next song”.
- the user interface 21 may be a user interface when the electronic device detects that the user inputs a first voice command while playing music, and the electronic device plays "first music". At this time, the electronic device may display the detected first voice instruction in the voice prompt box 211 shown on the user interface 21, where the first voice instruction is a non-specific voice instruction.
- the first voice instruction input by the user may be: "play the next song”. Since the first voice command includes a preset keyword, the electronic device may respond to the first voice command to perform an operation of playing the next piece of music, and display a user interface 22 as shown in FIG. 2b.
- the electronic device has switched the currently playing music from "first music” to "second music”.
- the electronic device can continuously respond to any voice command input by the user.
- the user interface 23 may be a user interface when the electronic device detects that the user inputs a second voice command while playing music, and the electronic device plays "second music". After the electronic device responds to the user inputting the first voice command, the user inputs a second voice command, and the second voice command is a non-specific voice command.
- the electronic device can detect the second voice command and display it in the voice prompt box 231 shown on the user interface 23 .
- the second voice instruction may be: "play the previous song”. Since the second voice command includes preset keywords, the electronic device can respond to the second voice command to perform an operation of playing the previous music, and display the user interface 24 as shown in FIG. 2d.
- the electronic device has switched the currently playing music from “second music” to “first music” again.
- the electronic device can not only save power consumption, but also meet the needs of the user to control the electronic device by voice.
- the user does not need to input a specific voice command to turn on the voice interaction function before inputting a non-specific voice command every time, so that the electronic device can be controlled by voice.
- FIG. 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
- an electronic device is taken as an example to describe the embodiment in detail. It should be understood that an electronic device may have more or fewer components than shown in the figures, two or more components may be combined, or may have a different configuration of components.
- the various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.
- the electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194 and user An identification module (subscriber identification module, SIM) card interface 195 and the like.
- SIM subscriber identification module
- the structure shown in the embodiment of the present invention does not constitute a specific limitation on the electronic device.
- the electronic device may include more or fewer components than shown in the illustrations, or combine certain components, or separate certain components, or arrange different components.
- the illustrated components can be realized in hardware, software or a combination of software and hardware.
- the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
- application processor application processor, AP
- modem processor graphics processing unit
- GPU graphics processing unit
- image signal processor image signal processor
- ISP image signal processor
- controller memory
- video codec digital signal processor
- DSP digital signal processor
- baseband processor baseband processor
- neural network processor neural-network processing unit
- the controller may be the nerve center and command center of the electronic equipment.
- the controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
- a memory may also be provided in the processor 110 for storing instructions and data.
- the memory in processor 110 is a cache memory.
- the memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
- processor 110 may include one or more interfaces.
- the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.
- I2C integrated circuit
- I2S integrated circuit built-in audio
- PCM pulse code modulation
- PCM pulse code modulation
- UART universal asynchronous transmitter
- MIPI mobile industry processor interface
- GPIO general-purpose input and output
- subscriber identity module subscriber identity module
- SIM subscriber identity module
- USB universal serial bus
- the interface connection relationship between the modules shown in the embodiment of the present invention is only a schematic illustration, and does not constitute a structural limitation of the electronic device.
- the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
- the charging management module 140 is configured to receive a charging input from a charger.
- the charger may be a wireless charger or a wired charger.
- the power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 .
- the power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 .
- the wireless communication function of the electronic device can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
- Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
- Each antenna in an electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas.
- Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
- the antenna may be used in conjunction with a tuning switch.
- the mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied to electronic devices.
- the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like.
- the mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation.
- the mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves through the antenna 1 for radiation.
- a modem processor may include a modulator and a demodulator.
- the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal.
- the demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing.
- the low-frequency baseband signal is passed to the application processor after being processed by the baseband processor.
- the application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 .
- the wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system, etc. (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions.
- the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
- the wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
- the wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
- the antenna 1 of the electronic device is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device can communicate with the network and other devices through wireless communication technology.
- the wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA) and the like.
- the electronic device realizes the display function through the GPU, the display screen 194, and the application processor.
- the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
- Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
- the display screen 194 is used to display images, videos and the like.
- the display screen 194 includes a display panel.
- the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), etc.
- the electronic device can realize the shooting function through ISP, camera 193 , video codec, GPU, display screen 194 and application processor.
- the ISP is used for processing the data fed back by the camera 193 .
- the ISP may be located in the camera 193 .
- Camera 193 is used to capture still images or video.
- the object generates an optical image through the lens and projects it to the photosensitive element.
- the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
- CMOS complementary metal-oxide-semiconductor
- the photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP for conversion into a digital image signal.
- Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when an electronic device selects a frequency point, a digital signal processor is used to perform Fourier transform on the frequency point energy, etc.
- Video codecs are used to compress or decompress digital video.
- An electronic device may support one or more video codecs.
- the electronic device can play or record video in multiple encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
- the NPU is a neural-network (NN) computing processor.
- NPU neural-network
- Applications such as intelligent cognition of electronic devices can be realized through NPU, such as: image recognition, face recognition, speech recognition, text understanding, etc.
- the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device.
- the external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.
- the internal memory 121 may be used to store computer-executable program codes including instructions.
- the processor 110 executes various functional applications and data processing of the electronic device by executing instructions stored in the internal memory 121 .
- the internal memory 121 may include an area for storing programs and an area for storing data.
- the electronic device can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.
- the audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal.
- the audio module 170 may also be used to encode and decode audio signals.
- the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .
- Speaker 170A also referred to as a "horn" is used to convert audio electrical signals into sound signals.
- the electronic device can listen to music through speaker 170A, or listen to hands-free calls.
- Receiver 170B also called “earpiece” is used to convert audio electrical signals into sound signals.
- the electronic device receives a call or a voice message, it can listen to the voice by placing the receiver 170B close to the human ear.
- the microphone 170C also called “microphone” or “microphone” is used to convert sound signals into electrical signals.
- the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C.
- the earphone interface 170D is used for connecting wired earphones.
- the earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.
- OMTP open mobile terminal platform
- CTIA cellular telecommunications industry association of the USA
- the pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal.
- pressure sensor 180A may be disposed on display screen 194 .
- the gyro sensor 180B can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device about three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization.
- the air pressure sensor 180C is used to measure air pressure.
- the electronic device calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.
- the magnetic sensor 180D includes a Hall sensor.
- the electronic device may detect opening and closing of the flip holster using the magnetic sensor 180D.
- the acceleration sensor 180E can detect the acceleration of the electronic device in various directions (generally three axes).
- the distance sensor 180F is used to measure the distance.
- Electronic devices can measure distance via infrared or laser light. In some embodiments, when shooting a scene, the electronic device can use the distance sensor 180F to measure the distance to achieve fast focusing.
- the proximity light sensor 180G may include, for example, a light emitting diode (LED) and a photodetector,
- the ambient light sensor 180L is used for sensing ambient light brightness.
- the electronic device can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
- the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
- the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device is in the pocket to prevent accidental touch.
- the fingerprint sensor 180H is used to collect fingerprints. Electronic devices can use the collected fingerprint features to unlock fingerprints, access application locks, take pictures with fingerprints, answer incoming calls with fingerprints, etc.
- the temperature sensor 180J is used to detect temperature.
- the electronic device uses the temperature detected by the temperature sensor 180J to implement a temperature treatment strategy.
- Touch sensor 180K also known as "touch panel”.
- the touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
- the keys 190 include a power key, a volume key and the like.
- the key 190 may be a mechanical key. It can also be a touch button.
- the electronic device can receive key input and generate key signal input related to user settings and function control of the electronic device.
- the motor 191 can generate a vibrating reminder.
- the motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback.
- the indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.
- the SIM card interface 195 is used for connecting a SIM card.
- the SIM card can be inserted into the SIM card interface 195 or pulled out from the SIM card interface 195 to realize contact and separation with the electronic device.
- Fig. 4 is a block diagram of a software structure of an electronic device provided by an embodiment of the present application.
- the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces. In some embodiments, the system is divided into five layers, which are application program layer, application program framework layer, hardware abstraction layer (hardware abstraction layer, HAL), digital signal processing layer and kernel layer from top to bottom.
- layers which are application program layer, application program framework layer, hardware abstraction layer (hardware abstraction layer, HAL), digital signal processing layer and kernel layer from top to bottom.
- the application layer can consist of a series of application packages.
- the application package may include application programs (also called applications) such as SMS, gallery, camera, calendar, bluetooth, and map.
- application programs also called applications
- the voice assistant is the first system application, which can provide electronic equipment with a function of managing voice control.
- the voice assistant can include an application scenario message sending module and a command word processing module.
- the application scenario message delivery module is used to monitor which scenario the electronic device is currently in, and determine the identifier of the application program corresponding to the application scenario.
- the message that the machine is in the specific application scene is sent to the following dynamic loading module, and the identification of the application program corresponding to the scene is sent to Command word processing module.
- the specific application scenarios involved in the embodiments of the present application may at least include music scenes, video scenes, navigation scenes, and call scenarios, and may also include other application scenarios, which are not limited in the embodiments of the present application.
- the command word processing module is used to generate an instruction corresponding to the preset keyword according to the preset keyword. Then, according to the identification of the application program received from the application scenario message delivery module, the instruction is delivered to the application program corresponding to the identification.
- the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
- the application framework layer includes some predefined functions.
- the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, etc.
- a window manager is used to manage window programs.
- the window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc.
- Content providers are used to store and retrieve data and make it accessible to applications.
- Said data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc.
- the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.
- the view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on.
- the view system can be used to build applications.
- a display interface can consist of one or more views.
- a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.
- the hardware abstraction layer is an interface layer between the operating system and the hardware of the electronic device (such as the microphone 170C), and its purpose is to abstract the hardware and provide a virtual hardware platform for the operating system.
- the hardware abstraction layer may at least include a voiceprint recognition module and a secondary command word module.
- the second-level command word recognition module is used to receive voice data, and recognize that part of voice data that only includes preset keywords in the voice data, and then send the part of voice data that only includes preset keywords to the next Describe the voiceprint recognition module.
- the second-level command word recognition module is also used to record all corresponding preset keywords in the following specific application scenarios.
- the voiceprint recognition module is used to record the preset voiceprint, which is the voiceprint identification of the user's voice data recorded by the electronic device, and is used to identify the user's identity.
- the voiceprint recognition module is also used to perform voiceprint recognition on the part of the voice data that only includes preset keywords, so as to obtain the voiceprint of the voice data, and judge whether the voiceprint matches the preset voiceprint.
- the digital signal processing layer is used to process digital signals.
- the digital signal processing layer may at least include a specific application scenario module, a dynamic loading module, and a first-level command word recognition module.
- the specific application scene module is used to record the preset keyword group corresponding to the specific application scene, and may include a music scene module, a video scene module, a navigation scene module and a call scene module.
- the corresponding keyword group involved in the specific application scenario is a set of preset keywords corresponding to the specific application scenario.
- the video scene module is used for recording corresponding preset keyword groups in the video scene.
- the navigation scene module is used to record the corresponding preset keyword groups in the navigation scene.
- the call scene module is used to record the corresponding preset keyword groups in the call scene.
- the dynamic loading module is used for loading the preset keywords related to the specific application scene from the specific application scene module after receiving the message of which specific application scene the electronic device is in. And send the preset keywords involved in the specific application scenario to the first-level command word recognition module.
- the first-level command word recognition module is used to receive voice instructions sent by the kernel layer. It is also possible to obtain the preset keywords involved in the specific application scenarios recorded in the specific application scenario module, combine the preset keywords to identify the part of the voice data that includes the preset keywords in the voice command, and then send the part of the voice data to To the above-mentioned secondary command word recognition module.
- the kernel layer is the layer between hardware and software.
- the kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.
- the audio driver is used to receive voice commands collected by the microphone 170C, and send the voice commands to the first-level command word recognition module.
- Fig. 5 is a schematic diagram of an exemplary information flow of the voice control method involved in the present application.
- FIG. 5 is only some software modules in the electronic device, and should not limit the software architecture of the electronic device.
- the exemplary information flow schematic diagram shown in FIG. 5 describes that in a music scene, after the electronic device detects the user's first voice command, the electronic device recognizes the first preset keyword in the first voice command and responds to the first voice command.
- the first preset keyword is not a specific keyword for opening the voice interaction function;
- the application scene message sending module 151A in the first system application 151 of the electronic device can detect that the electronic device is currently in a music scene, and determine that the music application 152 (music class application) ID. Then send the message that the electronic device is currently in the music scene to the dynamic loading module 155, and send the identification of the music application program to the command word processing module 151B.
- the dynamic loading module 155 loads all preset keywords involved in the music scene from the music scene module 157A. Then, the dynamic loading module 155 can deliver all preset keywords involved in the music scene to the first-level command word recognition module 156 .
- the microphone 170C of the electronic device can collect voice commands, and transmit the voice commands to the audio driver 158 of the electronic device.
- the audio driver 158 of the electronic device may deliver the voice command to the primary command word recognition module 156 of the electronic device.
- the first-level command word recognition module 156 of the electronic device can judge whether the voice instruction includes the first voice data according to all preset keywords involved in the music scene, and the first voice data includes at least the first preset keyword .
- the first preset keyword is a preset keyword among all preset keywords involved in the music scene. If it includes the first voice data, then send the first voice data to the second-level command word recognition module 153 . If the first voice data is not included, no message will be sent to the second-level command word recognition module 153 .
- the second-level command word recognition module is based on the first voice data and all preset keywords involved in the application scene. Identify the second voice data in the first voice data, the second voice data only includes the first preset keyword, and send the second voice data to the voiceprint recognition module 154 .
- the voiceprint recognition module 154 is configured to perform voiceprint recognition on the second voice data. If the voiceprint of the second voice data matches the preset voiceprint, the second voice data is transmitted to the command word processing module 151B. If the voiceprint of the second voice data does not match the preset voiceprint, no message will be sent to the command word processing module 151B.
- the command word processing module 151B is used to generate an instruction corresponding to the preset keyword from the first preset keyword in the second voice data, and then according to the identification of the music application program received from the above-mentioned application scenario message sending module, Send the instruction to the music application program corresponding to the identifier.
- the music application program may execute a corresponding operation in response to the instruction. For example, if the first preset keyword included in the voice instruction is "play the next song", the music application program of the electronic device executes to play the next music.
- each module mentioned above may be a set of codes, a set of functions, hardware including one or more signal processing and/or application-specific integrated circuits, or a combination of one or more of them. This embodiment of the present application does not limit it.
- the DSP of the electronic device can process digital signals and its power consumption is low, then the electronic device can store the dynamic loading module 155, the first-level command word recognition module 156 and the specific application scenario module 157 in the DSP of the electronic device In the built-in memory, DSP calls the instructions corresponding to these functional modules to realize the functions of these modules. In this way, these functional modules will not waste the power consumption of the electronic device during operation, and other functional modules, such as the secondary command word recognition module, can be stored in the internal memory 121 of the electronic device, and these functions will be called by the processor 110 The instructions corresponding to the modules implement the functions of these modules.
- the electronic device sets some preset keywords for different specific application scenarios.
- the electronic device may execute the first operation in response to the voice command.
- the first operation is an operation corresponding to the first voice instruction.
- FIG. 6 is a schematic flow chart of the voice control method provided by the embodiment of the present application.
- the electronic device determines that it is currently in the first application scenario
- the first application scenario is a specific application scenario.
- the electronic device may not enable the voice interaction function, and may also implement voice control of the electronic device.
- the first application scene may be a music scene.
- FIG. 2a it is a music playing interface of an electronic device. Since the electronic device opens a music application program, the electronic device can determine that it is currently in a music scene.
- the electronic device can be set to start running the first application program, and then enters the first application scenario.
- the first application program may be a music application program, or other application programs, such as a video application program, a navigation application program, etc., which is not limited in this embodiment of the present application.
- an identifier corresponding to the application program may be obtained, and the identifier is used to uniquely represent an application program.
- the electronic device can record the first identification of the first application, and when the electronic device starts an application, it obtains the second identification corresponding to the application, and the electronic device checks whether the second identification is the same as the first identification, and if they are the same, then It is considered that the current electronic device starts to run the first application program, and it is determined that the electronic device is currently in the first application scenario.
- the electronic device can monitor the data output of the local device, and judge whether the local device has entered the first application scenario according to what data is output by the electronic device.
- the data output refers to the data transmitted by the electronic device to the user, for example, video data, audio data and so on.
- the electronic device can pre-set what kind of data the output data is and what kind of application scene the device is in. For example, the electronic device can be set to be in a music scene when the data output by the device is audio data, and to be in a video scene when the electronic device outputs video data.
- This step S101 can be completed by the application scenario message sending module 151A and the dynamic loading module 155 in FIG. 5 mentioned above.
- the application scenario message sending module 151A may determine that the electronic device is currently in the first application scenario and send the message to the dynamic loading module 155 .
- the application scenario message sending module 151A may determine that the electronic device is currently in the first application scenario and send the message to the dynamic loading module 155 .
- the electronic device loads a first preset keyword group corresponding to the first application scenario, where the first preset keyword group includes at least a first preset keyword;
- the first preset keyword group is a set of preset keywords involved in the first application scenario, wherein any preset keyword in the first preset keyword group may be referred to as a first preset keyword.
- the user may input a voice command including the first preset keyword to perform voice control on the electronic device.
- the first preset keyword may be "play the next song”.
- This step S102 can be completed by the dynamic loading module 155 and the specific application scenario module 157 in FIG. 5 .
- the dynamic loading module 155 can load the first preset keyword group corresponding to the first application scenario from the specific application scenario module 157 . And send it to the first-level command word recognition module 156.
- the dynamic loading module 155 can load the first preset keyword group corresponding to the first application scenario from the specific application scenario module 157 . And send it to the first-level command word recognition module 156.
- the electronic device detects the first voice command
- the user inputs a first voice command, the first voice command is "play the next song", and the electronic device can detect the first voice command.
- the user inputs a first voice command, the first voice command is "play the previous song", and the electronic device can detect the first voice command.
- the electronic device may detect voice information around the electronic device at a certain frequency, and the microphone of the electronic device may collect voice data around the electronic device, including the first voice command.
- the first voice instruction may only include the first preset keyword, or may include other voice data.
- the situation that the first voice data also includes other voice data can be divided into the following three types:
- the first voice command detected by the electronic device may be a sentence, and the first voice command may include other voice data in addition to the first preset keyword.
- the first voice command can be: "XXX, play the next song XXXX", then in the first voice command, "play the next song” is the first preset keyword, but other voice data is not preset Key words.
- the first voice instruction may also include one or more other preset keywords, such as the second preset keyword and other voice data.
- the second preset keyword is a preset keyword corresponding to the first application scene, appears after the first preset keyword, and may or may not be the same as the first preset keyword.
- the first voice command can be: "XXXX plays the next song and pauses XXXX", then in the first voice command, the first preset keyword can be "play the next song", and the second preset keyword can be is "play", but other voice data are not preset keywords.
- the first voice instruction does not include any preset keywords.
- This step S103 can be completed by the aforementioned audio driver 158 in FIG. 5 .
- the audio driver 158 can detect the first voice command and send it to the primary command word recognition module 156 .
- the audio driver 158 can detect the first voice command and send it to the primary command word recognition module 156 .
- the electronic device determines whether the first voice command includes first voice data according to the first voice command and the first preset keyword group, and the first voice data includes at least the first preset keyword;
- the first voice data is a part of the first voice instruction, which at least includes the first preset keyword.
- the length of the first voice command is t seconds.
- the electronic device Starting from second 0 of the first voice command, the electronic device sequentially acquires voice data with a length of m seconds, where m is smaller than t.
- the voice data with a length of m seconds corresponds to one character. If the first preset keyword is included in the recently acquired N continuous voice data with a length of m seconds, the electronic device will not acquire the next voice data with a length of m seconds, and directly use the newly acquired continuous N voice data with a length of The voice data of m seconds is determined as the first voice data.
- X can generally be set to 1.5-2.5, such as 2.
- the electronic device may determine the M pieces of voice data with a length of m seconds as the first voice data.
- n seconds there is an overlap of n seconds between two consecutive voice data with a length of m seconds, and n is less than m, that is, the voice data of the last n seconds in the previous voice data with a length of m seconds is the next Voice data of length m seconds starts with n seconds of voice data.
- the process for the electronic device to determine that the first preset keyword is included in N consecutive speech data with a length of m seconds is as follows: First, the electronic device obtains the i-th speech data with a length of m seconds from the first speech instruction voice data. in.
- the first voice data with a length of m seconds is the voice data with a length of 0 seconds to m seconds in the first voice instruction.
- the electronic device judges whether the character corresponding to the i-th voice data with a length of m seconds is the same as the j-th character in the first preset keyword.
- the electronic device sets j to 1, that is, obtains the first preset The first character in the keyword, then, the electronic device judges whether the character corresponding to the i-th voice data with a length of m seconds is the same as the first character in the first preset keyword; if the i-th length is m The character corresponding to the second voice data is different from the jth character in the first preset keyword and j is equal to 1, then the electronic device continues to obtain the i+1th voice data with a length of m seconds, and then the electronic device judges Whether the character corresponding to the (i+1)th speech data with a length of m seconds is the same as the first character in the first preset keyword.
- the electronic device continues to obtain the i+1-th speech data with a length of m seconds, and obtains the first The j+1th character in a preset keyword, and then, the electronic device judges whether the character corresponding to the i+1th voice data with a length of m seconds is equal to the j+1th character in the first preset keyword same.
- FIG. 7 it is a schematic diagram of determining the first voice data.
- the electronic device sets the first voice data as voice data with a length of six consecutive m seconds.
- the electronic device continues to obtain the fifth (i+1) character “sound” corresponding to the voice data with a length of m seconds, and obtains the second (j+1) character “sound” in the first preset keyword.
- the electronic device judges that the character “sheng” is the same as the character “sheng”.
- the electronic device determines that the six most recently acquired voice data with a length of m seconds "play a little louder" as the first data.
- step S104 after the electronic device recognizes the first voice data according to the method described in step S104, it can continue to judge whether the voice data that has not been judged in the first voice instruction still contains another first voice data.
- the electronic device can use it to execute the following steps S105-step S108.
- This step S104 can be completed by the above-mentioned primary command word recognition module 156 in FIG. 5 .
- the primary command word recognition module 156 can determine the first voice data according to the first voice command detected by the audio driver 158 and the first preset keyword group issued by the dynamic loading module 155 . And send it to the secondary command word recognition module 153.
- the algorithm involved in determining the first voice data in the first-level command word recognition module 156 is simple, and the process is completed by calling these function modules from the DSP.
- the electronic device determines the second voice data according to all the preset keywords corresponding to the first voice data and the specific application scene, and the second voice data only includes the first preset keywords;
- step S105 For the process of determining the second voice data according to all the preset keywords corresponding to the specific application scene in the first voice data in the step S105, reference may be made to the related description of the aforementioned step S104.
- This step S105 can be completed by the aforementioned secondary command word recognition module 153 in FIG. 5 .
- the secondary command word recognition module 153 can determine the second voice data according to all preset keywords corresponding to the first voice data issued by the primary command word recognition module 156 and its recorded specific application scenarios. And send it to the voiceprint recognition module 154.
- the algorithm involved in determining the second voice data in the secondary command word recognition module 153 is more complex than the algorithm determining the first voice data in the primary command word recognition module 156, and this process is called by the processor. Finish.
- the electronic device judges whether the voiceprint of the second voice data matches the preset voiceprint
- the preset voiceprint is the voiceprint identification of the user's voice data recorded by the electronic device, and is used to identify the user's identity.
- the electronic device can use the voiceprint extracted from the second voice data to match the preset voiceprint. If they are consistent, the electronic device can determine that the second voice data comes from a user who matches the preset voiceprint, and can match the first voiceprint. In response to the voice command, step S107 is executed.
- the electronic device determines that the second voice data is not from a user that matches the preset voiceprint, and may not respond to the first voice command, and executes step S108.
- This step S106 can be completed by the aforementioned voiceprint recognition module 154 in FIG. 5 .
- the voiceprint recognition module 154 can determine whether the second voice data is from a user matching the preset voiceprint according to the second voice data issued by the secondary command word recognition module 153 and the preset voiceprint recorded therein.
- the voiceprint recognition module 154 can determine whether the second voice data is from a user matching the preset voiceprint according to the second voice data issued by the secondary command word recognition module 153 and the preset voiceprint recorded therein.
- this step S106 is optional, and the electronic device may directly perform step S107 without performing step S106 after performing step S105.
- the electronic device responds to the first voice instruction, and performs an operation corresponding to the first preset keyword in the second voice data;
- the user interface shown in FIG. 2b and the user interface shown in FIG. 2d are two user interfaces for the electronic device to respond to the first voice instruction. It can be seen that the electronic device displays the user interface after performing the operation corresponding to the first preset keyword.
- the electronic device generates an operation corresponding to the preset keyword according to the first preset keyword in the second voice data, and executes the operation corresponding to the first preset keyword in the second voice data in response to the first voice command.
- This step S107 can be completed by the aforementioned command word processing module 151B in FIG. 5 .
- the command word processing module 151B can convert the second voice data issued by the secondary command word recognition module 153 into the operation corresponding to the second voice data and issue it to the application program involved in the first application scenario, the application program The operation is performed in response to the first voice instruction.
- the command word processing module 151B can convert the second voice data issued by the secondary command word recognition module 153 into the operation corresponding to the second voice data and issue it to the application program involved in the first application scenario, the application program The operation is performed in response to the first voice instruction.
- the electronic device does not respond to the first voice instruction
- the electronic device After the user inputs the first voice command, the electronic device does not display the user interface after performing the operation corresponding to the first preset keyword, that is, the electronic device does not respond to the first voice command.
- the displayed user interface when the electronic device is in the first application scenario, the displayed user interface may be referred to as the first user interface, and the displayed user interface after responding to the first voice instruction may be referred to as the second user interface.
- the specific application scenario is preset by the electronic device.
- the electronic device is set to enter a video scene when the machine opens a video application program.
- the electronic device can set the video scene as a specific application scene.
- the electronic device can respond to the non-specific voice command and perform the first operation,
- the non-specific voice instruction may be "sound louder”.
- the first operation is an operation corresponding to the non-specific voice instruction. For example, when the non-specific voice instruction is "sound louder", the electronic device may set the sound of the video to be played louder.
- the internal memory 121 of the electronic device or the storage device connected to the external storage interface 120 may pre-store preset keywords, related instructions and preset voiceprints involved in the voice control method involved in the embodiment of the present application.
- the electronic device sets a memory in the DSP, and stores preset keywords involved in a specific application scene therein, so that the electronic device executes the voice control method in the embodiment of the present application.
- step S101-step S108 takes step S101-step S108 as an example to illustrate the workflow of the electronic device.
- the electronic device determines that it is currently in the first application scenario
- the touch sensor 180K of the electronic device receives a touch operation (triggered when the user touches the camera control), and a corresponding hardware interrupt is sent to the kernel layer.
- the kernel layer processes touch operations into original input events (including touch coordinates, time stamps of touch operations, and other information). Raw input events are stored at the kernel level.
- the application framework layer obtains the original input event from the kernel layer, and identifies the application of the control corresponding to the input event.
- the above touch operation is a touch click operation
- the control corresponding to the click operation is an icon of a music application program.
- the music application program calls the interface of the application framework layer, starts the music application program, and then determines that it is currently in the music scene.
- the electronic device loads a first preset keyword group corresponding to the first application scenario, where the first preset keyword group includes at least the first preset keyword.
- the DSP processor of the electronic device may load the stored first preset keyword group corresponding to the first application scenario from the built-in memory of the DSP.
- the electronic device detects the first voice instruction.
- the electronic device can detect voice information around the electronic device at a certain frequency, and the microphone of the electronic device can collect voice data around the electronic device, including the first voice command. And store the detected first voice instruction in the internal memory 121 or in a storage device connected to the external storage interface 120 .
- the electronic device determines the first voice data according to the first voice command and the first preset keyword group.
- the electronic device can acquire the first voice instruction and the first preset keyword group from the memory through the DSP, and call related computer instructions to determine the first voice data. And store the first voice data in the internal memory 121 or in a storage device connected to the external storage interface 120 .
- the electronic device determines the second voice data according to all preset keywords corresponding to the first voice data and specific application scenarios.
- the electronic device can acquire all preset keywords corresponding to the first voice data and specific application scenarios from the internal memory 121 through the processor 110, and call related computer instructions to determine the second voice data. And store the second voice data in the internal memory 121 or in a storage device connected to the external storage interface 120 .
- the electronic device judges whether the voiceprint of the second voice data matches the preset voiceprint.
- the electronic device can obtain the second voice data and the preset voiceprint matching from the internal memory 121 through the processor 110, and invoke relevant computer instructions to determine whether the voiceprint of the second voice data matches the preset voiceprint.
- the electronic device responds to the first voice instruction, and executes an operation corresponding to the first preset keyword in the second voice data.
- the electronic device may call the interface of the application framework layer to execute the operation corresponding to the first preset keyword in the second voice data. Furthermore, the display driver is started by invoking the kernel layer, and the user interface after performing the operation corresponding to the first preset keyword in the second voice data is displayed.
- the electronic device can receive non-specific voice commands input by the user in some specific application scenarios without turning on the voice interaction function, and respond to the non-specific voice commands .
- the term “when” may be interpreted to mean “if” or “after” or “in response to determining" or “in response to detecting" depending on the context.
- the phrases “in determining” or “if detected (a stated condition or event)” may be interpreted to mean “if determining" or “in response to determining" or “on detecting (a stated condition or event)” or “in response to detecting (a stated condition or event)”.
- all or part of them may be implemented by software, hardware, firmware or any combination thereof.
- software When implemented using software, it may be implemented in whole or in part in the form of a computer program product.
- the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part.
- the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices.
- the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, DSL) or wireless (eg, infrared, wireless, microwave, etc.) means.
- the computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media.
- the available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state hard disk), etc.
- the processes can be completed by computer programs to instruct related hardware.
- the programs can be stored in computer-readable storage media.
- When the programs are executed may include the processes of the foregoing method embodiments.
- the aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk, and other various media that can store program codes.
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- User Interface Of Digital Computer (AREA)
- Telephone Function (AREA)
Abstract
一种语音控制方法及电子设备。该方法包括:电子设备确定当前处于第一应用场景;电子设备检测到第一语音指令;第一语音指令为不用于打开语音交互功能的非特定语音指令;电子设备确定第一语音指令中是否包括第一预设关键词;在确定第一语音指令中包括第一预设关键词的情况下,电子设备不打开语音交互功能,响应第一语音指令,执行第一预设关键词对应的操作;在确定第一语音指令中不包括第一预设关键词的情况下,电子设备不响应第一语音指令。该方法所提供的技术方案,在一些特定应用场景中,电子设备不需要打开语音交互功能,也可以接收用户输入的非特定语音指令,并确定该非特定语音指令中的预设关键词,电子设备可以响应该非特定语音指令,执行该预设关键词对应的操作。
Description
本申请要求于2021年06月29日提交中国专利局、申请号为202110728832.5、申请名称为“一种语音控制方法及电子设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及终端及人工智能领域,尤其涉及一种语音控制方法及电子设备。
随着电子设备的发展,已经实现用户可以通过语音控制电子设备了。一种实现语音控制的典型代表就是语音交互。现阶段,越来越多的电子设备具有语音交互功能,配置有可以与用户进行语音交互的应用,例如语音助手。用户可以通过语音助手与电子设备进行语音交互,来实现以前需要多次手动操作才能实现的功能。例如,打电话、播放音乐等。浪费电子设备的功耗。
目前,电子设备要先检测到开启语音交互功能的输入,电子设备才能实现语音交互功能。该电子设备开启语音交互功能的过程较繁琐。如何简化用户通过语音控制电子设备的流程是当前研究的主要方向。
发明内容
本申请提供了一种语音控制方法及电子设备,在一些特定应用场景中,电子设备不需要打开语音交互功能,也可以接收用户输入的非特定语音指令,并响应该非特定语音指令。
第一方面,本申请提供了一种语音控制方法,该方法包括:电子设备确定当前处于第一应用场景;该电子设备检测到第一语音指令;该第一语音指令为不用于打开语音交互功能的非特定语音指令;该电子设备确定该第一语音指令中是否包括第一预设关键词;该第一预设关键词为该第一应用场景对应的全部预设关键词中的任一预设关键词;在确定该第一语音指令中包括第一预设关键词的情况下,该电子设备不打开语音交互功能,响应该第一语音指令,执行该第一预设关键词对应的操作;在确定该第一语音指令中不包括第一预设关键词的情况下,该电子设备不响应该第一语音指令。
实施第一方面的方法,电子设备不需要打开语音交互功能,也可以接收用户输入的非特定语音指令,并响应该非特定语音指令。通常,打开语音交互是需要消耗电子设备的功耗的,且语音交互状态下也需要消耗电子设备的功耗,采用该方法,可以不要大考语音交互,也能实现语音控制电子设备,这样可以节约电子设备的功耗。
结合第一方面,在一种实施方式中,电子设备确定当前处于第一应用场景之前,该方法还包括:该电子设备显示第一用户界面;该电子设备不打开语音交互功能,响应该第一语音指令,执行该第一预设关键词对应的操作,具体包括:该电子设备不打开语音交互功能,响应该第一语音指令,执行该第一预设关键词对应的操作,显示第二用户界面,该第二用户界面与该第一用户界面不同。
在上述实施例中,在一些特定应用场景下,电子设备可以根据语音指令更改电子设备的用户界面,使得电子设备可以为用户提供一些可视化服务。例如电子设备打开视频应用时,电子设备可以响应该第一语音指令,为用户提供视频播放的相关功能,不需要用户手动触摸电子设备的显示屏,即可实现对电子设备的控制。
结合第一方面,在一种实施方式中,根据权利要求1该的方法,其特征在于,该电子设备确定当前处于第一应用场景,具体包括:该电子设备前台运行第一应用程序,则该电子设备确定当前处于第一应用场景。
在上述实施例中,对于前台运行特定应用的情况,电子设备也可以确定当前处于特定应用场景,可以实现非特定语音指令语音控制电子设备。电子设备可以前台执行响应操作,实现语音控制电子设备。例如,在电子设备打开视频应用时,第一语音指令为:“播放下一集”,电子设备可以前台执行该操作,播放下一集。
结合第一方面,在一种实施方式中,该电子设备确定当前处于第一应用场景,具体包括:
该电子设备后台运行第一应用程序,则该电子设备确定当前处于第一应用场景。
在上述实施例中,对于后台运行特定应用的情况,电子设备也可以确定当前处于特定应用场景,可以实现非特定语音指令语音控制电子设备。电子设备可以后台执行响应操作,实现语音控制电子设备。例如,在电子设备打开音乐应用时,第一语音指令为:“播放声音大一点”,电子设备可以后台执行该操作,不涉及用户界面的改变,不影响用户使用前台正在运行的电子设备。
结合第一方面,在一种实施方式中,该电子设备确定该第一语音指令中是否包括第一预设关键词,具体包括:该电子设备加载该第一应用场景对应的全部预设关键词;该电子设备根据该第一语音指令与该第一应用场景对应的全部预设关键词,确定第一语音指令中是否包括第一语音数据,该第一语音数据中至少包括该第一预设关键词;在确定该第一语音指令中包括第一语音数据的情况下,该电子设备确定该第一语音指令中包括第一预设关键词;在确定该第一语音指令中不包括第一语音数据的情况下,该电子设备确定该第一语音指令中不包括第一预设关键词。
在上述实施例中,电子设备首先确定第一语音指令中是否包括第一语音数据,该过程涉及的算法不需要太复杂,只要可以确定出第一语音即可,电子设备可以利用数字信号处理器完成该过程,节约电子设备的计算资源。
结合第一方面,在一种实施方式中,该电子设备确定第一语音指令中包括第一语音数据之后,该方法还包括:该电子设备加载全部特定应用场景对应的全部预设关键词,该全部特定应用场景包括该第一应用场景,该全部预设关键词中包括第一预设关键词;该电子设备根据第一语音数据与该全部特定应用场景对应的全部预设关键词,确定第一语音数据中的一部分作为第二语音数据,该第二语音数据只包括第一预设关键词;响应该第一语音指令,执行该第一预设关键词对应的操作,具体包括:该电子设备根据该第二语音数据中的第一预设关键词生成该第一预设关键词对应的操作;响应该第一语音指令,执行该第一预设关键词对应的操作。
在上述实施例中,当电子设备通过比较简单的算法确定该第一语音指令中包括预设关 键词之后,电子设备可以通过比较精确的算法,提取该第一语音指令中的预设关键词,并利用该第一语音指令中的预设关键词生成相应的操作,电子设备可以响应该操作,实现语音控制电子设备。
结合第一方面,在一种实施方式中,该电子设备确定第一语音指令中包括第一语音数据之后,该方法还包括:该电子设备加载全部特定应用场景对应的全部预设关键词,该全部特定应用场景包括该第一应用场景,该全部预设关键词中包括第一预设关键词;该电子设备根据第一语音数据与该全部特定应用场景对应的全部预设关键词,确定第一语音数据中的一部分作为第二语音数据,该第二语音数据只包括第一预设关键词;该电子设备确定该第二语音数据的声纹是否与预设声纹匹配,该预设声纹为该电子设备录入的用户的语音数据的声纹标识,用于标识用户的身份;响应该第一语音指令,执行该第一预设关键词对应的操作,具体包括:在确定该第二语音数据的声纹与预设声纹匹配的情况下,该电子设备根据该第二语音数据中的第一预设关键词生成该第一预设关键词对应的操作;响应该第一语音指令,执行该第一预设关键词对应的操作;在确定该第二语音数据的声纹与预设声纹不匹配的情况下,不响应该第一语音指令。
在上述实施例中,电子设备在识别出第一语音指令中的预设关键词之后,还需要再判断一下该第一语音指令是否为电子设备的“拥有者”输入的。通常,电子设备的“拥有者”会通过电子设备输入自己的生物信息,例如声纹,则电子设备根据第一语音指令的声纹以及“拥有者”通过电子设备输入的声纹进行匹配即可确定该一语音指令是否为电子设备的“拥有者”输入的。若是才可以响应该第一语音指令,这样可避免任何人都可以语音控制电子设备,增加了实施该方法的安全性。
结合第一方面,在一种实施方式中,该特定应用场景是预先设置好,存储在电子设备中的,该特定应用场景至少包括一个应用运行时的场景,该第一应用场景为任一该特定应用场景。
结合第一方面,在一种实施方式中,该该特定应用场景对应的全部预设关键词是预先设置好,存储在电子设备中的,所述特定应用场景对应的全部预设关键词中任一预设关键词都对应一个与该预设关键词同意义的操作。
第二方面,本申请提供了一种电子设备,该电子设备包括:一个或多个处理器和存储器;该存储器与该一个或多个处理器耦合,该存储器用于存储计算机程序代码,该计算机程序代码包括计算机指令,该一个或多个处理器调用该计算机指令以使得该电子设备执行:确定当前处于第一应用场景;检测到第一语音指令;该第一语音指令为不用于打开语音交互功能的非特定语音指令;确定该第一语音指令中是否包括第一预设关键词;该第一预设关键词为该第一应用场景对应的全部预设关键词中的任一预设关键词;在确定该第一语音指令中包括第一预设关键词的情况下,不打开语音交互功能,响应该第一语音指令,执行该第一预设关键词对应的操作;在确定该第一语音指令中不包括第一预设关键词的情况下,不响应该第一语音指令。
上述实施例中,电子设备不需要打开语音交互功能,也可以接收用户输入的非特定语音指令,并响应该非特定语音指令。通常,打开语音交互是需要消耗电子设备的功耗的, 且语音交互状态下也需要消耗电子设备的功耗,采用该方法,可以不要大考语音交互,也能实现语音控制电子设备,这样可以节约电子设备的功耗。
结合第二方面,在一种实施方式中,该一个或多个处理器还用于调用该计算机指令以使得该电子设备执行:显示第一用户界面;该一个或多个处理器具体用于调用该计算机指令以使得该电子设备执行:不打开语音交互功能,响应该第一语音指令,执行该第一预设关键词对应的操作,显示第二用户界面,该第二用户界面与该第一用户界面不同。
在上述实施例中,在一些特定应用场景下,电子设备可以根据语音指令更改电子设备的用户界面,使得电子设备可以为用户提供一些可视化服务。例如电子设备打开视频应用时,电子设备可以响应该第一语音指令,为用户提供视频播放的相关功能,不需要用户手动触摸电子设备的显示屏,即可实现对电子设备的控制。
结合第二方面,在一种实施方式中,该一个或多个处理器具体用于调用该计算机指令以使得该电子设备执行:前台运行第一应用程序,则确定当前处于第一应用场景。
在上述实施例中,对于前台运行特定应用的情况,电子设备也可以确定当前处于特定应用场景,可以实现非特定语音指令语音控制电子设备。电子设备可以前台执行响应操作,实现语音控制电子设备。例如,在电子设备打开视频应用时,第一语音指令为:“播放下一集”,电子设备可以前台执行该操作,播放下一集。
结合第二方面,在一种实施方式中,该一个或多个处理器具体用于调用该计算机指令以使得该电子设备执行:后台运行第一应用程序,则确定当前处于第一应用场景。
在上述实施例中,对于后台运行特定应用的情况,电子设备也可以确定当前处于特定应用场景,可以实现非特定语音指令语音控制电子设备。电子设备可以后台执行响应操作,实现语音控制电子设备。例如,在电子设备打开音乐应用时,第一语音指令为:“播放声音大一点”,电子设备可以后台执行该操作,不涉及用户界面的改变,不影响用户使用前台正在运行的电子设备。
结合第二方面,在一种实施方式中,该一个或多个处理器具体用于调用该计算机指令以使得该电子设备执行:加载该第一应用场景对应的全部预设关键词;根据该第一语音指令与该第一应用场景对应的全部预设关键词,确定第一语音指令中是否包括第一语音数据,该第一语音数据中至少包括该第一预设关键词;在确定该第一语音指令中包括第一语音数据的情况下,确定该第一语音指令中包括第一预设关键词;在确定该第一语音指令中不包括第一语音数据的情况下,确定该第一语音指令中不包括第一预设关键词。
在上述实施例中,电子设备首先确定第一语音指令中是否包括第一语音数据,该过程涉及的算法不需要太复杂,只要可以确定出第一语音即可,电子设备可以利用数字信号处理器完成该过程,节约电子设备的计算资源。
结合第二方面,在一种实施方式中,该一个或多个处理器还用于调用该计算机指令以使得该电子设备执行:加载全部特定应用场景对应的全部预设关键词,该全部特定应用场景包括该第一应用场景,该全部预设关键词中包括第一预设关键词;根据第一语音数据与该全部特定应用场景对应的全部预设关键词,确定第一语音数据中的一部分作为第二语音数据,该第二语音数据只包括第一预设关键词;该一个或多个处理器具体用于调用该计算机指令以使得该电子设备执行:根据该第二语音数据中的第一预设关键词生成该第 一预设关键词对应的操作;响应该第一语音指令,执行该第一预设关键词对应的操作。
在上述实施例中,当电子设备通过比较简单的算法确定该第一语音指令中包括预设关键词之后,电子设备可以通过比较精确的算法,提取该第一语音指令中的预设关键词,并利用该第一语音指令中的预设关键词生成相应的操作,电子设备可以响应该操作,实现语音控制电子设备。
结合第二方面,在一种实施方式中,该一个或多个处理器还用于调用该计算机指令以使得该电子设备执行:加载全部特定应用场景对应的全部预设关键词,该全部特定应用场景包括该第一应用场景,该全部预设关键词中包括第一预设关键词;根据第一语音数据与该全部特定应用场景对应的全部预设关键词,确定第一语音数据中的一部分作为第二语音数据,该第二语音数据只包括第一预设关键词;确定该第二语音数据的声纹是否与预设声纹匹配,该预设声纹为该电子设备录入的用户的语音数据的声纹标识,用于标识用户的身份;该一个或多个处理器具体用于调用该计算机指令以使得该电子设备执行:在确定该第二语音数据的声纹与预设声纹匹配的情况下,根据该第二语音数据中的第一预设关键词生成该第一预设关键词对应的操作;响应该第一语音指令,执行该第一预设关键词对应的操作;在确定该第二语音数据的声纹与预设声纹不匹配的情况下,不响应该第一语音指令。
在上述实施例中,电子设备在识别出第一语音指令中的预设关键词之后,还需要再判断一下该第一语音指令是否为电子设备的“拥有者”输入的。通常,电子设备的“拥有者”会通过电子设备输入自己的生物信息,例如声纹,则电子设备根据第一语音指令的声纹以及“拥有者”通过电子设备输入的声纹进行匹配即可确定该一语音指令是否为电子设备的“拥有者”输入的。若是才可以响应该第一语音指令,这样可避免任何人都可以语音控制电子设备,增加了实施该方法的安全性。
第三方面,本申请实施例提供了一种电子设备,该电子设备包括一个或多个处理器和一个或多个存储器;其中,该一个或多个存储器与该一个或多个处理器耦合,该一个或多个存储器用于存储计算机程序代码,该计算机程序代码包括计算机指令,当该一个或多个处理器执行该计算机指令时,该一个或多个处理器用于调用该计算机指令以使得该电子设备执行如第一方面或第一方面的任意一种实施方式所描述的方法。
上述实施例中,电子设备不需要打开语音交互功能,也可以接收用户输入的非特定语音指令,并响应该非特定语音指令。通常,打开语音交互是需要消耗电子设备的功耗的,且语音交互状态下也需要消耗电子设备的功耗,采用该方法,可以不要大考语音交互,也能实现语音控制电子设备,这样可以节约电子设备的功耗。
第四方面,本申请实施例提供了一种芯片***,该芯片***包括一个或多个处理器,该处理器用于调用计算机指令以使得该电子设备执行如第一方面或第一方面的任意一种实施方式所描述的方法。
上述实施例中,电子设备不需要打开语音交互功能,也可以接收用户输入的非特定语音指令,并响应该非特定语音指令。通常,打开语音交互是需要消耗电子设备的功耗的, 且语音交互状态下也需要消耗电子设备的功耗,采用该方法,可以不要大考语音交互,也能实现语音控制电子设备,这样可以节约电子设备的功耗。
第五方面,本申请实施例提供了一种包含指令的计算机程序产品,当该计算机程序产品在电子设备上运行时,使得该电子设备执行如第一方面或第一方面的任意一种实施方式所描述的方法。
上述实施例中,电子设备不需要打开语音交互功能,也可以接收用户输入的非特定语音指令,并响应该非特定语音指令。通常,打开语音交互是需要消耗电子设备的功耗的,且语音交互状态下也需要消耗电子设备的功耗,采用该方法,可以不要大考语音交互,也能实现语音控制电子设备,这样可以节约电子设备的功耗。
第六方面,本申请实施例提供了一种计算机可读存储介质,当该指令在电子设备上运行时,使得该电子设备执行如第一方面或第一方面的任意一种实施方式所描述的方法。
上述实施例中,电子设备不需要打开语音交互功能,也可以接收用户输入的非特定语音指令,并响应该非特定语音指令。通常,打开语音交互是需要消耗电子设备的功耗的,且语音交互状态下也需要消耗电子设备的功耗,采用该方法,可以不要大考语音交互,也能实现语音控制电子设备,这样可以节约电子设备的功耗。
图1a-图1d是一种方案中用户通过语音控制电子设备的一组用户界面;
图2a-图2d为本申请提的用户通过语音控制电子设备的一组示例性用户界面;
图3是本申请实施例提供的电子设备的结构示意图;
图4是本申请实施例提供的电子设备的软件结构框图;
图5为本申请中涉及的语音控制方法的一个示例性信息流示意图;
图6为本申请实施例提供的语音控制方法的一个示意性流程图;
图7为本申请实施例中确定第一语音数据的一个示意图。
本申请以下实施例中所使用的术语只是为了描述特定实施例的目的,而并非旨在作为对本申请的限制。如在本申请的说明书和所附权利要求书中所使用的那样,单数表达形式“一个”、“一种”、“所述”、“上述”、“该”和“这一”旨在也包括复数表达形式,除非其上下文中明确地有相反指示。还应当理解,本申请中使用的术语“和/或”是指并包含一个或多个所列出项目的任何或所有可能组合。
以下,术语“第一”、“第二”仅用于描述目的,而不能理解为暗示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征,在本申请实施例的描述中,除非另有说明,“多个”的含义是两个或两个以上。
本申请以下实施例中的术语“用户界面(user interface,UI)”,是应用程序或操作***与用户之间进行交互和信息交换的介质接口,它实现信息的内部形式与用户可以接受形式之间的转换。用户界面是通过java、可扩展标记语言(extensible markup language,XML)等特定计算机语言编写的源代码,界面源代码在电子设备上经过解析,渲染,最终呈现为用户可以识别的内容。用户界面常用的表现形式是图形用户界面(graphic user interface,GUI),是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的文本、图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。
下面首先介绍本申请中涉及的相关术语。
(1)预设关键词
本申请实施中,预设关键词是预先设置好,存储在电子设备中的。任一预设关键词都对应一个与该预设关键词同意义的操作。例如,在特定应用场景为音乐场景时,当预设关键词为“暂停”时。则表示该预设关键词对应的操作为暂停音乐播放。当电子设备确定语音指令中包括某一预设关键词时则可以实现对电子设备的语音控制。
其中,特定应用场景是预先设置的,电子设备运行某一应用程序,则对应一个应用场景。特定应用场景则是可以对电子设备进行语音控制的应用场景,可以设定电子设备运行(包括后台运行与前台运行)第一应用程序时,电子设备则进入该第一应用程序对应的特定应用场景,例如,该第一应用程序可以为音乐类应用程序、视频类应用程序、导航类应用程序以及通话类应用程序等。
电子设备运行音乐类应用程序时为音乐场景,音乐场景中对应的预设关键词可以包括:“播放下一首”、“播放上一首”、“声音大一点”、“声音小一点”、“暂停”等常用的关键词。
电子设备运行视频类应用程序时为音乐场景,视频场景中对应的预设关键词可以包括:“播放下一首”、“播放上一首”、“声音大一点”、“声音小一点”、“暂停”等常用的关键词。
电子设备运行导航类应用程序时为音乐场景,导航场景中对应的预设关键词可以包括:“导航回家”、“导航上班”、“声音大一点”、“声音小一点”等常用的关键词。
电子设备运行通话类应用程序时为音乐场景,通话场景中对应的预设关键词可以包括:“声音大一点”、“声音小一点”等常用的关键词。
可以理解的是,本申请中还可以包括其他的特定应用场景,每个特定应用场景中可以包括比上述更多或者更少的预设关键词。
下面介绍一种方案中涉及的语音控制方法,在该方案中,电子设备要先检测到用户打开语音交互功能的输入才能打开语音交互功能,使得用户可以通过语音控制电子设备。其中,打开语音交互功能的输入可以是预设的特定语音指令,例如“YOYO,YOYO”,也可以是长按电源键等。
在一些实施例中,该特定语音指令也可以被称为唤醒词或者唤醒指令。
图1a-图1d示出了该方案中用户通过语音控制电子设备的一组用户界面。
如图1a所示,电子设备可以显示用户界面11,该用户界面11可以为电子设备的一个音乐播放界面,电子设备当前播放的是“第一音乐”。此时,电子设备可以检测到用户打开 语音交互功能的特定语音指令:“YOYO,YOYO”。响应于该特定语音指令,电子设备可以显示如图1b所示的用户界面12。
如图1b所示,用户界面12中可以包括语音提示框121,该语音提示框121中可以显示提示信息121A:“你说,我在听…”以及语音采集标识121B,该提示信息121A以及语音采集标识121B可以用于提示用户:当前电子设备可以接收语音指令,用户可以语音控制电子设备了。
如图1c所示,用户界面13可以为电子设备播放音乐时检测到用户输入的第一语音指令时的一个用户界面,该第一语音指令为非特定语音指令,非特定语音指令是指不同于特定语音指令的其他语音指令。此时,电子设备可以将检测到的第一语音指令显示到用户界面13所示的语音提示框131中。例如,用户输入的第一语音指令可以为:“播放下一首”。响应于该第一语音指令,电子设备可以执行播放下一首音乐,显示如图1d所示的用户界面14。
如图1d所示的用户界面14中,电子设备播放的音乐,已经从“第一音乐”切换到了“第二音乐”。
在该方案中,电子设备一直打开语音交互功能检测用户的语音指令会浪费电子设备的功耗,所以电子设备通常会设置在响应完一次非特定语音指令之后,就关闭语音交互功能。则下一次用户需要与电子设备进行语音交互时,有需要重新检测到特定语音指令,才能重新打开语音交互功能,使得用户可以再次语音控制电子设备。
如图1d所示,此时电子设备在响应了第一语音指令之后,就会关闭语音交互功能,则当用户输入第二语音指令:“播放上一首”时,电子设备不会响应该第二语音指令,仍然显示用户界面14。该第二语音指令为非特定语音指令。
这样,为了节约电子设备的功耗同时满足用户可以语音控制电子设备的需求,则需要用户在每一次输入非特定语音指令之前,都要打开语音交互功能,才能使得用户可以语音控制电子设备。例如,在输入第一定语音指令之前,先输入特定语音指令,才能使得电子设备响应该第一语音指令,使得用户可以语音控制电子设备。
为了解决上述方案中的缺陷,本申请实施例提供了一种语音控制方法,实施本申请实施例中的语音控制方法,在一些特定应用场景中,电子设备不需要打开语音交互功能,也可以接收用户输入的非特定语音指令,并响应该非特定语音指令。
在本申请实施例中,电子设备运行音乐类应用程序的应用场景为音乐场景,运行视频类应用程序的应用场景为视频场景,运行导航类应用程序的应用场景为导航场景,运行通话类应用程序的应用场景为通话场景。
非特定指令是指不同于特定语音指令的其他语音指令,不用于打开电子设备的语音交互功能。
电子设备可以针对不同的特定应用场景分别设置一些预设关键词,当电子设备检测到用户输入的语音指令中包括该预设关键词时,则电子设备可以响应该语音指令,执行第一操作。
如图2a-图2d为本申请提的用户通过语音控制电子设备的一组示例性用户界面。
假设此时,预先设置的特定应用场景中包括电子设备运行音乐类应用程序的应用场景。预设关键词可以包括:“播放上一首”、“播放下一首”。
如图2a所示,用户界面21可以为电子设备在进行音乐播放时,检测到用户输入第一语音指令时的一个用户界面,电子设备播放的是“第一音乐”。此时,电子设备可以将检测到的第一语音指令显示到用户界面21所示的语音提示框211中,该第一语音指令是非特定语音指令。例如,用户输入的第一语音指令可以为:“播放下一首”。由于该第一语音指令中包括预设关键词,则电子设备可以响应该第一语音指令,执行播放下一首音乐的操作,显示如图2b所示的用户界面22。
如图2b所示,用户界面22中,电子设备已经将当前播放的音乐,从“第一音乐”切换到了“第二音乐”。
在本申请提供的语音控制方法中,如果用户连续输入的任一语音指令中都包括预设关键词,则电子设备可以连续响应用户输入的任一语音指令。
如图2c所示,用户界面23可以为电子设备在进行音乐播放时,检测到用户输入第二语音指令时的一个用户界面,电子设备播放的是“第二音乐”。在电子设备响应了用户输入第一语音指令之后,用户又输入了第二语音指令,该第二语音指令是非特定语音指令。电子设备可以检测到该第二语音指令,并将其显示到用户界面23所示的语音提示框231中。例如,该第二语音指令可以为:“播放上一首”。由于该第二语音指令中包括预设关键词,则电子设备可以响应该第二语音指令,执行播放上一首音乐的操作,显示如图2d所示的用户界面24。
如图2d所示,用户界面24中,电子设备已经将当前播放的音乐,从“第二音乐”再一次切换到了“第一音乐”。
这样,电子设备既可以节约功耗,也可以满足用户语音控制电子设备的需求。用户不需要在每一次输入非特定语音指令之前,都输入特定语音指令打开语音交互功能,就可以实现语音控制电子设备。
下面首先介绍本申请实施例提供的示例性电子设备。
图3是本申请实施例提供的电子设备的结构示意图。
下面以电子设备为例对实施例进行具体说明。应该理解的是,电子设备可以具有比图中所示的更多的或者更少的部件,可以组合两个或多个的部件,或者可以具有不同的部件配置。图中所示出的各种部件可以在包括一个或多个信号处理和/或专用集成电路在内的硬件、软件、或硬件和软件的组合中实现。
电子设备可以包括:处理器110,外部存储器接口120,内部存储器121,通用串行总线(universal serial bus,USB)接口130,充电管理模块140,电源管理模块141,电池142,天线1,天线2,移动通信模块150,无线通信模块160,音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,传感器模块180,按键190,马达191,指示器192,摄像头193,显示屏194以及用户标识模块(subscriber identification module,SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A,陀螺仪传感器180B,气压传感器180C,磁传感器180D,加速度传感器180E,距离传感器180F,接近光传感器180G, 指纹传感器180H,温度传感器180J,触摸传感器180K,环境光传感器180L,骨传导传感器180M等。
可以理解的是,本发明实施例示意的结构并不构成对电子设备的具体限定。在本申请另一些实施例中,电子设备可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件,软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元,例如:处理器110可以包括应用处理器(application processor,AP),调制解调处理器,图形处理器(graphics processing unit,GPU),图像信号处理器(image signal processor,ISP),控制器,存储器,视频编解码器,数字信号处理器(digital signal processor,DSP),基带处理器,和/或神经网络处理器(neural-network processing unit,NPU)等。其中,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。
其中,控制器可以是电子设备的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号,产生操作控制信号,完成取指令和执行指令的控制。
处理器110中还可以设置存储器,用于存储指令和数据。在一些实施例中,处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据,可从所述存储器中直接调用。避免了重复存取,减少了处理器110的等待时间,因而提高了***的效率。
在一些实施例中,处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit,I2C)接口,集成电路内置音频(inter-integrated circuit sound,I2S)接口,脉冲编码调制(pulse code modulation,PCM)接口,通用异步收发传输器(universal asynchronous receiver/transmitter,UART)接口,移动产业处理器接口(mobile industry processor interface,MIPI),通用输入输出(general-purpose input/output,GPIO)接口,用户标识模块(subscriber identity module,SIM)接口,和/或通用串行总线(universal serial bus,USB)接口等。
可以理解的是,本发明实施例示意的各模块间的接口连接关系,只是示意性说明,并不构成对电子设备的结构限定。在本申请另一些实施例中,电子设备也可以采用上述实施例中不同的接口连接方式,或多种接口连接方式的组合。
充电管理模块140用于从充电器接收充电输入。其中,充电器可以是无线充电器,也可以是有线充电器。
电源管理模块141用于连接电池142,充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入,为处理器110,内部存储器121,外部存储器,显示屏194,摄像头193,和无线通信模块160等供电。
电子设备的无线通信功能可以通过天线1,天线2,移动通信模块150,无线通信模块160,调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用,以提高天线的利用率。例如:可以将天线1复用为无线局域网的分集天线。在另外一些实施例中,天线可以和调谐开关结合使用。
移动通信模块150可以提供应用在电子设备上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器,开关,功率放大器,低噪声放大器(low noise amplifier,LNA)等。移动通信模块150可以由天线1接收电磁波,并对接收的电磁波进行滤波,放大等处理,传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大,经天线1转为电磁波辐射出去。
调制解调处理器可以包括调制器和解调器。其中,调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后,被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A,受话器170B等)输出声音信号,或通过显示屏194显示图像或视频。
无线通信模块160可以提供应用在电子设备上的包括无线局域网(wireless local area networks,WLAN)(如无线保真(wireless fidelity,Wi-Fi)网络),蓝牙(bluetooth,BT),全球导航卫星***(global navigation satellite system,GNSS),调频(frequency modulation,FM),近距离无线通信技术(near field communication,NFC),红外技术(infrared,IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波,将电磁波信号调频以及滤波处理,将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号,对其进行调频,放大,经天线2转为电磁波辐射出去。
在一些实施例中,电子设备的天线1和移动通信模块150耦合,天线2和无线通信模块160耦合,使得电子设备可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯***(global system for mobile communications,GSM),通用分组无线服务(general packet radio service,GPRS),码分多址接入(code division multiple access,CDMA)等。
电子设备通过GPU,显示屏194,以及应用处理器等实现显示功能。GPU为图像处理的微处理器,连接显示屏194和应用处理器。GPU用于执行数学和几何计算,用于图形渲染。处理器110可包括一个或多个GPU,其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像,视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display,LCD),有机发光二极管(organic light-emitting diode,OLED),有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode,AMOLED)等。
电子设备可以通过ISP,摄像头193,视频编解码器,GPU,显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。例如,拍照时,打开快门,光线通过镜头被传递到摄像头感光元件上,光信号转换为电信号,摄像头感光元件将所述电信号传递给ISP处理,转化为肉眼可见的图像。在一些实施例中,ISP可以设置在摄像头193中。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device,CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor,CMOS)光电晶体管。感光元件把光信号转换成 电信号,之后将电信号传递给ISP转换成数字图像信号。
数字信号处理器用于处理数字信号,除了可以处理数字图像信号,还可以处理其他数字信号。例如,当电子设备在频点选择时,数字信号处理器用于对频点能量进行傅里叶变换等。
视频编解码器用于对数字视频压缩或解压缩。电子设备可以支持一种或多种视频编解码器。这样,电子设备可以播放或录制多种编码格式的视频,例如:动态图像专家组(moving picture experts group,MPEG)1,MPEG2,MPEG3,MPEG4等。
NPU为神经网络(neural-network,NN)计算处理器,通过借鉴生物神经网络结构,例如借鉴人脑神经元之间传递模式,对输入信息快速处理,还可以不断的自学习。通过NPU可以实现电子设备的智能认知等应用,例如:图像识别,人脸识别,语音识别,文本理解等。
外部存储器接口120可以用于连接外部存储卡,例如Micro SD卡,实现扩展电子设备的存储能力。外部存储卡通过外部存储器接口120与处理器110通信,实现数据存储功能。例如将音乐,视频等文件保存在外部存储卡中。
内部存储器121可以用于存储计算机可执行程序代码,所述可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令,从而执行电子设备的各种功能应用以及数据处理。内部存储器121可以包括存储程序区和存储数据区。
电子设备可以通过音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,以及应用处理器等实现音频功能。例如音乐播放,录音等。
音频模块170用于将数字音频信息转换成模拟音频信号输出,也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中,音频模块170可以设置于处理器110中,或将音频模块170的部分功能模块设置于处理器110中。
扬声器170A,也称“喇叭”,用于将音频电信号转换为声音信号。电子设备可以通过扬声器170A收听音乐,或收听免提通话。
受话器170B,也称“听筒”,用于将音频电信号转换成声音信号。当电子设备接听电话或语音信息时,可以通过将受话器170B靠近人耳接听语音。
麦克风170C,也称“话筒”,“传声器”,用于将声音信号转换为电信号。当拨打电话或发送语音信息时,用户可以通过人嘴靠近麦克风170C发声,将声音信号输入到麦克风170C。
耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130,也可以是3.5mm的开放移动电子设备平台(open mobile terminal platform,OMTP)标准接口,美国蜂窝电信工业协会(cellular telecommunications industry association of the USA,CTIA)标准接口。
压力传感器180A用于感受压力信号,可以将压力信号转换成电信号。在一些实施例中,压力传感器180A可以设置于显示屏194。压力传感器180A的种类很多,如电阻式压力传感器,电感式压力传感器,电容式压力传感器等。
陀螺仪传感器180B可以用于确定电子设备的运动姿态。在一些实施例中,可以通过陀螺仪传感器180B确定电子设备围绕三个轴(即,x,y和z轴)的角速度。陀螺仪传感器180B 可以用于拍摄防抖。
气压传感器180C用于测量气压。在一些实施例中,电子设备通过气压传感器180C测得的气压值计算海拔高度,辅助定位和导航。
磁传感器180D包括霍尔传感器。电子设备可以利用磁传感器180D检测翻盖皮套的开合。
加速度传感器180E可检测电子设备在各个方向上(一般为三轴)加速度的大小。
距离传感器180F,用于测量距离。电子设备可以通过红外或激光测量距离。在一些实施例中,拍摄场景,电子设备可以利用距离传感器180F测距以实现快速对焦。
接近光传感器180G可以包括例如发光二极管(LED)和光检测器,
环境光传感器180L用于感知环境光亮度。电子设备可以根据感知的环境光亮度自适应调节显示屏194亮度。环境光传感器180L也可用于拍照时自动调节白平衡。环境光传感器180L还可以与接近光传感器180G配合,检测电子设备是否在口袋里,以防误触。
指纹传感器180H用于采集指纹。电子设备可以利用采集的指纹特性实现指纹解锁,访问应用锁,指纹拍照,指纹接听来电等。
温度传感器180J用于检测温度。在一些实施例中,电子设备利用温度传感器180J检测的温度,执行温度处理策略。
触摸传感器180K,也称“触控面板”。触摸传感器180K可以设置于显示屏194,由触摸传感器180K与显示屏194组成触摸屏,也称“触控屏”。
按键190包括开机键,音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备可以接收按键输入,产生与电子设备的用户设置以及功能控制有关的键信号输入。
马达191可以产生振动提示。马达191可以用于来电振动提示,也可以用于触摸振动反馈。
指示器192可以是指示灯,可以用于指示充电状态,电量变化,也可以用于指示消息,未接来电,通知等。
SIM卡接口195用于连接SIM卡。SIM卡可以通过***SIM卡接口195,或从SIM卡接口195拔出,实现和电子设备的接触和分离。
图4是本申请实施例提供的电子设备的软件结构框图。
分层架构将软件分成若干个层,每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中,将***分为五层,从上至下分别为应用程序层,应用程序框架层,硬件抽象层(hardware abstraction layer,HAL),数字信号处理层以及内核层。
应用程序层可以包括一系列应用程序包。
如图4所示,应用程序包可以包括短信息,图库,相机,日历,蓝牙,地图等应用程序(也可以称为应用)。
本申请实施例中,应用程序包还可以包括语音助手,音乐,视频,导航,通话等应用程序。
其中,语音助手为第一***应用,可以为电子设备提供管理语音控制的功能。
语音助手可以包括应用场景消息下发模块、命令词处理模块。
其中,应用场景消息下发模块用于监测电子设备当前处于何种场景,并确定该应用场景对应的应用程序的标识。当检测到电子设备处于预设的某一特定应用场景时,则将本机处于该特定应用场景的消息下发到下述的动态加载模块,同时将该场景对应的应用程序的标识下发到命令词处理模块。
本申请实施例中涉及的特定应用场景至少可以包括音乐场景、视频场景、导航场景、通话场景,还可以包括其他的应用场景,本申请实施例对此不作限定。
如何监测电子设备当前处于何种场景将在下文详细描述,此处暂不赘述。
命令词处理模块用于根据预设关键词生成该预设关键词对应的指令。然后根据从上述应用场景消息下发模块中接收的应用程序的标识,将该指令下发到该标识对应的应用程序中。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface,API)和编程框架。应用程序框架层包括一些预先定义的函数。
如图4所示,应用程序框架层可以包括窗口管理器,内容提供器,视图***,电话管理器,资源管理器,通知管理器等。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小,判断是否有状态栏,锁定屏幕,截取屏幕等。
内容提供器用来存放和获取数据,并使这些数据可以被应用程序访问。所述数据可以包括视频,图像,音频,拨打和接听的电话,浏览历史和书签,电话簿等。
电话管理器用于提供电子设备的通信功能。例如通话状态的管理(包括接通,挂断等)。
资源管理器为应用程序提供各种资源,比如本地化字符串,图标,图片,布局文件,视频文件等等。
通知管理器使应用程序可以在状态栏中显示通知信息,可以用于传达告知类型的消息,可以短暂停留后自动消失,无需用户交互。比如通知管理器被用于告知下载完成,消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在***顶部状态栏的通知,例如后台运行的应用程序的通知,还可以是以对话界面形式出现在屏幕上的通知。例如在状态栏提示文本信息,发出提示音,电子设备振动,指示灯闪烁等。
视图***包括可视控件,例如显示文字的控件,显示图片的控件等。视图***可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如,包括短信通知图标的显示界面,可以包括显示文字的视图以及显示图片的视图。
硬件抽象层是位于操作***与电子设备的硬件(例如麦克风170C)之间的接口层,其目的在于将硬件抽象化,为操作***提供虚拟硬件平台。
本申请实施例中,硬件抽象层至少可以包括声纹识别模块以及二级命令词模块。
其中,二级命令词识别模块用于接收语音数据,并识别出该语音数据中只包括预设关键词的那部分语音数据,然后将该只包括预设关键的那部分语音数据下发到下述声纹识别模块。
该二级命令词识别模块还用于记录下述特定应用场景中对应的全部预设关键词。
关于预设关键词的详细描述可以参考下文,该处暂不赘述。
声纹识别模块用于记录预设声纹,该预设声纹为电子设备录入的用户的语音数据的声 纹标识,用于标识用户的身份。
该声纹识别模块还用于对只包括预设关键词的那部分语音数据进行声纹识别,以得到该语音数据的声纹,判断该声纹是否与预设声纹匹配。
数字信号处理层用于处理数字信号。
本申请实施例中,数字信号处理层至少可以包括特定应用场景模块、动态加载模块以及一级命令词识别模块。
其中,特定应用场景模块用于记录特定应用场景对应的预设关键词组,可以包括音乐场景模块、视频场景模块、导航场景模块以及通话场景模块。该特定应用场景涉及的对应关键词组为该特定应用场景对应的预设关键词的集合。
其中,音乐场景模块用于记录音乐场景中对应的预设关键词组。
视频场景模块用于记录视频场景中对应的预设关键词组。
导航场景模块用于记录导航场景中对应的预设关键词组。
通话场景模块用于记录通话场景中对应的预设关键词组。
动态加载模块用于接收在电子设备处于何种特定应用场景的消息之后,从特定应用场景模块中加载该特定应用场景涉及的预设关键词。并将该特定应用场景涉及的预设关键词下发给一级命令词识别模块。
一级命令词识别模块用于接收内核层发送的语音指令。还可以获取特定应用场景模块中记录的特定应用场景涉及的预设关键词,结合预设关键词识别出该语音指令中包括预设关键词的那部分语音数据,然后将该部分语音数据下发到上述二级命令词识别模块。
内核层是硬件和软件之间的层。内核层至少包含显示驱动,摄像头驱动,音频驱动,传感器驱动。
其中,音频驱动用于接收麦克风170C采集的语音指令,将该语音指令下发到一级命令词识别模块。
示意性的,结合图4所示的电子设备的软件结构框图。
图5为本申请中涉及的语音控制方法的一个示例性信息流示意图。
可以理解的是,图5中示出的仅仅为电子设备中的部分软件模块,不应对电子设备的软件构架构成限定。
图5示出的示例性信息流示意图描述了电子设备在音乐场景下,检测到用户的第一语音指令之后,电子设备识别出该第一语音指令中的第一预设关键词并响应该第一语音指令,执行该第一预设关键词对应的操作。该第一预设关键词不为打开语音交互功能的特定关键词;
具体的,当电子设备开启了音乐类应用程序时,电子设备的第一***应用151中的应用场景消息下发模块151A可以监测到电子设备当前处于音乐场景,并确定该音乐类应用152(音乐类应用程序)的标识。然后将电子设备当前处于音乐场景的消息下发到动态加载模块155,并将该音乐类应用程序的标识下发到命令词处理模块151B中。动态加载模块155从音乐场景模块157A中加载该音乐场景下涉及的全部预设关键词。然后,动态加载模块155可以将该音乐场景下涉及的全部预设关键词下发到一级命令词识别模块156中。电子 设备的麦克风170C可以采集语音指令,并将该语音指令传输给电子设备的音频驱动158。电子设备的音频驱动158可以将该语音指令下发到电子设备的一级命令词识别模块156中。
电子设备的一级命令词识别模块156可以根据该音乐场景下涉及的全部预设关键词,判断该语音指令中是否包括第一语音数据,该第一语音数据中至少包括第一预设关键词。该第一预设关键词为该音乐场景下涉及的全部预设关键词中的某一个预设关键词。如果包括第一语音数据,则将该第一语音数据下发到第二级命令词识别模块153中。如果不包括第一语音数据,则不会向第二级命令词识别模块153下发任何消息。
第二级命令词识别模块根据该第一语音数据以及应用场景中涉及的全部预设关键词。识别该第一语音数据中的第二语音数据,该第二语音数据中只包括第一预设关键词,并将该第二语音数据下发给声纹识别模块154。
声纹识别模块154用于对第二语音数据进行声纹识别。如果该第二语音数据的声纹与预设声纹匹配,则将该第二语音数据传输到命令词处理模块151B中。如果该第二语音数据的声纹与预设声纹不匹配,则不会向命令词处理模块151B下发任何消息。
命令词处理模块151B用于将第二语音数据中的第一预设关键词生成该预设关键词对应的指令,然后根据从上述应用场景消息下发模块中接收的音乐类应用程序的标识,将该指令下发到该标识对应的音乐类应用程序中。
该音乐类应用程序在接收到该第一预设关键词对应的指令之后,可以执行相应的操作,来响应该指令。例如,如果该语音指令中包括的第一预设关键词为“播放下一首”,则电子设备的音乐类应用程序执行播放下一首音乐。
可以理解的是,上述涉及的各个模块可以为一组代码、一组函数、包括一个或多个信号处理和/或专用集成电路在内的硬件,也可以是其中的一个或者多个的组合。本申请实施例对此不作限定。
在一些实施例中,电子设备的DSP可以处理数字信号并且其功耗低,则电子设备可以将该动态加载模块155、一级命令词识别模块156以及特定应用场景模块157存储在电子设备的DSP中内置的存储器中,由DSP调用这些功能模块对应的指令实现这些模块的功能。这样可以使得这些功能模块在运行的过程中不会浪费电子设备的功耗,其他的功能模块,例如二级命令词识别模块可以存储到电子设备的内部存储器121中,由处理器110调用这些功能模块对应的指令实现这些模块的功能。
下面结合上述示例性电子设备的软硬件结构,对本申请实施例中的方法进行具体描述:
在本申请实施例中,电子设备针对不同的特定应用场景分别设置了一些预设关键词。当电子设备检测到用户输入的语音指令中包括该预设关键词时,则电子设备可以响应该语音指令,执行第一操作。该第一操作为第一语音指令对应的操作。该过程涉及的详细解释可以参考下述对图6的介绍。
图6为本申请实施例提供的语音控制方法的一个示意性流程图。
S101.电子设备确定当前处于第一应用场景;
第一应用场景为特定应用场景。电子设备在该第一应用场景下可以不打开语音交互功能,也可以实现语音控制电子设备。例如,该第一应用场景可以是音乐场景。
例如,如图2a所示,为电子设备的一个音乐播放界面,由于电子设备打开了音乐类应用程序,则电子设备可以确定当前处于音乐场景。
在一种可能的实现方式中,电子设备可以设置开始运行第一应用程序,则进入了第一应用场景。该第一应用程序可以为音乐类应用程序,也可以为其他的应用程序,例如视频类应用程序、导航类应用程序等,本申请实施例对此不作限定。
具体的,当电子设备开启一个应用程序时,可以获取该应用程序对应的标识,该标识用于唯一表示一个应用程序。
电子设备可以记录第一应用程序的第一标识,当电子设备开启一个应用程序时,则获取该应用程序对应的第二标识,电子设备检查该第二标识与第一标识是否相同,如果相同则认为当前电子设备开始运行第一应用程序,确定电子设备当前处于第一应用场景。
在另一种可能的实现方式中,电子设备可以监测本机的数据输出,根据电子设备输出的是何种数据,判断本机的是否进入了第一应用场景。
其中,数据输出是指电子设备传输给用户的数据,例如,视频数据、音频数据等。
电子设备可以预先设置输出的数据为何种数据时本机处于何种应用场景。例如,电子设备可以设置本机输出的数据为音频数据时,电子设备处于音乐场景,电子设备输出的是视频数据时,电子设备为视频场景等。
应该理解的是,还可以有其他的方式可以确定电子设备当前处于第一应用场景,本申请实施例对此不作限定。
该步骤S101可以由前述图5中的应用场景消息下发模块151A以及动态加载模块155完成。具体的,该应用场景消息下发模块151A可以确定电子设备确定当前处于第一应用场景并将该消息下发到动态加载模块155中。详细过程可以参考前述对图5中相关部分的描述,此处不再赘述。
S102.电子设备加载该第一应用场景对应的第一预设关键词组,该第一预设关键词组至少包括第一预设关键词;
该第一预设关键词组为第一应用场景涉及的预设关键词的集合,其中,该第一预设关键词组中的任一预设关键词可以被称为第一预设关键词。电子设备处于第一应用场景时,用户可以输入包括该第一预设关键词的语音指令对电子设备进行语音控制。
例如,当电子设备处于音乐场景时,该第一预设关键词可以是“播放下一首”。
该步骤S102可以由前述图5中的动态加载模块155以及特定应用场景模块157完成。具体的,该动态加载模块155可以从特定应用场景模块157中加载该第一应用场景对应的第一预设关键词组。并将其下发到一级命令词识别模块156中。详细过程可以参考前述对图5中相关部分的描述,此处不再赘述。
S103.电子设备检测到第一语音指令;
例如,图2a所示电子设备中,用户输入第一语音指令,该第一语音指令为“播放下一首”,电子设备可以检测到该第一语音指令。图2c所示电子设备中,用户输入第一语音指令,该第一语音指令为“播放上一首”,电子设备可以检测到该第一语音指令。
具体的,电子设备可以在按照一定的频率检测电子设备周围的语音信息,电子设备的麦克风可以采集电子设备周围的语音数据,其中包括第一语音指令。
第一语音指令中既可以只包括第一预设关键词,也可以包括其他的语音数据。其中,第一语音数据中还包括其他语音数据以外的情况可以分为以下三种:
情况1:在一些实施例中,电子设备检测到的第一语音指令可以是一句话,该第一语音指令中除第一预设关键词外,还可以包括其他的语音数据。例如,第一语音指令可以是:“XXX,播放下一首XXXX”,则在该第一语音指令中,“播放下一首”为第一预设关键词,但其他的语音数据不是预设关键词。
情况2:在一些实施例中,该第一语音指令中除第一预设关键词外,还可以包括其他一个或多个预设关键词,例如第二预设关键词以及其他的语音数据。该第二预设关键词为第一应用场景对应的预设关键词,出现在第一预设关键词之后,可以与第一预设关键词相同,也可以不相同。例如,第一语音指令可以是:“XXX播放下一首暂停XXXX”,则在该第一语音指令中,第一预设关键词可以为“播放下一首”,第二预设关键词可以为“播放”,但其他的语音数据不是预设关键词。
情况3:在另一些实施例中,该第一语音指令中不包括任何预设关键词。
该步骤S103可以由前述图5中的音频驱动158完成。具体的,该音频驱动158可以检测到第一语音指令,并将其下发到一级命令词识别模块156中。详细过程可以参考前述对图5中相关部分的描述,此处不再赘述。
S104.电子设备根据第一语音指令与第一预设关键词组,确定第一语音指令中是否包括第一语音数据,该第一语音数据中至少包括第一预设关键词;
第一语音数据为第一语音指令中的一部分,其中,至少包括第一预设关键词。
第一语音指令的长度为t秒。电子设备从该第一语音指令的第0秒开始,依次获取长度为m秒的语音数据,其中,m小于t。该长度为m秒的语音数据对应一个字符。最近获取的连续N个长度为m秒的语音数据中,如果包括第一预设关键词,则电子设备不再获取下一个长度为m秒的语音数据,直接将该最近获取的连续N个长度为m秒的语音数据确定为第一语音数据。
在一些实施例中,电子设备可以设置该第一语音数据的长度为X秒,并设置N*m=X。X一般可以设置为1.5-2.5,例如2。在一种特殊情况下,如果电子设备已经确定了第一语音指令中,前0秒-M秒的语音数据中包括第一预设关键词,其中M小于X。则电子设备可以将该M个长度为m秒的语音数据确定为第一语音数据。
在一些实施例中,两个连续的长度为m秒的语音数据没有交叠部分。
在另一些实施中,两个连续的长度为m秒的语音数据之间有n秒交叠部分,n小于m,即前一个长度为m秒的语音数据中最后n秒的语音数据为后一个长度为m秒的语音数据开始n秒的语音数据。
在一些实施例中,电子设备确定连续N个长度为m秒的语音数据中包括第一预设关键词的过程为:首先,电子设备从第一语音指令中获取第i个长度为m秒的语音数据。其中。第一个长度为m秒的语音数据为第一语音指令中长度为0秒-m秒的语音数据。并获取第一 预设关键词中的第j(j为大于等于1的正整数)个字符。其中,第一预设关键词中的第一个字符对应的j为1。然后,电子设备判断第i个长度为m秒的语音数据对应的字符与第一预设关键词中的第j个字符是否相同。
如果第i个长度为m秒的语音数据对应的字符与第一预设关键词中的第j个字符不相同且j不等于1,则电子设备将j置为1,即获取第一预设关键词中的第一个字符,然后,电子设备判断第i个长度为m秒的语音数据对应的字符与第一预设关键词中的第一个字符是否相同;如果第i个长度为m秒的语音数据对应的字符与第一预设关键词中的第j个字符不相同且j等于1,则电子设备继续获取第i+1个长度为m秒的语音数据,然后,电子设备判断第i+1个长度为m秒的语音数据对应的字符与第一预设关键词中的第一个字符是否相同。
如果第i个长度为m秒的语音数据对应的字符与第一预设关键词中的第j个字符相同,则电子设备继续获取第i+1个长度为m秒的语音数据,以及获取第一预设关键词中的第j+1个字符,然后,电子设备判断第i+1个长度为m秒的语音数据对应的字符与第一预设关键词中的第j+1个字符是否相同。
依次类推,直到电子设备根据第一语音指令与第一预设关键词组,确定出第一语音数据。
如图7所示,为确定第一语音数据的一个示意图。
电子设备设置第一语音数据为连续六个长度为m秒的语音数据。
首先电子设备获取第一个长度为m秒的语音数据对应的字符“X”,并获取第一预设关键词中的第一个字符“声”,此时i=1,j=1。然后,电子设备判断字符“X”与字符“声”不相同。
由于j=1,则电子设备继续获取第二(i+1)个长度为m秒的语音数据对应的字符“播”,此时i=2,j=1。然后,电子设备判断字符“播”与字符“声”不相同。
由于j=1,则电子设备继续获取第三(i+1)个长度为m秒的语音数据对应的字符“放”,此时i=3,j=1。然后,电子设备判断字符“放”与字符“声”不相同。
由于j=1,则电子设备继续获取第四(i+1)个长度为m秒的语音数据对应的字符“声”,此时i=3,j=1。然后,电子设备判断字符“声”与字符“声”相同。
电子设备继续获取第五(i+1)个长度为m秒的语音数据对应的字符“音”,并获取第一预设关键词中的第二(j+1)个字符“声”,此时i=5,j=2。然后,电子设备判断字符“声”与字符“声”相同。
依次类推,电子设备获取第八个长度为m秒的语音数据对应的字符“点”,并获取第一预设关键词中的第五个字符“点”,此时i=8,j=5。然后,电子设备判断字符“点”与字符“点”相同。
则电子设备确定最近获取的六个长度为m秒的语音数据“放声音大一点”为第一数据。
在一些实施例中,则电子设备根据该步骤S104所述的方法识别出第一语音数据之后,还可以继续判断第一语音指令中没有被判断过的语音数据中是否还包含另外的第一语音数据。
如果第一语音指令为步骤S103中的情况1和情况2所述,则对于第一语音指令中的任 一第一语音数据,电子设备都可以利用其执行下述步骤S105-步骤S108。
如果第一语音指令为步骤S103中的情3所述,不包含任何第一预设关键词,则电子设备执行步骤S108。
在该步骤S104可以由前述图5中的一级命令词识别模块156完成。具体的,该一级命令词识别模块156可以根据音频驱动158检测到的第一语音指令与动态加载模块155下发的第一预设关键词组,确定第一语音数据。并将其下发到二级命令词识别模块153中。详细过程可以参考前述对图5中相关部分的描述,此处不再赘述。为了节约手机功耗,一级命令词识别模块156中确定第一语音数据涉及的算法简单,该过程由DSP调用这些功能模块完成。
S105.电子设备根据第一语音数据与特定应用场景对应的全部预设关键词,确定第二语音数据,该第二语音数据只包括第一预设关键词;
该步骤S105中根据第一语音数据与特定应用场景对应的全部预设关键词,确定第二语音数据的过程可以参考前述对步骤S104的相关描述。
该过程中,电子设备只需要确定出第一语音数据中只包括第一预设关键词的那分语音数据作为第二语音数据。
在该步骤S105可以由前述图5中的二级命令词识别模块153完成。具体的,该二级命令词识别模块153可以根据一级命令词识别模块156下发的第一语音数据与其记录的特定应用场景对应的全部预设关键词,确定第二语音数据。并将其下发到声纹识别模块154中。详细过程可以参考前述对图5中相关部分的描述,此处不再赘述。为了提高准确率,二级命令词识别模块153中确定第二语音数据涉及的算法相比于一级命令词识别模块156中确定第一语音数据的算法复杂,该过程由处理器调用这些功能模块完成。
S106.电子设备判断该第二语音数据的声纹是否与预设声纹匹配;
该预设声纹为电子设备录入的用户的语音数据的声纹标识,用于标识用户的身份。
电子设备可以利用提取第二语音数据中的声纹,与该预设声纹进行匹配,如果一致,则电子设备判断该第二语音数据来自与预设声纹匹配的用户,可以对该第一语音指令进行响应,执行步骤S107。
如果不一致,则电子设备判断该第二语音数据不是来自与预设声纹匹配的用户,可以不对该第一语音指令进行响应,执行步骤S108。
在该步骤S106可以由前述图5中的声纹识别模块154完成。具体的,该声纹识别模块154可以根据二级命令词识别模块153下发的第二语音数据与其记录的预设声纹,确定第二语音数据是否来自与预设声纹匹配的用户。详细过程可以参考前述对图5中相关部分的描述,此处不再赘述。
在一些实施例中,该步骤S106是可选的,电子设备可以在执行步骤S105之后,可以不执行该步骤S106,直接执行步骤S107。
S107.电子设备响应该第一语音指令,执行该第二语音数据中的第一预设关键词对应的 操作;
例如,图2b示出的用户界面,以及图2d示出的用户界面,为电子设备响应该第一语音指令的两个用户界面。可以看出,电子设备显示的是执行该第一预设关键词对应的操作之后的用户界面。
电子设备根据第二语音数据中的第一预设关键词生成该预设关键词对应的操作,执行该第二语音数据中的第一预设关键词对应的操作来响应该第一语音指令。
在该步骤S107可以由前述图5中的命令词处理模块151B完成。具体的,该命令词处理模块151B可以根据二级命令词识别模块153下发的第二语音数据转换成该第二语音数据对应的操作并下发到第一应用场景涉及的应用程序,应用程序执行该操作来响应该第一语音指令。详细过程可以参考前述对图5中相关部分的描述,此处不再赘述。
S108.电子设备不响应该第一语音指令;
用户输入第一语音指令之后,电子设备没有显示执行该第一预设关键词对应的操作之后的用户界面,即电子设备不响应该第一语音指令。
本申请实施例中,电子设备处于第一应用场景时,显示的用户界面可以被称为第一用户界面,响应第一语音指令之后显示的用户界面可以被称为第二用户界面。
可以理解的是,上述特定应用场景下,电子设备都可以利用本申请实施例涉及的语音控制方法。该特定应用场景是电子设备预先设置的。例如,电子设备设置当本机打开视频类应用程序时,则进入视频场景。电子设备可以设置该视频场景为一格特定应用场景。则为当电子设备打开视频类应用程序时,检测到用户输入的非特定语音指令中如果包含该视频场景下的预设关键词,则电子设备可以响应该非特定语音指令,执行第一操作,例如,该非特定语音指令可以为“声音大一点”。该第一操作为该非特定语音指令对应的操作,例如当该非特定语音指令为“声音大一点”时,电子设备可以设置播放视频的声音大一点。
应该理解的是,该特定应用场景还可以包括其他应用场景,本申请实施例对此不作限定。
本申请实施例中,电子设备的内部存储器121中或者外部存储接口120外接的存储设备中可以预先存储预设关键词以及本申请实施例涉及的语音控制方法涉及的相关指令以及预设声纹,电子设备在DSP中设置存储器,并将特定应用场景涉及的预设关键词存储在其中,使得电子设备执行本申请实施例中的语音控制方法。
下面结合步骤S101-步骤S108为例,示例性说明电子设备的工作流程。
1.电子设备确定当前处于第一应用场景;
在一些实施例中,电子设备的触摸传感器180K接收到触摸操作(用户触摸拍摄控件时触发的),相应的硬件中断被发给内核层。内核层将触摸操作加工成原始输入事件(包括触摸坐标,触摸操作的时间戳等信息)。原始输入事件被存储在内核层。应用程序框架层从内核层获取原始输入事件,识别该输入事件所对应的控件的应用。
例如,一第一应用场景为音乐场景为例,以上触摸操作是触摸单击操作,该单击操作所对应的控件为音乐类应用程序的图标。音乐类应用程序调用应用框架层的接口,启动音乐应用类应用程序,进而确定当前处于音乐场景。
2.电子设备加载该第一应用场景对应的第一预设关键词组,该第一预设关键词组至少包括第一预设关键词。
电子设备的DSP处理器可以从DSP中内置的存储器中加载其存储的该第一应用场景对应的第一预设关键词组。
3.电子设备检测到第一语音指令。
电子设备可以在按照一定的频率检测电子设备周围的语音信息,电子设备的麦克风可以采集电子设备周围的语音数据,其中包括第一语音指令。并将检测到第一语音指令存储在内部存储器121中或者外部存储接口120外接的存储设备中。
4.电子设备根据第一语音指令与第一预设关键词组,确定第一语音数据。
电子设备可以通过DSP从存储器中获取第一语音指令与第一预设关键词组,并调用相关计算机指令,确定第一语音数据。并将该第一语音数据存储在内部存储器121中或者外部存储接口120外接的存储设备中。
5.电子设备根据第一语音数据与特定应用场景对应的全部预设关键词,确定第二语音数据。
电子设备可以通过处理器110从内部存储器121中获取第一语音数据与特定应用场景对应的全部预设关键词,并调用相关计算机指令,确定第二语音数据。并将该第二语音数据存储在内部存储器121中或者外部存储接口120外接的存储设备中。
6.电子设备判断该第二语音数据的声纹是否与预设声纹匹配。
电子设备可以过处理器110从内部存储器121中获取第二语音数据以及预设声纹匹配,并调用相关计算机指令,判断该第二语音数据的声纹是否与预设声纹匹配。
7.电子设备响应该第一语音指令,执行该第二语音数据中的第一预设关键词对应的操作。
电子设备可以调用应用框架层的接口,执行该第二语音数据中的第一预设关键词对应的操作。进而通过调用内核层启动显示驱动,显示执行该第二语音数据中的第一预设关键词对应的操作之后的用户界面。
综上所述,实施本申请实施例涉及的语音控制方法,电子设备可以一些特定应用场景中,不需要打开语音交互功能,也可以接收用户输入的非特定语音指令,并响应该非特定语音指令。
以上所述,以上实施例仅用以说明本申请的技术方案,而非对其限制;尽管参照前述实施例对本申请进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。
上述实施例中所用,根据上下文,术语“当…时”可以被解释为意思是“如果…”或 “在…后”或“响应于确定…”或“响应于检测到…”。类似地,根据上下文,短语“在确定…时”或“如果检测到(所陈述的条件或事件)”可以被解释为意思是“如果确定…”或“响应于确定…”或“在检测到(所陈述的条件或事件)时”或“响应于检测到(所陈述的条件或事件)”。
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时,可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时,全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如DVD)、或者半导体介质(例如固态硬盘)等。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,该流程可以由计算机程序来指令相关的硬件完成,该程序可存储于计算机可读取存储介质中,该程序在执行时,可包括如上述各方法实施例的流程。而前述的存储介质包括:ROM或随机存储记忆体RAM、磁碟或者光盘等各种可存储程序代码的介质。
以上所述,仅为本发明的具体实施方式,但本发明的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本发明揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本发明的保护范围之内。因此,本发明的保护范围应以所述权利要求的保护范围为准。
Claims (17)
- 一种语音控制方法,其特征在于,包括:电子设备确定当前处于第一应用场景;所述电子设备检测到第一语音指令;所述第一语音指令为不用于打开语音交互功能的非特定语音指令;所述电子设备确定所述第一语音指令中是否包括第一预设关键词;所述第一预设关键词为所述第一应用场景对应的全部预设关键词中的任一预设关键词;在确定所述第一语音指令中包括第一预设关键词的情况下,所述电子设备不打开语音交互功能,响应所述第一语音指令,执行所述第一预设关键词对应的操作;在确定所述第一语音指令中不包括第一预设关键词的情况下,所述电子设备不响应所述第一语音指令。
- 根据权利要求1所述的方法,其特征在于,电子设备确定当前处于第一应用场景之前,所述方法还包括:所述电子设备显示第一用户界面;所述电子设备不打开语音交互功能,响应所述第一语音指令,执行所述第一预设关键词对应的操作,具体包括:所述电子设备不打开语音交互功能,响应所述第一语音指令,执行所述第一预设关键词对应的操作,显示第二用户界面,所述第二用户界面与所述第一用户界面不同。
- 根据权利要求1所述的方法,其特征在于,所述电子设备确定当前处于第一应用场景,具体包括:所述电子设备前台运行第一应用程序,则所述电子设备确定当前处于第一应用场景;或者,所述电子设备后台运行第一应用程序,则所述电子设备确定当前处于第一应用场景。
- 根据权利要求1-3中任一项所述的方法,所述电子设备确定所述第一语音指令中是否包括第一预设关键词,具体包括:所述电子设备加载所述第一应用场景对应的全部预设关键词;所述电子设备根据所述第一语音指令与所述第一应用场景对应的全部预设关键词,确定第一语音指令中是否包括第一语音数据,所述第一语音数据中至少包括所述第一预设关键词;在确定所述第一语音指令中包括第一语音数据的情况下,所述电子设备确定所述第一语音指令中包括第一预设关键词;在确定所述第一语音指令中不包括第一语音数据的情况下,所述电子设备确定所述第一语音指令中不包括第一预设关键词。
- 根据权利要求4所述的方法,其特征在于,所述电子设备确定第一语音指令中包括 第一语音数据之后,所述方法还包括:所述电子设备加载全部特定应用场景对应的全部预设关键词,所述全部特定应用场景包括所述第一应用场景,所述全部预设关键词中包括第一预设关键词;所述电子设备根据第一语音数据与所述全部特定应用场景对应的全部预设关键词,确定第一语音数据中的一部分作为第二语音数据,所述第二语音数据只包括第一预设关键词;响应所述第一语音指令,执行所述第一预设关键词对应的操作,具体包括:所述电子设备根据所述第二语音数据中的第一预设关键词生成所述第一预设关键词对应的操作;响应所述第一语音指令,执行所述第一预设关键词对应的操作。
- 根据权利要求4所述的方法,其特征在于,所述电子设备确定第一语音指令中包括第一语音数据之后,所述方法还包括:所述电子设备加载全部特定应用场景对应的全部预设关键词,所述全部特定应用场景包括所述第一应用场景,所述全部预设关键词中包括第一预设关键词;所述电子设备根据第一语音数据与所述全部特定应用场景对应的全部预设关键词,确定第一语音数据中的一部分作为第二语音数据,所述第二语音数据只包括第一预设关键词;所述电子设备确定所述第二语音数据的声纹是否与预设声纹匹配,所述预设声纹为所述电子设备录入的用户的语音数据的声纹标识,用于标识用户的身份;响应所述第一语音指令,执行所述第一预设关键词对应的操作,具体包括:在确定所述第二语音数据的声纹与预设声纹匹配的情况下,所述电子设备根据所述第二语音数据中的第一预设关键词生成所述第一预设关键词对应的操作;响应所述第一语音指令,执行所述第一预设关键词对应的操作;在确定所述第二语音数据的声纹与预设声纹不匹配的情况下,不响应所述第一语音指令。
- 根据权利要求1-6中任一项所述的方法,其特征在于,所述特定应用场景是预先设置好,存储在电子设备中的,所述特定应用场景至少包括一个应用运行时的场景,所述第一应用场景为任一所述特定应用场景。
- 根据权利要求1-7中任一项所述的方法,其特征在于,所述特定应用场景对应的全部预设关键词是预先设置好,存储在电子设备中的,所述特定应用场景对应的全部预设关键词中任一预设关键词都对应一个与该预设关键词同意义的操作。
- 一种电子设备,其特征在于,所述电子设备包括一个或多个处理器和一个或多个存储器;其中,所述一个或多个存储器与所述一个或多个处理器耦合,所述一个或多个存储器用于存储计算机程序代码,所述计算机程序代码包括计算机指令,当所述一个或多个处理器执行所述计算机指令时,所述一个或多个处理器用于调用所述计算机指令以使得所述电子设备执行:确定当前处于第一应用场景;检测到第一语音指令;所述第一语音指令为不用于打开语音交互功能的非特定语音指令;确定所述第一语音指令中是否包括第一预设关键词;所述第一预设关键词为所述第一应用场景对应的全部预设关键词中的任一预设关键词;在确定所述第一语音指令中包括第一预设关键词的情况下,不打开语音交互功能,响应所述第一语音指令,执行所述第一预设关键词对应的操作;在确定所述第一语音指令中不包括第一预设关键词的情况下,不响应所述第一语音指令。
- 根据权利要求9所述的电子设备,其特征在于,所述一个或多个处理器还用于调用所述计算机指令以使得所述电子设备执行:显示第一用户界面;不打开语音交互功能,响应所述第一语音指令,执行所述第一预设关键词对应的操作,具体包括:不打开语音交互功能,响应所述第一语音指令,执行所述第一预设关键词对应的操作,显示第二用户界面,所述第二用户界面与所述第一用户界面不同。
- 根据权利要求9所述的电子设备,其特征在于,所述一个或多个处理器具体用于调用所述计算机指令以使得所述电子设备执行:前台运行第一应用程序,则所述电子设备确定当前处于第一应用场景;或者,后台运行第一应用程序,则所述电子设备确定当前处于第一应用场景。
- 根据权利要求9-11中任一项所述的电子设备,其特征在于,所述一个或多个处理器具体用于调用所述计算机指令以使得所述电子设备执行:加载所述第一应用场景对应的全部预设关键词;根据所述第一语音指令与所述第一应用场景对应的全部预设关键词,确定第一语音指令中是否包括第一语音数据,所述第一语音数据中至少包括所述第一预设关键词;在确定所述第一语音指令中包括第一语音数据的情况下,确定所述第一语音指令中包括第一预设关键词;在确定所述第一语音指令中不包括第一语音数据的情况下,确定所述第一语音指令中不包括第一预设关键词。
- 根据权利要求12所述的电子设备,其特征在于,所述电子设备确定第一语音指令中包括第一语音数据之后,所述一个或多个处理器还用于调用所述计算机指令以使得所述电子设备执行:加载全部特定应用场景对应的全部预设关键词,所述全部特定应用场景包括所述第一应用场景,所述全部预设关键词中包括第一预设关键词;根据第一语音数据与所述全部特定应用场景对应的全部预设关键词,确定第一语音数 据中的一部分作为第二语音数据,所述第二语音数据只包括第一预设关键词;所述一个或多个处理器具体用于调用所述计算机指令以使得所述电子设备执行:根据所述第二语音数据中的第一预设关键词生成所述第一预设关键词对应的操作;响应所述第一语音指令,执行所述第一预设关键词对应的操作。
- 根据权利要求12所述的电子设备,其特征在于,所述电子设备确定第一语音指令中包括第一语音数据之后,所述一个或多个处理器还用于调用所述计算机指令以使得所述电子设备执行:加载全部特定应用场景对应的全部预设关键词,所述全部特定应用场景包括所述第一应用场景,所述全部预设关键词中包括第一预设关键词;根据第一语音数据与所述全部特定应用场景对应的全部预设关键词,确定第一语音数据中的一部分作为第二语音数据,所述第二语音数据只包括第一预设关键词;确定所述第二语音数据的声纹是否与预设声纹匹配,所述预设声纹为所述电子设备录入的用户的语音数据的声纹标识,用于标识用户的身份;所述一个或多个处理器具体用于调用所述计算机指令以使得所述电子设备执行:在确定所述第二语音数据的声纹与预设声纹匹配的情况下,根据所述第二语音数据中的第一预设关键词生成所述第一预设关键词对应的操作;响应所述第一语音指令,执行所述第一预设关键词对应的操作;在确定所述第二语音数据的声纹与预设声纹不匹配的情况下,不响应所述第一语音指令。
- 一种芯片***,所述芯片***应用于电子设备,所述芯片***包括一个或多个处理器,所述处理器用于调用计算机指令以使得所述电子设备执行如权利要求1-8中任一项所述的方法。
- 一种包含指令的计算机程序产品,其特征在于,当所述计算机程序产品在电子设备上运行时,使得所述电子设备执行如权利要求1-8中任一项所述的方法。
- 一种计算机可读存储介质,包括指令,其特征在于,当所述指令在电子设备上运行时,使得所述电子设备执行如权利要求1-8中任一项所述的方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110728832.5 | 2021-06-29 | ||
CN202110728832.5A CN113488042B (zh) | 2021-06-29 | 2021-06-29 | 一种语音控制方法及电子设备 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023273321A1 true WO2023273321A1 (zh) | 2023-01-05 |
Family
ID=77936552
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/073135 WO2023273321A1 (zh) | 2021-06-29 | 2022-01-21 | 一种语音控制方法及电子设备 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN113488042B (zh) |
WO (1) | WO2023273321A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117711410A (zh) * | 2023-05-30 | 2024-03-15 | 荣耀终端有限公司 | 语音唤醒方法及相关设备 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113488042B (zh) * | 2021-06-29 | 2022-12-13 | 荣耀终端有限公司 | 一种语音控制方法及电子设备 |
CN114120979A (zh) * | 2022-01-25 | 2022-03-01 | 荣耀终端有限公司 | 语音识别模型的优化方法、训练方法、设备及介质 |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140122078A1 (en) * | 2012-11-01 | 2014-05-01 | 3iLogic-Designs Private Limited | Low Power Mechanism for Keyword Based Hands-Free Wake Up in Always ON-Domain |
CN108766427A (zh) * | 2018-05-31 | 2018-11-06 | 北京小米移动软件有限公司 | 语音控制方法及装置 |
CN111161734A (zh) * | 2019-12-31 | 2020-05-15 | 苏州思必驰信息科技有限公司 | 基于指定场景的语音交互方法及装置 |
CN111816192A (zh) * | 2020-07-07 | 2020-10-23 | 云知声智能科技股份有限公司 | 语音设备及其控制方法、装置和设备 |
CN112201246A (zh) * | 2020-11-19 | 2021-01-08 | 深圳市欧瑞博科技股份有限公司 | 基于语音的智能控制方法、装置、电子设备及存储介质 |
WO2021027267A1 (zh) * | 2019-08-15 | 2021-02-18 | 华为技术有限公司 | 语音交互方法、装置、终端及存储介质 |
CN113488042A (zh) * | 2021-06-29 | 2021-10-08 | 荣耀终端有限公司 | 一种语音控制方法及电子设备 |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140272823A1 (en) * | 2013-03-15 | 2014-09-18 | Phonics Mouth Positions + Plus | Systems and methods for teaching phonics using mouth positions steps |
CN110083444B (zh) * | 2013-12-10 | 2024-06-11 | 华为终端有限公司 | 一种任务管理方法及设备 |
CN106373575B (zh) * | 2015-07-23 | 2020-07-21 | 阿里巴巴集团控股有限公司 | 一种用户声纹模型构建方法、装置及*** |
CN111131601B (zh) * | 2018-10-31 | 2021-08-27 | 华为技术有限公司 | 一种音频控制方法、电子设备、芯片及计算机存储介质 |
CN110197662A (zh) * | 2019-05-31 | 2019-09-03 | 努比亚技术有限公司 | 语音控制方法、可穿戴设备及计算机可读存储介质 |
CN110473556B (zh) * | 2019-09-17 | 2022-06-21 | 深圳市万普拉斯科技有限公司 | 语音识别方法、装置和移动终端 |
CN110970026A (zh) * | 2019-12-17 | 2020-04-07 | 用友网络科技股份有限公司 | 语音交互匹配方法、计算机设备以及计算机可读存储介质 |
CN111078017A (zh) * | 2019-12-19 | 2020-04-28 | 珠海格力电器股份有限公司 | 构建虚拟场景的控制方法、装置、电子设备及存储介质 |
CN112230877A (zh) * | 2020-10-16 | 2021-01-15 | 惠州Tcl移动通信有限公司 | 一种语音操作方法、装置、存储介质及电子设备 |
CN112802468B (zh) * | 2020-12-24 | 2023-07-11 | 合创汽车科技有限公司 | 汽车智能终端的交互方法、装置、计算机设备和存储介质 |
CN112837159B (zh) * | 2021-02-24 | 2024-04-02 | 中国工商银行股份有限公司 | 基于场景要素的交易引导方法、装置、电子设备及介质 |
-
2021
- 2021-06-29 CN CN202110728832.5A patent/CN113488042B/zh active Active
-
2022
- 2022-01-21 WO PCT/CN2022/073135 patent/WO2023273321A1/zh active Application Filing
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140122078A1 (en) * | 2012-11-01 | 2014-05-01 | 3iLogic-Designs Private Limited | Low Power Mechanism for Keyword Based Hands-Free Wake Up in Always ON-Domain |
CN108766427A (zh) * | 2018-05-31 | 2018-11-06 | 北京小米移动软件有限公司 | 语音控制方法及装置 |
WO2021027267A1 (zh) * | 2019-08-15 | 2021-02-18 | 华为技术有限公司 | 语音交互方法、装置、终端及存储介质 |
CN111161734A (zh) * | 2019-12-31 | 2020-05-15 | 苏州思必驰信息科技有限公司 | 基于指定场景的语音交互方法及装置 |
CN111816192A (zh) * | 2020-07-07 | 2020-10-23 | 云知声智能科技股份有限公司 | 语音设备及其控制方法、装置和设备 |
CN112201246A (zh) * | 2020-11-19 | 2021-01-08 | 深圳市欧瑞博科技股份有限公司 | 基于语音的智能控制方法、装置、电子设备及存储介质 |
CN113488042A (zh) * | 2021-06-29 | 2021-10-08 | 荣耀终端有限公司 | 一种语音控制方法及电子设备 |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117711410A (zh) * | 2023-05-30 | 2024-03-15 | 荣耀终端有限公司 | 语音唤醒方法及相关设备 |
Also Published As
Publication number | Publication date |
---|---|
CN113488042B (zh) | 2022-12-13 |
CN113488042A (zh) | 2021-10-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3872807B1 (en) | Voice control method and electronic device | |
US20220147228A1 (en) | Display Method and Related Apparatus | |
WO2021052263A1 (zh) | 语音助手显示方法及装置 | |
EP4325840A1 (en) | Display method of video call applied to electronic device and related apparatus | |
US11385857B2 (en) | Method for displaying UI component and electronic device | |
CN112399390B (zh) | 一种蓝牙回连的方法及相关装置 | |
WO2023273321A1 (zh) | 一种语音控制方法及电子设备 | |
US20230216990A1 (en) | Device Interaction Method and Electronic Device | |
EP3876506A1 (en) | Method for presenting video on electronic device when incoming call comes, and electronic device | |
US20230351048A1 (en) | Application Permission Management Method and Apparatus, and Electronic Device | |
CN114115770B (zh) | 显示控制的方法及相关装置 | |
WO2021052204A1 (zh) | 基于通讯录的设备发现方法、音视频通信方法及电子设备 | |
US12045446B2 (en) | Display interaction system, and display method and device | |
US11973895B2 (en) | Call method and apparatus | |
CN114079893A (zh) | 蓝牙通信方法、终端设备及计算机可读存储介质 | |
CN113641271A (zh) | 应用窗口的管理方法、终端设备及计算机可读存储介质 | |
WO2023088209A1 (zh) | 一种跨设备音频数据传输的方法和电子设备 | |
CN114125793A (zh) | 一种蓝牙数据传输方法及相关装置 | |
WO2022143258A1 (zh) | 一种语音交互处理方法及相关装置 | |
WO2022088964A1 (zh) | 一种电子设备的控制方法和装置 | |
EP4354831A1 (en) | Cross-device method and apparatus for synchronizing navigation task, and device and storage medium | |
WO2023284555A1 (zh) | 安全调用服务的方法、安全注册服务的方法及装置 | |
CN113380240B (zh) | 语音交互方法和电子设备 | |
WO2021129453A1 (zh) | 一种截屏方法及相关设备 | |
WO2024114212A1 (zh) | 跨设备切换焦点的方法、电子设备及*** |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22831155 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 22831155 Country of ref document: EP Kind code of ref document: A1 |