CN115641840A - Vehicle voice control method and device, terminal equipment and readable storage medium - Google Patents
Vehicle voice control method and device, terminal equipment and readable storage medium Download PDFInfo
- Publication number
- CN115641840A CN115641840A CN202211183254.2A CN202211183254A CN115641840A CN 115641840 A CN115641840 A CN 115641840A CN 202211183254 A CN202211183254 A CN 202211183254A CN 115641840 A CN115641840 A CN 115641840A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- voice
- control
- information
- lip
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to the technical field of vehicles, in particular to a method, a device, terminal equipment and a computer readable storage medium for vehicle voice control, wherein the method comprises the following steps: the method comprises the steps that a vehicle control voice command is obtained through a vehicle-based sound collection device, and visual identification information of a vehicle is obtained through an image collection device of the vehicle; detecting a lip change state of an in-vehicle person of the vehicle when the presence of the person in the vehicle is determined according to the visual identification information; and controlling the vehicle to execute the operation of the vehicle control voice command according to the lip change state. The invention effectively improves the overall safety of the vehicle.
Description
Technical Field
The present invention relates to the field of vehicle technologies, and in particular, to a method and an apparatus for vehicle voice control, a terminal device, and a computer-readable storage medium.
Background
With the continuous development of the internet of vehicles technology and the continuous increase of the demand of people on intelligent vehicles in production and life, vehicle intellectualization has become the development trend of the coming vehicle enterprises, and the vehicle voice control also becomes the label of the intelligent vehicle. Therefore, the user puts higher requirements on the safety of the intelligent vehicle for controlling the whole vehicle according to the vehicle voice control mode.
The existing vehicle voice control mode receives a language instruction after a user wakes up a vehicle machine every time, and a whole vehicle system of the vehicle executes the language instruction no matter the language instruction is sent by the user in the vehicle or outside the vehicle, so that the mode has great defects, for example, when someone in the vehicle of the vehicle stops and has a rest, when personnel outside the vehicle bound with the vehicle system controls the vehicle to open a vehicle window through vehicle voice, personal and property safety of the personnel in the vehicle can be threatened, and the whole vehicle system has safety loopholes.
In conclusion, the existing vehicle voice control mode has the technical problem that the safety of the whole vehicle is low.
Disclosure of Invention
The invention mainly aims to provide a method, a device, terminal equipment and a computer readable storage medium for vehicle voice control, and aims to optimize a flow for vehicle overall control through vehicle voice control so as to improve the safety of the entire vehicle.
In order to achieve the above object, the present invention provides a method for vehicle voice control, comprising:
the method comprises the steps that a vehicle control voice command is obtained through a vehicle-based sound collection device, and visual identification information of a vehicle is obtained through an image collection device of the vehicle;
when the person is determined to exist in the vehicle according to the visual identification information, detecting the lip change state of the person in the vehicle of the vehicle;
and controlling the vehicle to execute the operation of the vehicle control voice command according to the lip change state.
Optionally, the image capturing device includes a first camera and a second camera, and the step of obtaining the visual identification information of the vehicle by the image capturing device of the vehicle includes:
acquiring front row image information of the vehicle through the first camera;
acquiring rear-row image information of the vehicle through the second camera;
and summarizing the front row image information and the rear row image information to obtain the visual identification information of the vehicle.
Optionally, the step of detecting a lip change state of an occupant in the vehicle includes:
controlling the image acquisition device to perform face recognition on the person in the vehicle so as to acquire lip information of the person in the vehicle;
and acquiring the lip change state of the personnel in the vehicle when outputting the voice information based on the lip information.
Optionally, after the step of obtaining the lip change state of the vehicle occupant when outputting the voice information based on the lip information, the method further includes:
and controlling the vehicle not to execute the operation of the vehicle control voice instruction when the fact that the person in the vehicle does not send the voice information is determined based on the lip information.
Optionally, after the step of acquiring the visual identification information of the vehicle through the image acquisition device of the vehicle, the method further comprises:
and when the fact that no person exists in the vehicle is determined according to the visual identification information, controlling the vehicle not to execute the operation of the vehicle control voice instruction.
Optionally, after the step of controlling the vehicle to perform the operation of the car control voice command according to the lip change state, the method further comprises:
and determining a real-time execution condition corresponding to the vehicle control voice instruction executed by the vehicle, and broadcasting the real-time execution condition through a voice assistant of the vehicle.
Optionally, the sound collection device includes a user terminal microphone bound to the vehicle and a microphone of the vehicle, and the step of acquiring the vehicle control voice command by the vehicle-based sound collection device includes:
waking up a voice assistant through a user terminal microphone bound with the vehicle or a microphone of the vehicle;
and acquiring a vehicle control voice instruction based on the voice assistant.
In order to achieve the above object, the present invention also provides a vehicle voice control apparatus, comprising:
the system comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring a vehicle control voice command based on a sound acquisition device of a vehicle and acquiring visual identification information of the vehicle through an image acquisition device of the vehicle;
the detection module is used for detecting the lip change state of people in the vehicle of the vehicle when the people in the vehicle are determined to exist according to the visual identification information;
and the control module is used for controlling the vehicle to execute the operation of the vehicle control voice command according to the lip change state.
The individual functional modules of the voice-controlled device of the vehicle according to the invention implement the steps of the voice-controlled method of the vehicle according to the invention as described above when in operation.
In addition, in order to achieve the above object, the present invention further provides a terminal device, which includes a memory, a processor, and a vehicle voice control program stored in the memory and operable on the processor, wherein the vehicle voice control program implements the steps of the vehicle voice control method when executed by the processor.
Further, to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a program for vehicle voice control, which when executed by a processor, implements the steps of the above-described method for vehicle voice control.
According to the method, the vehicle control language instruction is obtained through the sound collecting device of the vehicle, then the visual identification information in the vehicle is obtained through the image collecting device of the vehicle, then the in-vehicle personnel corresponding to the visual identification information are determined, and the vehicle is controlled to execute the operation of the vehicle control voice instruction through the lip change state corresponding to the in-vehicle personnel.
Be different from current vehicle speech control's mode, language instruction and visual identification information are obtained respectively to the image acquisition device of voice assistant sound collection system and vehicle that this application combines the vehicle, then confirm according to visual identification information the vehicle exists personnel, detect the interior personnel's of this vehicle lip change state, then change the operation of state control vehicle execution vehicle control instruction through sound collection system input according to lip, avoided applying current vehicle speech control mode effectively, when nobody or interior personnel are at the rest in the car, cause easily losing the interior article of vehicle car to and the phenomenon of damage vehicle or interior personnel of vehicle takes place, and then improved the factor of safety of vehicle effectively, improved the whole car security of vehicle promptly.
Drawings
FIG. 1 is a schematic flow chart diagram of a first embodiment of a method of voice control of a vehicle of the present invention;
FIG. 2 is a flowchart illustrating an exemplary application of a method for voice control of a vehicle according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a vehicle voice-controlled device module according to the present invention;
fig. 4 is a schematic structural diagram of a terminal device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
An embodiment of the present invention provides a method for vehicle voice control, and as shown in fig. 1, fig. 1 is a schematic flow diagram of a first embodiment of the method for vehicle voice control according to the present invention.
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present application.
In this embodiment, the method for vehicle voice control according to the present invention is applied to a terminal device that controls a vehicle to perform a vehicle control command operation, and the method for vehicle voice control according to the present invention includes:
step S10: the method comprises the steps that a vehicle control voice command is obtained through a vehicle-based sound collection device, and visual identification information of a vehicle is obtained through an image collection device of the vehicle;
in this embodiment, referring to fig. 2, fig. 2 is a schematic view of a specific application flow related to an embodiment of a method for controlling a vehicle by using voice, in which a terminal device wakes up a voice assistant by using a voice acquisition device, then obtains a vehicle control language instruction of the vehicle by using the voice assistant, and obtains visual identification information of the vehicle by using an image acquisition device of the vehicle, that is, the terminal device can be understood as summarizing front-row image information acquired by a first camera of the vehicle and rear-row image information acquired by a second camera of the vehicle to obtain the visual identification information of the vehicle.
It should be noted that the sound collection device may be understood as a user terminal microphone bound to the vehicle, and may also be understood as a microphone device of the vehicle itself.
The voice assistant can be understood as an application program for realizing human-vehicle language interaction and can also be understood as a human-vehicle language interaction mode, human-vehicle conversation is completed through intelligent voice interaction, operation can be directly completed through one instruction, theoretically, any function can be directly achieved, driving behaviors of a driver through eyes, hands and feet are not influenced, and the voice assistant is safer, more humanized and more direct compared with button operation.
The vehicle control language instruction can be understood as that the terminal device obtains an instruction sent by a user through a voice assistant to control the vehicle condition, such as controlling the vehicle to open or close a vehicle window, controlling the vehicle to adjust a vehicle-mounted air conditioner, and the like.
The visual identification information may also be referred to as a visual identification perception result, and may be understood as an accurate perception of the environmental information in the vehicle by using the image acquisition device as a sensor input through a series of calculations and processing, for example, the terminal device obtains the real-time environmental condition in the vehicle through the image acquisition device.
For example, after the terminal device determines that the user wakes up the language assistant, when the voice assistant acquires a vehicle control voice command of "open a window", the terminal device simultaneously acquires real-time environment conditions in the vehicle through an image acquisition device of the vehicle, wherein the image acquisition device can be understood as an in-vehicle camera of the vehicle, and can include a camera arranged in the front row of the vehicle and a camera arranged in the rear row of the vehicle.
Step S20: detecting a lip change state of an in-vehicle person of the vehicle when the presence of the person in the vehicle is determined according to the visual identification information;
in this embodiment, when the terminal device determines that a person is present in the vehicle according to the visual recognition information, the terminal device controls the image capturing device of the vehicle to perform facial recognition on the person in the vehicle of the vehicle to obtain lip information of the person in the vehicle, and then detects whether the lip of the person in the vehicle is in a state when the voice information is output according to the lip information, if the lip of the person in the vehicle is in the state when the voice information is output, that is, the terminal device obtains the lip change state of the person in the vehicle when the voice information is output based on the lip information.
It should be noted that the lip change state may be understood as a state when the lip moves while the vehicle occupant of the vehicle is communicating with the voice assistant, for example, the lip change when the vehicle occupant says "open the window" to the voice assistant.
In addition, it should be noted that, in some possible embodiments, before the step of detecting the lip change state of the person in the vehicle of the vehicle when the person is determined to be present in the vehicle according to the visual recognition information, the method for vehicle voice control may further include:
whether the vehicle has the person in the vehicle is detected according to the visual identification information, namely, the terminal equipment detects whether the visual identification information shoots the image of the person in the vehicle of the vehicle.
For example, referring to fig. 2, fig. 2 is a schematic diagram of a specific application flow related to an embodiment of a method for controlling a vehicle by voice according to the present invention, and a terminal device may detect whether a driver of the vehicle is driving the vehicle according to visual identification information, may detect whether a co-driver of the vehicle is driving the vehicle by a real person according to the visual identification information, and may detect whether a rear seat of the vehicle is driving by a real person according to the visual identification information.
Step S30: and controlling the vehicle to execute the operation of the vehicle control voice command according to the lip change state.
In this embodiment, after the terminal device obtains the lip change state of the vehicle occupant when outputting the voice information based on the lip information, it can be understood that the lip change state of the vehicle occupant matches the lip movement state information of the vehicle occupant preset by the vehicle, and then the terminal device controls the vehicle to execute the operation of the vehicle control language instruction.
It should be noted that the vehicle control voice command can also be understood as an anthropomorphic action of the vehicle, such as opening or closing a window, a door, a vehicle-mounted air conditioner, and the like by the vehicle itself.
For example, after determining that the lip change state of the person in the vehicle matches with the preset lip movement state of the person in the vehicle, the terminal device controls the vehicle to open the window by itself according to the command of "opening the window" acquired by the voice assistant, wherein the vehicle control voice command includes, but is not limited to, the command of "opening the window", and also includes other operations related to the demand of the person in the vehicle.
According to the method, the vehicle control language instruction is obtained through the sound collecting device of the vehicle, then the visual identification information in the vehicle is obtained through the image collecting device of the vehicle, then the in-vehicle personnel corresponding to the visual identification information are determined, and the vehicle is controlled to execute the operation of the vehicle control voice instruction through the lip change state corresponding to the in-vehicle personnel.
Be different from current vehicle speech control's mode, language instruction and visual identification information are obtained respectively to the image acquisition device of voice assistant sound collection system and vehicle that this application combines the vehicle, then confirm according to visual identification information the vehicle exists personnel, detect the interior personnel's of this vehicle lip change state, then change the operation of state control vehicle execution vehicle control instruction through sound collection system input according to lip, avoided applying current vehicle speech control mode effectively, when nobody or interior personnel are at the rest in the car, cause easily losing the interior article of vehicle car to and the phenomenon of damage vehicle or interior personnel of vehicle takes place, and then improved the factor of safety of vehicle effectively, improved the whole car security of vehicle promptly.
Further, a second embodiment of the vehicle voice control of the invention is proposed based on the first embodiment of the vehicle voice control of the invention.
In this embodiment, the number of the cameras configured for the vehicle is at least two, the first camera is arranged on a front seat of the vehicle, and the second camera is arranged on a rear seat of the vehicle, wherein the first camera and the second camera include, but are not limited to, 360 ° blind-corner-free cameras; the step S10: the step of collecting the visual identification information of the vehicle through the camera of the vehicle may include:
step S101: acquiring front row image information of the vehicle through the first camera;
in this embodiment, the terminal device collects front row image information of the vehicle through the first camera of the vehicle.
The front image information may be understood as an environmental condition of the front of the vehicle captured by the first camera.
Step S102: acquiring rear-row image information of the vehicle through the second camera;
in the embodiment, the terminal device collects the front row image information of the vehicle through the second camera of the vehicle.
The front-row image information may be understood as an environmental condition of the front row of the vehicle captured by the second camera.
Step S103: and summarizing the front row image information and the rear row image information to obtain the visual identification information of the vehicle.
In the embodiment, the terminal device collects the front row image information of the vehicle through the first camera of the vehicle and collects the front row image information of the vehicle through the second camera of the vehicle to obtain the visual identification information of the vehicle.
Further, in some possible embodiments, the step S20: detecting a lip change state of an occupant of the vehicle may further include:
step S201: controlling the image acquisition device to perform face recognition on the person in the vehicle so as to acquire lip information of the person in the vehicle;
in this embodiment, the terminal device first controls the image capture device of the vehicle to directly lock onto the lips of the persons in the vehicle of the vehicle, and then identifies the lips of the persons in the vehicle according to a preset face identification algorithm to obtain the lip information of the persons in the vehicle.
Step S202: and acquiring the lip change state of the personnel in the vehicle when outputting the voice information based on the lip information.
In this embodiment, the terminal device acquires the lip change state of the vehicle occupant when outputting the voice information based on the lip information.
In this embodiment, if the terminal device determines that the lip of the vehicle occupant of the main driver, the passenger driver, or the rear seat is in a state in which the voice information is uttered, the lip change state is acquired, that is, the mouth of the vehicle occupant of the main driver, the passenger driver, or the rear seat is in a lip movement state.
For example, referring to fig. 2, fig. 2 is a schematic diagram of a specific application flow related to an embodiment of a method for controlling a vehicle by voice according to the present invention, where when a terminal device detects that a person is actually present in a main driver of the vehicle according to visual recognition information, an image capturing device of the vehicle is first controlled to recognize lips of the person in the main driver by a preset face recognition algorithm to obtain lip information of the person in the main driver, and then a lip change state of the person in the main driver when the voice information is output is obtained based on the lip information.
In other possible embodiments, if the terminal device can further detect that a person actually exists in the passenger compartment or the rear seat of the vehicle according to the visual identification information, the terminal device controls the image acquisition device of the vehicle to recognize lips of the person in the passenger compartment or the rear seat of the vehicle through a preset face recognition algorithm to acquire lip information of the person in the passenger compartment or the rear seat of the vehicle, and then acquires a lip change state of the person in the passenger compartment or the rear seat of the vehicle when outputting the voice information based on the lip information.
In some possible embodiments, before the step of obtaining the lip change state of the person in the vehicle when outputting the voice information based on the lip information, the method for vehicle voice control may further include:
the terminal equipment judges whether the lip of the person in the vehicle is in a state when the voice information is output according to the lip information, namely the terminal equipment judges whether the person in the vehicle on the main driver of the vehicle has lip movement according to the lip information, and the terminal equipment judges whether the person in the vehicle on the assistant driver or the rear seat of the vehicle has lip movement according to the lip information.
Further, in other possible embodiments, in the step S202: after the lip change state of the in-vehicle person when outputting the voice information is acquired based on the lip information, the method for controlling the vehicle voice may further include:
step A10: and controlling the vehicle not to execute the operation of the vehicle control voice instruction when the fact that the person in the vehicle does not send the voice information is determined based on the lip information.
In this embodiment, the terminal device controls the vehicle not to perform the operation of the vehicle control voice command when it is determined that the person in the vehicle has not uttered the voice information based on the lip information.
For example, referring to fig. 2, if the terminal device determines from the lip information that there is no lip movement of the person in the vehicle on which the vehicle is driving, the vehicle is controlled not to respond to the vehicle control type language instruction. The lip information includes, but is not limited to, whether lip movement exists or not when the passenger drives the vehicle, and if the passenger drives the vehicle, the vehicle is controlled not to respond to the vehicle control type language instruction; the lip information can also be understood as judging whether the persons in the vehicle on the rear seats have lip movement or not, and if the persons in the vehicle on the rear seats do not have lip movement, controlling the vehicle not to respond to the vehicle control language instruction. In other words, if the terminal device determines that there is no lip movement by the vehicle occupant in the main driver, the passenger driver, and the rear seat of the vehicle based on the lip information, the vehicle is controlled not to respond to the vehicle control type language command of "open window".
In some possible embodiments, in step 10 above: after the visual recognition information of the vehicle is acquired by the image acquisition device of the vehicle, the method for vehicle voice control may further include:
step B10: and when the fact that no person exists in the vehicle is determined according to the visual identification information, controlling the vehicle not to execute the operation of the vehicle control voice instruction.
In this embodiment, the terminal device needs to detect whether the vehicle has a person in the vehicle according to the visual identification information, and control the vehicle not to execute the operation of the vehicle control voice instruction after determining that the person does not exist in the vehicle according to the visual identification information.
For example, referring to fig. 2, if the terminal device detects that there is no person in the vehicle based on the visual recognition information, the vehicle is controlled not to respond to the vehicle control type language command. In other words, if the terminal device detects that no real person exists in the main driver, the assistant driver and the rear seat of the vehicle according to the visual identification information, the vehicle is controlled not to respond to the vehicle control type language instruction of opening the window.
In some possible embodiments, in step 30 above: after controlling the vehicle to perform the operation of the vehicle control voice command according to the lip change state, the vehicle voice control method may further include:
step C10: and determining a real-time execution condition corresponding to the vehicle control voice instruction executed by the vehicle, and broadcasting the real-time execution condition through a voice assistant of the vehicle.
In this embodiment, the terminal device first determines a real-time execution situation corresponding to the vehicle control language instruction executed by the vehicle, and broadcasts the real-time execution situation through a voice assistant.
For example, the terminal device collects the real-time execution situation of the vehicle control language instruction executed by the vehicle, for example, the real-time execution situation of 'opening a vehicle window', and then feeds the situation of 'opening the vehicle window' back to the voice assistant for broadcasting, so that a user can conveniently know the current situation of the vehicle control instruction executed by the vehicle.
In other possible embodiments, the sound collecting device includes a user terminal microphone bound to the vehicle and a microphone of the vehicle, and the step S10: the step of obtaining the vehicle control voice command by the vehicle-based sound collection device can further comprise:
step S101: and awakening the voice assistant through a user terminal microphone bound with the vehicle or a microphone of the vehicle.
In this embodiment, the terminal device may wake up the voice assistant through a microphone of the user terminal bound to the vehicle, or may wake up the voice assistant through a microphone of the vehicle.
It should be noted that the user terminal may be a mobile phone, a tablet, a personal computer, or the like.
In addition, it should be noted that the area where the voice assistant is awakened by the user terminal microphone bound to the vehicle may be understood as overlapping with the communication area covered by the vehicle, that is, outside the communication area of the vehicle, the voice assistant may not be awakened by the user terminal microphone bound to the vehicle.
In conclusion, the visual identification information in the vehicle is provided to the voice assistant through the camera in the vehicle, and the voice assistant establishes the safety strategy by combining the visual identification information in the vehicle, so that whether the vehicle performs vehicle control voice control or not can be controlled by detecting whether the vehicle has the personnel in the vehicle or not and judging whether the personnel in the vehicle moves on the basis of determining whether the vehicle has the personnel in the vehicle, and the safety of the whole vehicle of the vehicle is effectively improved.
Furthermore, the invention also provides a device for controlling the vehicle by voice. Referring to fig. 3, fig. 3 is a schematic diagram of a vehicle voice-controlled device module according to the present invention.
The voice control apparatus for a vehicle of the present invention includes:
the acquisition module H01 is used for acquiring a vehicle control voice instruction based on a sound acquisition device of a vehicle and acquiring visual identification information of the vehicle through an image acquisition device of the vehicle;
the detection module H02 is used for detecting the lip change state of the person in the vehicle when the person in the vehicle is determined to exist according to the visual identification information;
and the control module H03 is used for controlling the vehicle to execute the operation of the vehicle control voice command according to the lip change state.
Optionally, the obtaining module H01 may further include:
the front row acquisition unit is used for acquiring front row image information of the vehicle through a first camera;
the rear row acquisition unit is used for acquiring rear row image information of the vehicle through a second camera;
and the summarizing unit is used for summarizing the front row image information and the rear row image information to obtain the visual identification information of the vehicle.
Optionally, the detection module H02 may further include:
the identification unit is used for controlling the image acquisition device to perform face identification on the person in the vehicle so as to acquire lip information of the person in the vehicle;
and the lip change unit is used for acquiring the lip change state of the person in the vehicle when the person outputs the voice information based on the lip information.
Optionally, the detection module H02 may further include:
and the first operation unit is used for controlling the vehicle not to execute the operation of the vehicle control voice instruction when the fact that the person in the vehicle does not send the voice information is determined based on the lip information.
Optionally, the obtaining module H01 may further include:
a second operation unit configured to control the vehicle not to perform an operation of the vehicle control voice instruction when it is determined that there is no person in the vehicle according to the visual recognition information;
optionally, the control module H03 may further include:
and the broadcasting unit is used for determining the real-time execution condition corresponding to the vehicle control voice command executed by the vehicle and broadcasting the real-time execution condition through a voice assistant of the vehicle.
Optionally, the obtaining module H01 may further include:
the wake-up unit is used for waking up the voice assistant through a user terminal microphone bound with the vehicle or a microphone of the vehicle;
and the vehicle control voice acquisition unit is used for acquiring a vehicle control voice instruction based on the voice assistant.
The individual functional modules of the voice-controlled device of the vehicle according to the invention implement the steps of the voice-controlled method of the vehicle according to the invention as described above when in operation.
In addition, the invention also provides terminal equipment. Referring to fig. 4, fig. 4 is a schematic structural diagram of a terminal device according to an embodiment of the present invention. The terminal equipment of the embodiment of the invention can be equipment for voice control of a locally operated vehicle.
As shown in fig. 4, the terminal device according to the embodiment of the present invention may include: a processor 1001, e.g. a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a Wi-Fi interface).
A memory 1005 is provided on the terminal apparatus main body, and the memory 1005 stores a program that realizes a corresponding operation when executed by the processor 1001. The memory 1005 is also used to store parameters for use by the terminal device. The memory 1005 may be a high-speed RAM memory or a non-volatile memory such as a disk memory. The memory 1005 may alternatively be a storage device separate from the processor 1001 described previously.
Those skilled in the art will appreciate that the terminal device configuration shown in fig. 4 does not constitute a limitation of the terminal device and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 4, a memory 1005, which is a kind of storage medium, may include therein an operating system, a network communication module, a user interface module, and an intelligent connection program of a terminal device.
In the terminal device shown in fig. 4, the processor 1001 may be configured to call the intelligent connection program of the terminal device stored in the memory 1005 and execute the steps of the various embodiments of the inventive vehicle voice control method, wherein the steps of the inventive vehicle voice control method may include:
the method comprises the steps that a vehicle control voice command is obtained through a vehicle-based sound collection device, and visual identification information of a vehicle is obtained through an image collection device of the vehicle;
detecting a lip change state of an in-vehicle person of the vehicle when the presence of the person in the vehicle is determined according to the visual identification information;
and controlling the vehicle to execute the operation of the vehicle control voice command according to the lip change state.
Further, the image capturing device includes a first camera and a second camera, and the step of obtaining the visual identification information of the vehicle by the image capturing device of the vehicle includes:
collecting front row image information of the vehicle through the first camera;
acquiring rear-row image information of the vehicle through the second camera;
and summarizing the front row image information and the rear row image information to obtain the visual identification information of the vehicle.
Further, the step of detecting a lip change state of an occupant in the vehicle includes:
controlling the image acquisition device to perform face recognition on the person in the vehicle so as to acquire lip information of the person in the vehicle;
and acquiring the lip change state of the personnel in the vehicle when outputting the voice information based on the lip information.
Further, after the step of obtaining the lip change state of the vehicle occupant when outputting the voice information based on the lip information, the method further includes:
and controlling the vehicle not to execute the operation of the vehicle control voice instruction when the fact that the person in the vehicle does not send the voice information is determined based on the lip information.
Further, after the step of acquiring the visual identification information of the vehicle through the image capturing device of the vehicle, the method further includes:
and when the fact that no person exists in the vehicle is determined according to the visual identification information, controlling the vehicle not to execute the operation of the vehicle control voice instruction.
Further, after the step of controlling the vehicle to perform the operation of the vehicle control voice command according to the lip change state, the method further includes:
and determining a real-time execution condition corresponding to the vehicle control voice instruction executed by the vehicle, and broadcasting the real-time execution condition through a voice assistant of the vehicle.
Further, the sound collection device includes a user terminal microphone bound with the vehicle and a microphone of the vehicle, and the step of acquiring the vehicle control voice command by the vehicle-based sound collection device includes:
waking up a voice assistant through a user terminal microphone bound with the vehicle or a microphone of the vehicle;
and acquiring a vehicle control voice command based on the voice assistant.
In addition, the invention also provides a computer readable storage medium. Referring to fig. 5, fig. 5 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
The present invention also provides a computer readable storage medium having stored thereon a vehicle voice control program, the vehicle voice control program when executed by a processor implementing the steps of the vehicle voice control method of the present invention, wherein the steps of the vehicle voice control method of the present invention may include:
the method comprises the steps that a vehicle control voice command is obtained through a vehicle-based sound collection device, and visual identification information of a vehicle is obtained through an image collection device of the vehicle;
detecting a lip change state of an in-vehicle person of the vehicle when the presence of the person in the vehicle is determined according to the visual identification information;
and controlling the vehicle to execute the operation of the vehicle control voice command according to the lip change state.
Further, the image capturing device includes a first camera and a second camera, and the step of obtaining the visual identification information of the vehicle by the image capturing device of the vehicle includes:
acquiring front row image information of the vehicle through the first camera;
acquiring rear-row image information of the vehicle through the second camera;
and summarizing the front row image information and the rear row image information to obtain the visual identification information of the vehicle.
Further, the step of detecting a lip change state of an occupant in the vehicle includes:
controlling the image acquisition device to perform face recognition on the person in the vehicle so as to acquire lip information of the person in the vehicle;
and acquiring the lip change state of the personnel in the vehicle when outputting the voice information based on the lip information.
Further, after the step of obtaining the lip change state of the vehicle occupant when outputting the voice information based on the lip information, the method further includes:
and controlling the vehicle not to execute the operation of the vehicle control voice instruction when the fact that the person in the vehicle does not send the voice information is determined based on the lip information.
Further, after the step of acquiring the visual recognition information of the vehicle through the image capturing device of the vehicle, the method further includes:
and when the fact that no person exists in the vehicle is determined according to the visual identification information, controlling the vehicle not to execute the operation of the vehicle control voice instruction.
Further, after the step of controlling the vehicle to perform the operation of the car control voice command according to the lip change state, the method further includes:
and determining a real-time execution condition corresponding to the vehicle control voice instruction executed by the vehicle, and broadcasting the real-time execution condition through a voice assistant of the vehicle.
Further, the sound collection device includes a user terminal microphone bound with the vehicle and a microphone of the vehicle, and the step of acquiring the vehicle control voice command by the vehicle-based sound collection device includes:
waking up a voice assistant through a user terminal microphone bound with the vehicle or a microphone of the vehicle;
and acquiring a vehicle control voice instruction based on the voice assistant.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present invention or directly or indirectly applied to other related technical fields are also included in the scope of the present invention.
Claims (10)
1. A method of vehicle voice control, the method comprising:
the method comprises the steps that a vehicle control voice command is obtained through a vehicle-based sound collection device, and visual identification information of a vehicle is obtained through an image collection device of the vehicle;
detecting a lip change state of an in-vehicle person of the vehicle when the presence of the person in the vehicle is determined according to the visual identification information;
and controlling the vehicle to execute the operation of the vehicle control voice command according to the lip change state.
2. The method of voice control for a vehicle according to claim 1, wherein the image capturing device includes a first camera and a second camera, and the step of acquiring the visual recognition information of the vehicle through the image capturing device for the vehicle includes:
collecting front row image information of the vehicle through the first camera;
acquiring rear-row image information of the vehicle through the second camera;
and summarizing the front row image information and the rear row image information to obtain the visual identification information of the vehicle.
3. The method of voice control for a vehicle of claim 1, wherein the step of detecting a lip change state of an occupant of the vehicle comprises:
controlling the image acquisition device to perform face recognition on the person in the vehicle so as to acquire lip information of the person in the vehicle;
and acquiring the lip change state of the personnel in the vehicle when outputting the voice information based on the lip information.
4. The method of voice control for a vehicle according to claim 3, wherein after the step of acquiring the lip change state of the vehicle occupant when outputting voice information based on the lip information, the method further comprises:
and controlling the vehicle not to execute the operation of the vehicle control voice instruction when the fact that the person in the vehicle does not send the voice information is determined based on the lip information.
5. The method of voice control of a vehicle according to claim 1, wherein after the step of acquiring the visual recognition information of the vehicle by an image capturing device of the vehicle, the method further comprises:
and when the fact that no person exists in the vehicle is determined according to the visual identification information, controlling the vehicle not to execute the operation of the vehicle control voice instruction.
6. The method of voice control for a vehicle according to claim 1, wherein after the step of controlling the vehicle to perform the operation of the vehicle control voice command according to the lip variation state, the method further comprises:
and determining a real-time execution condition corresponding to the vehicle control voice instruction executed by the vehicle, and broadcasting the real-time execution condition through a voice assistant of the vehicle.
7. The method for voice control of a vehicle according to claim 1, wherein the sound collection device comprises a user terminal microphone bound with the vehicle and a microphone of the vehicle, and the step of acquiring the vehicle control voice command by the vehicle-based sound collection device comprises:
waking up a voice assistant through a user terminal microphone bound with the vehicle or a microphone of the vehicle;
and acquiring a vehicle control voice command based on the voice assistant.
8. A vehicle voice-controlled apparatus, characterized in that the vehicle voice-controlled apparatus comprises:
the system comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring a vehicle control voice command based on a sound acquisition device of a vehicle and acquiring visual identification information of the vehicle through an image acquisition device of the vehicle;
the detection module is used for detecting the lip change state of people in the vehicle of the vehicle when the people in the vehicle are determined to exist according to the visual identification information;
and the control module is used for controlling the vehicle to execute the operation of the vehicle control voice command according to the lip change state.
9. A terminal device, characterized in that the terminal device comprises a memory, a processor and a vehicle voice control program stored on the memory and operable on the processor, and the processor implements the steps of the vehicle voice control method according to any one of claims 1 to 7 when executing the vehicle voice control program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a program for vehicle voice control, which when executed by a processor implements the steps of the method for vehicle voice control according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211183254.2A CN115641840A (en) | 2022-09-27 | 2022-09-27 | Vehicle voice control method and device, terminal equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211183254.2A CN115641840A (en) | 2022-09-27 | 2022-09-27 | Vehicle voice control method and device, terminal equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115641840A true CN115641840A (en) | 2023-01-24 |
Family
ID=84941072
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211183254.2A Pending CN115641840A (en) | 2022-09-27 | 2022-09-27 | Vehicle voice control method and device, terminal equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115641840A (en) |
-
2022
- 2022-09-27 CN CN202211183254.2A patent/CN115641840A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110047487B (en) | Wake-up method and device for vehicle-mounted voice equipment, vehicle and machine-readable medium | |
| CN108725357B (en) | Parameter control method and system based on face recognition and cloud server | |
| US8442820B2 (en) | Combined lip reading and voice recognition multimodal interface system | |
| US9865258B2 (en) | Method for recognizing a voice context for a voice control function, method for ascertaining a voice control signal for a voice control function, and apparatus for executing the method | |
| CN111190480A (en) | Control device, agent device, and computer-readable storage medium | |
| WO2023273064A1 (en) | Object speaking detection method and apparatus, electronic device, and storage medium | |
| CN112026790B (en) | Control method and device for vehicle-mounted robot, vehicle, electronic device and medium | |
| US11577688B2 (en) | Smart window apparatus, systems, and related methods for use with vehicles | |
| CN111176434A (en) | Gaze detection device, computer-readable storage medium, and gaze detection method | |
| WO2018230654A1 (en) | Interaction device, interaction method, and program | |
| CN112667084B (en) | Control method and device for vehicle-mounted display screen, electronic equipment and storage medium | |
| CN112309395A (en) | Man-machine conversation method, device, robot, computer device and storage medium | |
| CN115268334A (en) | Vehicle window control method, device, equipment and storage medium | |
| CN111192583B (en) | Control device, agent device, and computer-readable storage medium | |
| CN111717136A (en) | Vehicle-mounted multimedia equipment, control method thereof and automobile | |
| CN111144539A (en) | Control device, agent device, and computer-readable storage medium | |
| CN110705483B (en) | Driving reminding method, device, terminal and storage medium | |
| CN115641840A (en) | Vehicle voice control method and device, terminal equipment and readable storage medium | |
| US20220219717A1 (en) | Vehicle interactive system and method, storage medium, and vehicle | |
| CN111210814B (en) | Control device, agent device, and computer-readable storage medium | |
| CN112418162B (en) | Method, device, storage medium and apparatus for controlling vehicle | |
| CN116204253A (en) | Voice assistant display method and related device | |
| CN111422200B (en) | Method and device for adjusting vehicle equipment and electronic equipment | |
| CN111857638A (en) | Voice interaction method and system based on face recognition and automobile | |
| CN113997898B (en) | Living body detection method, apparatus, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |