CN116132718A

CN116132718A - Control method and control device of display equipment, display equipment and electronic equipment

Info

Publication number: CN116132718A
Application number: CN202310125075.1A
Authority: CN
Inventors: 潘维
Original assignee: BOE Technology Group Co Ltd
Current assignee: BOE Technology Group Co Ltd
Priority date: 2023-02-07
Filing date: 2023-02-07
Publication date: 2023-05-16

Abstract

The disclosure provides a control method and device of a display device, the display device and an electronic device. The control method comprises the following steps: controlling the display equipment to enter a working mode under the condition that the user watches the display equipment; and under the condition that the display equipment is in the working mode, if the voice control instruction is detected, controlling a corresponding functional module in the display equipment to be adjusted to be in a state matched with the voice control instruction according to the voice control instruction. According to the technical scheme, the defect that the display equipment is controlled by the remote controller in actual life is overcome, and users of all ages can conveniently control the display equipment.

Description

Control method and control device of display equipment, display equipment and electronic equipment

Technical Field

The disclosure relates to the technical field of display, and in particular relates to a control method and device of display equipment, the display equipment and electronic equipment.

Background

With the advancement of society, almost every household has a display device such as a television. At present, a remote controller is adopted to realize functions of switching on and off, program conversion, volume adjustment and the like of a television. However, in life, there often occurs a problem that the remote controller cannot be found, and as the population ages, many elderly people do not operate the remote controller. In addition, users often forget to turn off the television, resulting in wasted power consumption.

Disclosure of Invention

The disclosure provides a control method and device of a display device, the display device and an electronic device.

According to a first aspect of the present disclosure, there is provided a control method of a display apparatus, including:

controlling the display equipment to enter a working mode under the condition that the user watches the display equipment;

and under the condition that the display equipment is in the working mode, if the voice control instruction is detected, controlling a corresponding functional module in the display equipment to be adjusted to be in a state matched with the voice control instruction according to the voice control instruction.

In one embodiment, the control method further comprises:

and controlling the display device to enter a sleep mode under the condition that the user is not watching or does not watch the display device.

In one embodiment, the control method further comprises:

when the display equipment enters a working mode, controlling a play timing module to perform timing of play time;

and under the condition that a voice control instruction is detected and/or the condition that the user watches the display equipment is determined in the playing time, controlling the playing timing module to reset and re-timing the playing time, otherwise, controlling the display equipment to enter a sleep mode.

In one embodiment, the control method further comprises:

And determining whether the user views the display device or not according to the acquired image or voice wake-up instruction.

In one embodiment, determining whether a user views a display device based on the acquired image includes:

and carrying out human shape recognition on the acquired image, and determining that the user watches the display equipment under the condition that the image comprises a preset human shape image, or else, determining that the user does not watch or does not watch the display equipment.

In one embodiment, the preset humanoid image includes at least a user's head and shoulders.

and carrying out gaze recognition on the acquired image, and determining that the user watches the display equipment under the condition that the image comprises a preset gaze image, or else, determining that the user does not watch or does not watch the display equipment.

In one embodiment, the preset gaze image comprises at least the pupil of the user's eye.

In one embodiment, the voice wake instruction includes a wake word, and determining whether the user views the display device according to the collected voice wake instruction includes:

under the condition that a wake-up word is identified from a voice wake-up instruction, controlling the display equipment to enter a wake-up mode, and controlling a wake-up timing module to perform the timing of wake-up time;

And in the wake-up time, determining whether the user views the display device or not according to the acquired image.

In one embodiment, determining whether the user views the display device according to the collected voice wake-up instruction further comprises:

under the condition that a voice control instruction is detected in the wake-up time, determining that a user watches the display equipment;

and controlling the display equipment to enter a working mode, and controlling corresponding functional modules in the display equipment to be adjusted to a state matched with the voice control instruction.

In one embodiment, the voice wake instruction includes a function word, and determining whether the user views the display device according to the collected voice wake instruction includes:

determining that the user views the display device when the function word is identified from the voice wakeup instruction;

and controlling the display equipment to enter a working mode, and controlling corresponding functional modules in the display equipment to be adjusted to a state matched with the functional words.

According to a second aspect of the present disclosure, there is provided a control apparatus of a display device, including:

the mode control unit is used for controlling the display equipment to enter a working mode under the condition that the user is confirmed to watch the display equipment;

and the function control unit is used for controlling the corresponding function module in the display equipment to be adjusted to be matched with the voice control instruction according to the voice control instruction if the voice control instruction is detected under the condition that the display equipment is in the working mode.

According to a third aspect of the present disclosure, there is provided a display apparatus including a display device and a control device of the display apparatus in any one of the embodiments of the present disclosure.

According to a fourth aspect of the present disclosure, there is provided an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the present disclosure.

According to the technical scheme, when the display equipment is watched by a user, the display equipment is controlled to enter a working mode; under the condition that the display equipment is in a working mode, the corresponding functional modules in the display equipment can be regulated to be in a state matched with the voice control instruction through the voice control instruction, so that the control of the display equipment by voice is realized, the defect that the display equipment is controlled by a remote controller in actual life is overcome, and the control of the display equipment by users of all ages is facilitated. And when the display equipment is confirmed to be watched by the user, the display equipment is controlled to enter the working mode, so that the continuous work of the display equipment under the condition of no user watching is avoided, and the power consumption of the display equipment is reduced.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a schematic diagram of a control method of a display device according to an embodiment of the disclosure;

FIG. 2 is a block diagram illustrating a control apparatus of a display device according to an embodiment of the present disclosure;

FIG. 3 is a block diagram of a hardware configuration of a display device according to an embodiment of the present disclosure;

FIG. 4 is a flow chart of image recognition and speech recognition of a memory module according to an embodiment of the disclosure;

FIG. 5 is a schematic workflow diagram of a memory module according to another embodiment of the disclosure;

FIG. 6 is a schematic workflow diagram of a control module according to an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In order to solve the inconvenience caused by the adoption of a remote controller to control the display equipment, the embodiment of the disclosure provides a control method of the display equipment.

Fig. 1 is a schematic diagram of a control method of a display device according to an embodiment of the disclosure. As shown in fig. 1, the control method of the display device may include steps S110 to S120.

In step S110, in the case where it is determined that the user views the display device, the display device is controlled to enter an operation mode.

In step S120, if the voice control command is detected while the display device is in the working mode, the corresponding function module in the display device is controlled to be adjusted to a state matching the voice control command according to the voice control command.

The display device may include a television, a projector, a home theater, a computer, or the like having a display function.

The display device may have an operating mode. The display device is in an operating mode, the screen of the display device is lit, the display device can emit sound, etc. In the case where the display device is in the operation mode, conversion of a display program, volume adjustment, display brightness adjustment, and the like can be performed.

The display device may also have a sleep mode in which the power consumption of the display device is less than the power consumption of the operating mode. In the sleep mode, the screen of the display device is in an unlit state, and the power consumption of the display device is low.

The voice control instruction may be an instruction corresponding to a function of the display device, and for example, the voice control instruction may include a volume adjustment instruction, a program conversion instruction, a display screen brightness adjustment instruction, and the like. The corresponding functional modules can be a volume adjusting module, a program converting module, a brightness adjusting module and the like. When the voice control instruction is detected, the display device can control the corresponding functional module in the display device to be adjusted to be matched with the voice control instruction according to the voice control instruction. For example, the voice control instruction includes a volume adjustment instruction, and then the display device may control the volume adjustment module to adjust to a volume level matching the volume adjustment instruction according to the volume adjustment instruction. The voice control instructions may also include a power-on instruction and a power-off instruction, and correspondingly, the functional module may include a power module for providing power to the display screen. When the power-on instruction or the power-off instruction is detected, the display device can control the power supply module to supply power to the display screen or stop supplying power to the display screen according to the power-on instruction or the power-off instruction.

According to the control method of the display equipment, when the display equipment is watched by a user, the display equipment is controlled to enter a working mode; under the condition that the display equipment is in a working mode, the corresponding functional modules in the display equipment can be controlled by the voice control instruction to be adjusted to be in a state matched with the voice control instruction, so that the control of the display equipment by voice is realized, the defect that the display equipment is controlled by a remote controller in actual life is overcome, and the control of the display equipment by users of all ages is facilitated. And when the display equipment is confirmed to be watched by the user, the display equipment is controlled to enter the working mode, so that the continuous work of the display equipment under the condition of no user watching is avoided, and the power consumption of the display equipment is reduced.

In one embodiment, the control method of the display device may further include: and controlling the display device to enter a sleep mode under the condition that the user is not watching or does not watch the display device.

The user does not watch, and it is understood that although the user exists within the preset range of the display device, the user does not watch the display device. No user viewing is understood to be the absence of a user within a preset range of the display device. In both cases, the display device may be controlled to enter a sleep mode. In the sleep mode, the screen of the display device is in an unlit state, and the power consumption of the display device is lower and smaller than that of the working mode. Therefore, under the condition that the user is not watched or does not watch the display device, the display device is controlled to enter the sleep mode, and the power consumption of the display device can be reduced.

In one embodiment, the control method of the display device may further include: and determining whether the user views the display device according to the acquired image or voice wake-up instruction.

By way of example, the display device may comprise an image acquisition means, such as a camera. The image capturing device may capture an image of the surroundings of the display apparatus.

The display device may comprise a voice acquisition means, such as a microphone. The voice acquisition device can perform voice acquisition. The voice acquisition device can acquire voice control instructions and also can acquire voice wake-up instructions. For example, in the case that the display device is in the working mode, the voice signal collected by the voice collection device may be a voice control instruction; under the condition that the display equipment is in a sleep mode, the voice signal collected by the voice collection device can be a voice wake-up instruction.

In one embodiment, determining whether the user views the display device based on the captured image may include: and carrying out human shape recognition on the acquired image, and determining that the user watches the display equipment under the condition that the image comprises a preset human shape image, or else, determining that the user does not watch or does not watch the display equipment.

For example, after the image is acquired by the image acquisition device, object detection, boundary extraction, human-shaped object matching, human-shaped object recognition, and the like may be performed on the acquired image so as to recognize a human-shaped image from the acquired image.

In the case that the human-shaped image recognized from the collected images matches with the preset human-shaped image, it may be determined that the collected images include the preset human-shaped image, and then it may be determined that the user views the display device. When the identified human form image includes a preset human form image, the identified human form image may be considered to match the preset human form image.

In the case where no human-shaped image exists in the captured image, or in the case where the human-shaped image identified from the captured image does not match the preset human-shaped image, it may be determined that the preset human-shaped image is not included in the image, and then it may be determined that the user is not viewing or does not view the display device.

In one embodiment, the preset humanoid image includes at least a user's head and shoulders. It will be appreciated that at least the head and shoulders of the user may come within the acquisition range of the image acquisition device when the user wants to view the display device. Therefore, the preset humanoid image is set to at least comprise the head and the shoulder of the user, whether the user watches the display equipment can be judged more accurately, and misjudgment is avoided.

For example, the preset human-shaped image may further include an arm of the user or an upper body of the user, so that accuracy in determining whether the user views the display device may be further improved. The preset human-shaped image can be set according to the needs, so long as the user can be determined to watch the display screen through the preset human-shaped image.

In one embodiment, determining whether a user views a display device based on the acquired image includes: and carrying out gaze recognition on the acquired image, and determining that the user watches the display equipment under the condition that the image comprises a preset gaze image, or else, determining that the user does not watch or does not watch the display equipment.

Gaze recognition may be understood as pupil recognition. It will be appreciated that in practice, although the person image identified from the captured image matches the pre-set person image, it is not indicative that the user is viewing the display device, for example, the person image identified from the captured image includes the user's head and shoulders, but the user's head is not facing the display device. In this case, if it is determined that the user views the display device, a judgment error occurs.

The preset gaze image may include at least a pupil of the user's eye. When the acquired image comprises a preset staring image, the user is shown to watch the display screen of the display device. Thus, in the case that a preset gaze image is identified from the acquired images, determining that the user views the display device; in the case that the preset gaze image is not included in the captured image, it may be determined that the user is not viewing or that no user is viewing the display device. In this way, the accuracy of determining whether the user views the display device can be further improved.

In one embodiment, determining whether the user views the display device based on the acquired image may further include: and identifying a preset humanoid image from the acquired images, and determining that the user watches the display equipment under the condition that the distance between the preset humanoid image and the display equipment is within a preset distance range, or determining that the user does not watch or does not watch the display equipment.

The preset distance may be a proper distance for the user to view the display device, and when the distance between the user and the display device is within the preset distance range, a better viewing effect may be obtained.

When the user is too close to or too far from the display device, the user cannot watch the display device well, and thus, in the case that the distance between the preset humanoid image and the display device is not within the preset distance range, it can be judged that the user is not watching or is not watching the display device. Thus, the accuracy of judging whether the user views the display device can be further improved.

For example, the distance between the user and the display device may be determined by presetting the ratio of the humanoid image in the captured image. Alternatively, the distance between the viewer and the display device may be detected by a distance detection means. The distance detection device may detect in synchronization with the image acquisition device.

The preset distance may be, for example, 1m to 5m. The specific value of the preset distance can be determined according to the actual. The value of the preset distance may be smaller for a display device of a small-sized screen and larger for a display device of a large-sized screen.

In one embodiment, determining whether the user views the display device based on the acquired image may further include: and identifying the first stay time of the preset humanoid image in the preset distance range from the acquired image, and determining that the user watches the display equipment under the condition that the first stay time is greater than or equal to the preset stay time, or determining that the user does not watch or does not watch the display equipment.

It will be appreciated that in real life, the user may pass before the display device, and if it is determined that the user views the display device at this time, a determination error may occur. If the user views the display device, the user may stay within a preset distance from the display device for a longer period of time. In this embodiment, the first residence time of the preset humanoid image within the preset distance range is compared with the preset residence time, when the first residence time is greater than or equal to the preset residence time, it may be determined that the user views the display device, or else, it is determined that the user does not view or does not view the display device, and it may be further determined whether the user views the display device.

For example, images may be acquired a plurality of times within a preset acquisition time, and a dwell time of the humanoid image within a preset distance range may be calculated from the plurality of acquired images. Or detecting the time of the human body entering the preset distance range and the time of the human body leaving the preset distance range through the human body sensor, and calculating the two time differences to obtain the residence time of the human-shaped image in the preset distance range.

In one embodiment, determining whether the user views the display device based on the acquired image may further include: and identifying a second stay time of the preset staring image from the acquired image, and determining that the user watches the display equipment under the condition that the second stay time is greater than or equal to the preset stay time, or determining that the user does not watch or does not watch the display equipment.

It is understood that in real life, if a user watches a program of a display device, the pupil of the human eye looks at the display screen of the display device, and the time of continuing to look at the display screen is greater than or equal to a preset time. By judging whether the second dwell time is greater than or equal to the preset dwell time to judge whether the user views the display device, whether the user views the display device can be judged more accurately, and the state of the display device can be controlled more accurately.

For example, the image may be acquired multiple times within a preset acquisition time, and the dwell time of the preset gaze image may be calculated from the multiple acquired images.

In one embodiment, the voice wake instruction may include a wake word. According to the collected voice wake-up instruction, determining whether the user views the display device may include: under the condition that a wake-up word is identified from a voice wake-up instruction, controlling the display equipment to enter a wake-up mode, and controlling a wake-up timing module to perform the timing of wake-up time; and in the wake-up time, determining whether the user views the display device or not according to the acquired image.

Illustratively, the wake-up word may be a preset word or phrase that is capable of waking up the display device. For example, the wake-up word may include the name of the display device or a predefined name for the display device. When the wake-up word is identified from the voice wake-up instruction, it may be determined that the user is about to watch the display device. At this time, the display device may be controlled to enter the awake mode. In the wake-up mode, all the devices are in a power-on standby working state. The power consumption of the display device in the wake-up mode is greater than the power consumption in the sleep mode and less than the power consumption in the operational mode.

When the display equipment enters the wake-up mode, the wake-up timing module is controlled to perform the timing of the wake-up time. The wake-up time may be the time when the display device is in the wake-up mode. The wake-up time may be set as desired, for example, the wake-up time may be 1 minute.

During the wake-up time, it may be determined from the acquired image whether the user is viewing the display device. And if the user watches the display equipment according to the acquired image in the wake-up time, controlling the display equipment to enter a working mode, and resetting the wake-up timing module. And if the user does not watch or does not watch the display equipment according to the acquired image in the wake-up time, controlling the display equipment to enter a sleep mode.

In real life, the user may need to start the display device in advance to perform work, and at this time, the user may not enter the acquisition range of the image acquisition device. The display device may be controlled to enter an operational mode by voice control instructions, for example.

In one embodiment, determining whether the user views the display device according to the collected voice wake-up instruction may further include: under the condition that a voice control instruction is detected in the wake-up time, determining that a user watches the display equipment; and controlling the display equipment to enter a working mode, and controlling corresponding functional modules in the display equipment to be adjusted to a state matched with the voice control instruction.

And when the display equipment is in the state of the wake-up mode, if a voice control instruction is detected in the wake-up time, indicating that the user is about to watch the display equipment, and the display equipment needs to enter the working mode in advance, at the moment, determining that the user watches the display equipment. And then, controlling the display equipment to enter a working mode, and controlling the corresponding functional modules in the display equipment to be adjusted to a state matched with the voice control instruction, so that the requirements of customers can be better met.

If no voice control instruction is detected in the wake-up time, the display device can be controlled to enter a sleep mode when the wake-up time is over.

For example, if it is determined from the acquired image that the user is not viewing or is not viewing the display device during the wake-up time, and no voice control instruction is detected, the display device may be controlled to enter a sleep mode at the end of the wake-up time.

In one embodiment, the voice wake instruction may include a function word, and determining whether the user views the display device according to the collected voice wake instruction includes: determining that the user views the display device when the function word is identified from the voice wakeup instruction; and controlling the display equipment to enter a working mode, and controlling corresponding functional modules in the display equipment to be adjusted to a state matched with the functional words.

The function words may be words or phrases that are preset in relation to the function of the display device. For example, the function words may include words related to the display device function such as "volume", "sound", "brightness", "channel", and the like. When the function word is recognized from the voice wake instruction, it may be determined that the user views the display device. At this time, the display device can be controlled to enter a working mode, and the corresponding function module in the display device is controlled to be adjusted to a state matched with the function word.

In one embodiment, when the display device enters the working mode, the play timing module is controlled to perform timing of the play time; and under the condition that the voice control instruction is detected and/or the condition that the user watches the display equipment is determined in the playing time, controlling the playing timing module to reset and timing the playing time again, otherwise, controlling the display equipment to enter a sleep mode.

When the display device enters the working mode, the playing timing module starts timing the playing time. For example, the playing time is a duration of playing the program by the display device, and the playing time may be a first preset time, for example, 15 minutes. If the voice control instruction is detected within the playing time, the voice control instruction is executed, namely, the corresponding functional module in the display equipment is adjusted to be in a state matched with the voice control instruction according to the voice control instruction. And controlling the play timing module to reset and re-time the play time. If the voice control instruction is not detected within the playing time, the user can be considered to be no longer watching the display device, and the display device can be controlled to enter the sleep mode.

In real life, after the display device enters the working mode, the currently playing program of the display device may be the program that the user wants to watch, and then the voice control instruction may not be detected. In one embodiment, the voice control command is not detected in the playing time, but the user is determined to watch the display device in the playing time, and then the playing timing module is controlled to reset and the timing of the playing time is carried out again.

In one embodiment, if it is determined that the user views the display device during the playing time, the playing timing module is controlled to reset and perform timing of the playing time again. If the display equipment is not watched by the user or is not watched by the user in the playing time, the display equipment can be controlled to enter a sleep mode when the playing time is over, and the power consumption of the display equipment is reduced.

If the voice control instruction is not detected in the playing time and it is determined that the user does not watch or does not watch the display device, the display device can be controlled to enter a sleep mode when the playing time is over, and power consumption of the display device is reduced.

By setting the playing time, the display device can be controlled to enter the sleep mode to save power consumption in a timely manner under the condition that the user does not watch or does not watch the display device, and the power consumption waste caused by long-time work of the display device under the condition that the user does not watch or does not watch the display device is avoided.

According to the control method of the display device, the display device is controlled to enter a working mode under the condition that the user watches the display device; under the condition that the user does not watch or does not watch the display equipment, the display equipment is controlled to enter a sleep mode, so that the waste of power consumption under the condition that the user does not watch or does not watch the display equipment is avoided, and the power consumption of the display equipment is reduced. And under the condition that the display equipment is in a working mode, the corresponding functional module in the display equipment can be controlled by the voice control instruction to be adjusted to be in a state matched with the voice control instruction, so that the voice control of the display equipment is realized, the defect that the display equipment is controlled by adopting a remote controller in actual life is overcome, and the control of users of all ages on the display equipment is facilitated.

Fig. 2 is a block diagram illustrating a control apparatus of a display device according to an embodiment of the present disclosure. An embodiment of the present disclosure further provides a control apparatus 200 of a display device, where the control apparatus 200 includes a mode control unit 201 and a function control unit 202. The mode control unit 201 is configured to control the display device to enter the operation mode if it is determined that the user views the display device. The function control unit 202 is configured to, when detecting a voice control instruction while the display device is in the operation mode, control a corresponding function module in the display device to adjust to a state matching the voice control instruction according to the voice control instruction.

In one embodiment, the mode control unit 201 is further configured to control the display device to enter a sleep mode if it is determined that the user is not viewing or is not viewing the display device.

In one embodiment, the function control unit is further configured to control the play timing module to perform timing of the play time when the display device enters the working mode; the function control unit is also used for controlling the play timing module to reset and re-time the play time under the condition that the voice control instruction is detected and/or the user is confirmed to watch the display equipment in the play time, otherwise, controlling the display equipment to enter the sleep mode.

In one embodiment, the control apparatus of the display device may further include: and the determining unit is used for determining whether the user views the display equipment or not according to the collected image or voice wake-up instruction.

In an embodiment, the determining unit is further configured to perform human shape recognition on the acquired image, and determine that the user views the display device if the image includes a preset human shape image, and otherwise determine that the user does not view or does not view the display device.

In an embodiment, the determining unit is further configured to perform gaze recognition on the acquired image, determine that the user views the display device in case a preset gaze image is included in the image, and otherwise determine that the user does not view or does not view the display device.

In one embodiment, the voice wake-up instruction includes a wake-up word, and the determining unit is further configured to control the display device to enter a wake-up mode and control the wake-up timing module to perform timing of a wake-up time when the wake-up word is identified from the voice wake-up instruction; and in the wake-up time, determining whether the user views the display device or not according to the acquired image.

In an embodiment, the determining unit is further configured to determine that the user views the display device if the voice control instruction is detected within the wake-up time; and controlling the display equipment to enter a working mode, and controlling corresponding functional modules in the display equipment to be adjusted to a state matched with the voice control instruction.

In an embodiment, the voice wake instruction comprises a function word, and the determining unit is further configured to determine that the user views the display device if the function word is identified from the voice wake instruction; and controlling the display equipment to enter a working mode, and controlling corresponding functional modules in the display equipment to be adjusted to a state matched with the functional words.

The embodiment of the disclosure also provides a display device, which comprises a display device and a control device of the display device in any embodiment of the disclosure.

Fig. 3 is a block diagram of a hardware structure of a display device in an embodiment of the present disclosure. Illustratively, as shown in fig. 3, the display apparatus may include a display module, an image capturing device, a voice capturing device, a memory module, a control module, a power module, a sound playing module, and the like. The image acquisition device may include a camera, the voice acquisition device may include a microphone, and the sound playing module may include a sound.

The image acquisition device may acquire images, the acquired images may be used for image recognition, and the image recognition may include human shape recognition or gaze recognition. The voice acquisition device may be used to acquire voice signals, which may include voice wake-up instructions or voice control instructions. The image acquisition device and the voice acquisition device are both connected with the memory calculation module. The memory module can store an image recognition algorithm and a voice recognition algorithm and output an algorithm result. The calculation module can adopt an image recognition algorithm to carry out image recognition on the collected image, can also adopt a voice recognition algorithm to carry out voice recognition on the collected voice signal, and outputs an image recognition result and a voice recognition result. The image recognition may include human shape recognition and/or gaze recognition for determining whether the user is viewing the display device. Speech recognition is used to recognize speech signals, which may include voice wake-up instructions or voice control instructions.

The control module is in communication with the memory module, and is capable of executing the control method of the display device in any embodiment of the disclosure, and controlling the display module and the sound playing module according to the image recognition result and the sound recognition result.

The display module may include a display screen and a display controller. The power supply module can realize alternating current/direct current conversion and supply power for the display equipment.

Illustratively, the memory module may include a memory integrated chip and the control module may include a control chip.

Storage wall: under classical von neumann architecture, data storage and computation are separated, and data exchange is performed between the processor and the memory through a data bus. But there is also a great difference in performance due to the internal structure, process and packaging of the processor and memory. Since 1980, the performance gap between the processor and the memory is continuously increased, the access speed of the memory is far from the data processing speed of the processor, a memory wall is formed between the memory and the processor, and the overall performance improvement of the chip is seriously restricted.

Power consumption wall: under classical von neumann architecture, the processor and memory are separate. In the process of processing data, the data is firstly transported from the memory to the processor through the bus, and after the processing is completed, the data is transported back to the memory for storage. The energy consumption of the data in the process of carrying is 4-1000 times of that of floating point operation. As semiconductor processes progress, the overall power consumption decreases, but the power consumption ratio occupied by data handling becomes larger and larger.

The integrated memory operation is to convert the design centered on calculation in the traditional von neumann architecture into the design centered on data storage, namely, the data is operated by using a memory, thereby avoiding the memory wall and the power consumption wall generated by data carrying and greatly improving the parallelism and the energy efficiency of the data. The architecture is particularly suitable for terminal equipment requiring high computing power and low power consumption, such as wearable equipment, mobile equipment, intelligent home and the like. The memory-computing integrated chip is embedded with an algorithm on the memory particles, so that the memory unit has a computing function. In this way, the data does not need a separate operation part to complete the calculation, but completes the storage and calculation in the storage unit, thereby eliminating the delay and the power consumption of the data access and storage, and being a true storage calculation fusion. Meanwhile, as the calculation is completely coupled with the storage, the parallelism with finer granularity can be developed, and higher performance and energy efficiency are obtained.

The core idea of the integrated memory and calculation chip is to perform data processing in a memory, and the integrated memory and calculation chip can play unique advantages aiming at the characteristics of intensive memory access and calculation regularity in an artificial intelligent algorithm. The memory module utilizes the memory integrated chip to realize the image recognition algorithm and the voice recognition algorithm, and the control module realizes the control of the display module and the sound playing module, thereby overcoming the defect of adopting a remote controller and reducing the power consumption of the display device.

Fig. 4 is a schematic flow chart of image recognition and voice recognition of the memory module in an embodiment of the disclosure. As shown in fig. 4, the system is initialized after the display device is powered on. The image acquisition device acquires images, and the storage module performs image recognition according to the acquired images, wherein the image recognition can comprise human shape recognition and/or gaze recognition. The storage module can determine whether a person is present according to the image recognition result, namely, whether the person views the display device. The storage module can transmit the image recognition result to the control module, and the control module controls the display device according to the image recognition result. The voice acquisition device acquires voice signals, and the memory module performs voice recognition according to the acquired voice signals, wherein the voice recognition can comprise recognition of voice control instructions and/or recognition of voice wake-up instructions. The calculation module can transmit the voice recognition result to the control module, and the control module controls the display device according to the voice recognition result.

FIG. 5 is a schematic workflow diagram of a memory module according to another embodiment of the disclosure. The flow shown in fig. 5 may be applied to an operation mode of a display device. As shown in fig. 5, the system is initialized after the display device is powered on. The image acquisition device acquires images, and the calculation storage module performs image identification according to the acquired images. When the user watches the display equipment, the image recognition result is that the voice recognition timing is started, and the image recognition result is transmitted to the control module, so that the control module performs mode control on the display equipment according to the image recognition result.

And in the voice recognition timing time, the voice acquisition device acquires voice signals, and the memory module performs voice recognition. Under the condition that the voice recognition timing is not overtime, voice signal acquisition and voice recognition can be performed for a plurality of times. And when the voice control instruction is detected within the voice recognition timing time, transmitting the voice control instruction to the control module so that the control module can control the functions of the display equipment according to the voice control instruction. And when the voice control instruction is not detected within the voice recognition timing time, namely the voice recognition timing is overtime, the image acquisition device re-acquires the image. That is, in the case where the voice control instruction is not recognized, it is necessary to detect whether the user views the display device at intervals.

FIG. 6 is a schematic workflow diagram of a control module according to an embodiment of the disclosure. The flowchart shown in fig. 6 may be applied to a control method of a display device in an embodiment of the present disclosure. As shown in fig. 6, the system is initialized after the display device is powered on. Illustratively, after the initialization is completed, the wake-up timer is started, that is, the wake-up timer module is controlled to count the wake-up time. And in the wake-up time, the control module receives the image recognition result of the storage module and determines whether a person exists or not (namely, whether the user watches the display device or not) according to the image recognition result. Illustratively, the control module may determine whether a person is present based on the image recognition result; or the calculation module determines whether a person exists according to the image recognition result and sends the information of whether the person exists to the control module. When the control module determines that a person exists, the wake-up timing module is controlled to reset, the display module is opened, and the display module plays a default program or picture. When the display module is opened, the playing timing is started, namely, the playing timing module is controlled to perform timing of playing time. And under the condition that the control module receives the voice control instruction in the playing time of the display module, the control module executes the voice control instruction, namely, controls the corresponding functional module in the display equipment to be adjusted to be in a state matched with the voice control instruction according to the voice control instruction, resets the playing timing, namely, controls the playing timing module to reset and re-perform the timing of the playing time. If the control module does not receive the voice control instruction in the playing time of the display module, but determines that someone is present (i.e. the user watches the display device), the control module controls the playing timing module to reset and re-perform the timing of the playing time. If the control module does not receive the voice control instruction within the playing time of the display module and determines that no person is present (i.e. the user is not watching or is not watching the display device), the control module controls the display device to enter a sleep mode.

Determining no person (i.e. the user is not watching or is not watching the display device) in the wake-up time, and if the wake-up time is not overtime, determining whether a person exists or not for many times; and if the unmanned state is determined and the wake-up time is overtime, the control module controls the display device to enter a sleep mode.

When the display equipment is in the sleep mode, a wake-up signal from the memory module is received, the control module controls the display equipment to enter the wake-up mode, and controls the wake-up timing module to wake-up timing.

The communication mode between the memory module and the control module can be UART, RS485 or Ethernet, etc. Illustratively, the communication protocol data frame format may be as shown in table 1.

Table 1 communication protocol data frame format

Table 2 command code and function correspondence table

/>

It should be noted that table 1 only shows the format of the communication protocol data frame by way of example, and in practical implementation, the format of the communication protocol data frame may be set according to needs. Table 2 shows the functions corresponding to the command codes by way of example only, and in practical implementation, the command codes of the respective functions may be set as needed, and are not limited to the form in table 2.

In one embodiment, human body sensors may be used for human shape detection.

In one embodiment, the user may be identified as being in face recognition to control the display device to enter an operational mode if it is determined that the target user is viewing the display device.

The display device may be: any product or component with display function such as a mobile phone, a tablet computer, a television, a display, a notebook computer, a digital photo frame, a navigator and the like.

According to an embodiment of the disclosure, the disclosure further provides an electronic device. An electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor. Wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the display control method of any one of the embodiments of the present disclosure.

The embodiments of the present disclosure also provide a readable storage medium storing a computer program which, when executed by a processor, implements a display control method as in any of the embodiments of the present disclosure.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A control method of a display device, characterized by comprising:

and under the condition that the display equipment is in a working mode, if a voice control instruction is detected, controlling a corresponding functional module in the display equipment to be adjusted to a state matched with the voice control instruction according to the voice control instruction.

2. The method as recited in claim 1, further comprising:

and controlling the display equipment to enter a sleep mode under the condition that the user is not watching or does not watch the display equipment.

3. The method as recited in claim 1, further comprising:

4. The method as recited in claim 1, further comprising:

And determining whether the user watches the display equipment or not according to the acquired image or voice wake-up instruction.

5. The method of claim 4, wherein determining whether the user is viewing the display device based on the captured image comprises:

6. The method of claim 5, wherein the pre-set humanoid image includes at least a user's head and shoulders.

7. The method of claim 4, wherein determining whether the user is viewing the display device based on the captured image comprises:

and carrying out gaze recognition on the acquired image, and determining that the user watches the display equipment under the condition that the image comprises a preset gaze image, otherwise, determining that the user does not watch or does not watch the display equipment.

8. The method of claim 7, wherein the preset gaze image comprises at least a pupil of a user's eye.

9. The method of claim 4, wherein the voice wake instruction comprises a wake word, and wherein determining whether the user is viewing the display device based on the collected voice wake instruction comprises:

under the condition that a wake-up word is identified from the voice wake-up instruction, controlling the display equipment to enter a wake-up mode, and controlling a wake-up timing module to perform the timing of wake-up time;

10. The method of claim 9, wherein determining whether the user is viewing the display device based on the collected voice wake-up instruction, further comprises:

11. The method of claim 4, wherein the voice wake instruction comprises a function word, and wherein determining whether the user is viewing the display device based on the collected voice wake instruction comprises:

Determining that a user views the display device under the condition that a function word is identified from the voice wakeup instruction;

12. A control apparatus of a display device, comprising:

a mode control unit, configured to control the display device to enter a working mode when it is determined that the user views the display device;

13. A display device comprising display means and control means of the display device according to claim 12.

14. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.