WO2021082947A1 - 电子设备及非易失性存储介质 - Google Patents

电子设备及非易失性存储介质 Download PDF

Info

Publication number
WO2021082947A1
WO2021082947A1 PCT/CN2020/121423 CN2020121423W WO2021082947A1 WO 2021082947 A1 WO2021082947 A1 WO 2021082947A1 CN 2020121423 W CN2020121423 W CN 2020121423W WO 2021082947 A1 WO2021082947 A1 WO 2021082947A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
display
operation image
item
voice
Prior art date
Application number
PCT/CN2020/121423
Other languages
English (en)
French (fr)
Inventor
久富秀一
小野寺祐
大村弦
西村勇人
澤岡俊宏
北村哲也
Original Assignee
海信视像科技股份有限公司
东芝视频解决方案株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 海信视像科技股份有限公司, 东芝视频解决方案株式会社 filed Critical 海信视像科技股份有限公司
Priority to CN202080004679.5A priority Critical patent/CN112602330B/zh
Publication of WO2021082947A1 publication Critical patent/WO2021082947A1/zh
Priority to US17/644,536 priority patent/US12039228B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04Q—SELECTING
    • H04Q9/00—Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom

Definitions

  • the embodiments of the present invention relate to electronic devices and non-volatile storage media.
  • Patent Document 1 Japanese Patent Application Publication No. 2009-125077
  • the above-mentioned voice input can be used not only for the operation of the game, but also for the technology of remotely operating a video device such as a television and a video recorder by performing a voice operation input to an electronic device such as a smartphone. In this case, it is desired to reduce the load (difficulty level) of the operation input to the sound of the electronic device and improve the convenience.
  • the electronic device of the embodiment includes a display processing unit and a voice operation processing unit.
  • the display processing unit displays the first operation image including the display of the operation item for operation input on the display unit, and when the predetermined condition is satisfied in the display of the first operation image, displays the second operation image on the display
  • the second operation image is an operation image in which text information corresponding to the operation item in the first operation image is highlighted.
  • the voice operation processing unit receives the operation input of the voice corresponding to the text information highlighted in the second operation image as an operation on the video device.
  • FIG. 1A is an illustrative and schematic diagram showing an example of a situation where the technology of the embodiment is applied;
  • FIG. 1B is an illustrative and schematic diagram showing another example of a situation where the technology of the embodiment is applied;
  • FIG. 2 is an exemplary and schematic block diagram showing the hardware configuration of the information terminal of the embodiment
  • FIG. 3 is an exemplary and schematic block diagram showing the functions of the information terminal and the video device of the embodiment
  • FIG. 4 is an exemplary and schematic diagram showing the processing performed by the information terminal and the video device of the embodiment in the form of a flowchart
  • FIG. 5 is an illustrative and schematic diagram showing an example of a touch operation image of the embodiment
  • FIG. 6 is an illustrative and schematic diagram showing another example of the touch operation image of the embodiment.
  • FIG. 7 is an illustrative and schematic diagram showing an example of a sound operation image of the embodiment.
  • FIG. 8 is an exemplary and schematic diagram for explaining retrieval via voice manipulation images according to the embodiment.
  • FIG. 9 is an illustrative and schematic diagram showing another example of the sound operation image of the embodiment.
  • FIG. 1A is an illustrative and schematic diagram showing an example of a situation where the technology of the embodiment is applied.
  • the technique of the embodiment is applied as an example to a situation where an information terminal 100 such as a smartphone is used to remotely operate a video device 120 such as a video recorder 121 and a television 122.
  • the information terminal 100 is an example of "electronic equipment”.
  • the information terminal 100 and the video device 120 are communicably connected to each other via a relay device 110 such as a router.
  • a relay device 110 such as a router
  • the information terminal 100 and the video device 120 may also be connected via Connect to a network like the Internet.
  • FIG. 1B is an illustrative and schematic diagram showing another example of a situation where the technology of the embodiment is applied.
  • the information terminal 100 and the video device 120 are directly connected (peer-to-peer) without passing through the relay device 110.
  • the technology of the embodiment can also be applied to a situation in which the video device 120 is remotely operated by the information terminal 100 based on such a connection relationship.
  • the information terminal 100 shown in FIGS. 1A and 1B has the following hardware configuration as shown in FIG. 2.
  • FIG. 2 is an exemplary and schematic block diagram showing the hardware configuration of the information terminal of the embodiment.
  • the information terminal 100 includes a communication interface (I/F) 101, a voice input interface 102, a sensor group 103, a display unit 104, a graphics controller 105, a touch panel controller 106, a CPU (Central Processing Unit) Processor) 107, and memory 108.
  • the information terminal 100 may also include a camera, a speaker, and the like.
  • the communication interface 101 is a device used for communication with other devices such as the video device 120 via the relay device 110 or the like.
  • the voice input interface 102 is a voice input device such as a microphone, and detects voice input by the user of the information terminal 100.
  • the sensor group 103 includes, for example, at least any one of an acceleration sensor, an orientation sensor, and a gyroscope sensor.
  • the acceleration sensor is a sensor device configured to detect the direction and magnitude of the acceleration of the information terminal 100 when the information terminal 100 is moved.
  • the orientation sensor is a sensor device configured to detect the orientation of the information terminal 100.
  • the gyro sensor is a sensor device configured to detect the angular velocity of the information terminal 100 when the information terminal 100 is rotated.
  • the display unit 104 is configured as a so-called touch panel device in which a display 104a and a touch panel 104b are combined.
  • the display 104a is configured as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display, for example.
  • the touch panel 104b detects a touch position on the display screen of the display 104a touched by a user's finger, a touch pen, or the like.
  • the graphics controller 105 controls the display 104 a of the display unit 104.
  • the touch panel controller 106 controls the touch panel 104b of the display unit 104, and acquires coordinate data indicating the touched position on the display screen touched by the user from the touch panel 104b.
  • the CPU 107 controls various components of the information terminal 100 by executing various computer programs.
  • the memory 108 includes ROM (Read Only Memory), RAM (Random Access Memory, random access processor), etc., and stores various computer programs and various data used in various arithmetic processes executed by the CPU 107.
  • the embodiment realizes the remote operation of the video device 120 using the information terminal 100. More specifically, in the embodiment, the information terminal 100 transmits the video from the communication interface 101 based on the user's manual operation input detected via the touch panel 104b or the user's voice operation input detected via the voice input interface 102 The video device 120 operates according to the instruction from the information terminal 100 according to the instruction from the device 120.
  • FIG. 3 is an exemplary and schematic block diagram showing the functions of the information terminal 100 and the video device 120 according to the embodiment.
  • the information terminal 100 includes a communication processing unit 311, a display processing unit 312, a manual operation processing unit 313, and a voice operation processing unit 314.
  • the communication processing unit 311 uses the communication interface 101 to perform control of communication with the video device 120.
  • the display processing unit 312 uses the graphics controller 105 to perform control of the display 104a.
  • the manual operation processing unit 313 executes control related to the user's manual operation input via the touch panel 104b.
  • the voice operation processing unit 314 performs control related to the operation input of the user's voice via the voice input interface 102.
  • the video device 120 includes a communication processing unit 321, a content management unit 322, a first operation image generation unit 323, a second operation image generation unit 324, and an instruction execution unit 325.
  • the communication processing unit 321 performs control of communication with the information terminal 100.
  • the content management unit 322 performs management of the content recorded in the video device 120 (for example, recorded video, etc.).
  • the first operation image generation unit 323 generates a touch operation image as a first operation image that should be displayed on the information terminal 100 in order to receive a user's manual operation input.
  • the second operation image generation unit 324 generates a voice operation image as a second operation image that should be displayed on the information terminal 100 in order to receive an operation input by the user's voice.
  • specific examples of the first operation image and the second operation image will be described later.
  • the instruction execution unit 325 controls each component (each function) of the video device 120 so that the operation corresponding to the operation instruction sent from the information terminal 100 is executed in accordance with the operation input of the user input to the information terminal 100 via a touch operation image or a sound operation image Actions.
  • the information terminal 100 and the video device 120 of the embodiment execute processing according to the flow shown in FIG. 4 below.
  • FIG. 4 is an exemplary and schematic diagram showing the processing performed by the information terminal 100 and the video device 120 of the embodiment in a flowchart form.
  • S411 when the communication processing unit 311 of the information terminal 100 realizes the remote operation of the video device 120, it requests the video device 120 for a touch operation image (touch operation Information required for image display) is sent.
  • the processing of S411 is executed, for example, when the user of the information terminal 100 executes an application program for realizing remote operation of the video device 120 on the information terminal 100 side, and the application program provided to the display 104a, for example, When a predetermined operation is performed on the initial screen (not shown) called the main menu or the like.
  • the communication processing unit 321 of the video device 120 receives the request from the information terminal 100, in S421, the first operation image generation unit 323 of the video device 120 generates a touch operation image (information necessary for the display of the touch operation image). ), the communication processing unit 321 of the video device 120 transmits the generated touch operation image to the information terminal 100. Then, if the communication processing unit 311 of the information terminal 100 receives the touch operation image from the video device 120, in S412, the display processing unit 312 of the information terminal 100 displays the received touch operation image on the display 104a.
  • FIG. 5 is an illustrative and schematic diagram showing an example of a touch operation image of the embodiment.
  • the image 500 shown in FIG. 5 is displayed on the display 104a when the user of the information terminal 100 performs a predetermined operation on the initial screen provided by the application program described above.
  • buttons, icons, or areas representing operation items are displayed.
  • Manual operation input touch or click input
  • Item 501 is an item for receiving manual operation input corresponding to a switching operation from a touch operation image to a sound operation image
  • item 502 is an item for receiving manual operation input corresponding to an update operation of the touch operation image.
  • items 503 to 511 are items for receiving manual operation input corresponding to various operations on the video device 120. These items 503 to 511 are provided based on the information transmitted from the video device 120 in accordance with a request for transmission of a touch operation image by the information terminal 100.
  • the item 503 is an item for receiving a retrieval operation of the content recorded in the video device 120.
  • the item 504 is an item for receiving an operation for displaying on-air programs (list of programs) currently being received by the video device 120 on the information terminal 100 or the video device 120.
  • the item 505 is an item for receiving an operation to display a recorded program (a list of programs) recorded in the video device 120 on the information terminal 100 or the video device 120.
  • the item 506 is an item for receiving an operation to display a recorded program (a list of programs) recorded in the video device 120 on the information terminal 100 or the video device 120 in the form of a program list.
  • the item 507 is an item for receiving an operation to cause the recorded program recorded in the video device 120 to be displayed on the information terminal 100 or the video device 120 at a higher playback speed.
  • the item 508 is an item for receiving an operation for displaying an image imitating the UI of the remote controller for operating the video device 120 on the information terminal 100.
  • the item 509 is an item for receiving an operation to display the ranking of a program on the information terminal 100 or the video device 120, wherein the program is collected by the video device 120 installed in multiple places. Programs provided by the operator’s server, etc., with reservations for recording or viewing.
  • the item 510 is an item for receiving an operation of downloading a program list from a server or the like and displaying it on the information terminal 100.
  • the item 511 is an item for receiving an operation for displaying a list of programs subject to a recording reservation on the information terminal 100 or the video device 120.
  • the item 512 is an item for receiving an operation to display the initial screen of an application program on the information terminal 100, where the application program is an application program executed in the information terminal 100 in order to realize the remote operation of the video device 120 .
  • This item 512 is also provided by the application program in the same way as the aforementioned items 501 and 502.
  • FIG. 6 is an illustrative and schematic diagram showing another example of the touch operation image of the embodiment.
  • the image 600 shown in FIG. 6 is displayed on the display 104a of the information terminal 100 in a form of switching from the image 500. Therefore, the image 600 shown in FIG. 6 displays a list of programs in at least any one of terrestrial digital broadcasting, BS (Broadcasting Satellites) broadcasting, and CS (Communication Satellites) broadcasting.
  • BS Broadcasting Satellites
  • CS Common Communication Satellites
  • buttons, icons, or areas of operation items of manual operation input corresponding to various operations of the device 120.
  • the items 601, 602, and 610 are the same as the aforementioned items 501, 502, and 512, respectively, and therefore, the description is omitted here.
  • the item 603 is an item for receiving an operation to call up the list of operation items similar to the above-mentioned image 500.
  • item 604 is to receive a list of broadcast programs of terrestrial wave digital broadcasting displayed in image 600, a list of broadcast programs of BS broadcast in image 600, or broadcast programs of CS broadcast in image 600 The item to perform the selected operation.
  • items 605 to 609 are items for receiving an operation of selecting an on-air program from the list of on-air programs corresponding to the selection result in the item 604 to cause the video device 120 to display the on-air program. Therefore, the user of the information terminal 100 can perform a manual operation input (touch or click input) of any one of the selection items 605 to 609 to cause the video device 120 to display and play the selected broadcast program.
  • the instruction execution unit 325 of the video device 120 controls each component (each function) of the video device 120 to Make it perform the action corresponding to the operation instruction.
  • one of the objectives of the embodiments is to reduce the load (difficulty level) of the operation input of the voice for remotely operating the video device 120 using the information terminal 100 as described above, and to improve the convenience.
  • the display processing unit 312 of the embodiment displays the voice operation image as the second operation image that can achieve the above-mentioned purpose in the information.
  • the display 104a of the terminal 100 On the display 104a of the terminal 100.
  • the communication processing unit 311 of the information terminal 100 requests the video device 120 for a sound operation image (image Display the required information) to send. Then, in S423, the second operation image generating unit 324 of the video device 120 generates a sound operation image (information necessary for the display of the image) based on the reception of the request from the information terminal 100, and the communication processing unit of the video device 120 321 transmits the generated sound operation image to the information terminal 100. Then, in S415, the display processing unit 312 of the information terminal 100 displays the sound operation image on the display 104a based on the reception of the sound operation image from the video device 120.
  • FIG. 7 is an illustrative and schematic diagram showing an example of a sound operation image of the embodiment.
  • the image 700 shown in FIG. 7 is configured to have a certain degree of transparency.
  • a manual operation input to the item 501 is performed in the image 500 shown in FIG. 5, it is displayed in a form overlapping with the image 500 On the display 104a of the information terminal 100.
  • the image 700 with a certain transparency is displayed on the display 104a in a form overlapping with the image 500 is described.
  • the image 700 with no transparency may be displayed from the image The 500 switching mode is displayed on the display 104a.
  • the overlap of the image 700 is not limited to the case where the manual operation input to the item 501 is performed, and when the information terminal 100 is moved in a predetermined direction or position, or a predetermined sound is input to the information In the case of the terminal 100, it can also be executed when a predetermined condition set in advance is satisfied.
  • items 703 to 712 are displayed in an image 700 that is an example of a voice operation image.
  • the items 703 to 712 are configured to correspond to the items 503 to 503 to operation items in the image 500 shown in FIG.
  • the text information (at least a part of the information) corresponding to 512 is highlighted and displayed at positions overlapping the items 503 to 512.
  • the item 703 is an item configured to highlight text information (character string) such as “search” which is part of the “cross-database search” in the item 503.
  • the item 704 is an item configured to highlight text information such as “programs on the air” which is part of the “programs on the air” in the item 504.
  • the item 705 is an item configured to emphasize and display text information such as "recorded program” which is part of the "recorded program” in the item 505.
  • the item 706 is an item configured to highlight text information such as "past program list” which is a part of "view past program list” in the item 506.
  • the item 707 is an item configured to highlight text information such as “short time” which is part of the “short time view” in the item 507.
  • the item 708 is an item configured to highlight all the text information of "remote control" in the item 508.
  • the item 709 is an item configured to highlight text information such as “ranking” which is a part of “reservation from rank” in the item 509.
  • the item 710 is an item configured to highlight text information such as "program schedule” which is part of the "reservation from the program schedule” in the item 510.
  • the item 711 is an item configured to highlight all the text information of the "reservation list” in the item 511.
  • the item 712 is an item configured to highlight text information such as “main menu” which is a part of the “go to main menu” in the item 512.
  • the text information highlighted in the above-mentioned items 703 to 712 functions as guidance (keywords) for voice operation input. Therefore, by inputting the text information highlighted in the items 703 to 712 as voice into the information terminal 100 while viewing the image 700, the user can obtain the same result as the case where the items 503 to 512 are manually input.
  • the display of the item 703 changes in the format shown in FIG. 8 below.
  • FIG. 8 is an exemplary and schematic diagram for explaining retrieval by means of voice manipulation images according to the embodiment.
  • search box 800 In the vicinity of the search box 800, a display contains a prompting search word.
  • the user can input the search term in the search box 800 by uttering the search term after uttering the text information "input" in the item 801.
  • a bubble-shaped item 802 including text information such as “cancel” for canceling the input of the search term is displayed in the vicinity of the search frame 800 instead of the above-mentioned item 801.
  • the item 713 that is not displayed in the image 500 is displayed in the image 700.
  • the item 713 is an item for receiving an operation to scroll the image 700. Similar to the above-mentioned items 703 to 712, the item 713 also emphasizes and displays text information such as “next” which is a guidance of voice operation input. Therefore, the user can perform scrolling of the image 700 by inputting text information such as “next” highlighted in the item 713 into the information terminal 100 as a sound.
  • the embodiment uses the items 703 to 712 to emphasize the text information used as the keywords of the voice-based operation input, thereby reducing the load (difficulty level) of the voice operation input and improving the convenience.
  • the positions of the items 703 to 712 and the items 503 to 512 are overlapped with each other, thereby further improving convenience.
  • the text information used as the keyword of the voice operation input is determined in advance. Therefore, in the embodiment, only when the highlighted text information is input as a voice, the corresponding operation is performed, and when text information other than the highlighted text information is input as a voice, the corresponding operation is not performed. Perform any operation.
  • the following configuration may be adopted: multiple modes of text information predetermined as keywords for voice operation input are prepared, and one of the multiple modes is used as voice information.
  • the corresponding operation is performed.
  • “short time” in item 507 corresponds to keywords such as "Jidan (Japanese pronunciation)", which is a regular pronunciation, and keywords such as "Yidan (Japanese pronunciation)", etc. Correspondence is also established.
  • the sound operation image is not limited to the image 700 described above.
  • a voice operation image corresponding to the image 600 shown in FIG. 6 there is also an image 900 shown in FIG. 9 below.
  • Fig. 9 is an illustrative and schematic diagram showing another example of a sound operation image of the embodiment.
  • an operation input of a voice corresponding to the text information "program on the air" of the item 704 is performed in the image 700 shown in FIG. 7, or the operation input of the item 601 is performed in the image 600 shown in FIG.
  • the image 900 shown in FIG. 9 is displayed on the display 104a of the information terminal 100.
  • the items 903 to 910 corresponding to the items 603 to 610 as the operation items in the image 600 shown in FIG.
  • item 904 is configured to integrate text information such as "terrestrial digital", "BS”, and "CS” as guidance for voice input for realizing the same operation as the above-mentioned item 604 (refer to FIG. 6).
  • the item 604 overlaps the position of the highlighted item. Therefore, the user can select whether to display a list of broadcast programs of terrestrial digital broadcasting in the image 900 by inputting any of the text information of "Terrestrial Digital", "BS”, and "CS” by voice.
  • the image 900 still displays a list of broadcast programs broadcast by the BS, or the image 900 displays the broadcast programs broadcast by the CS.
  • items 905 to 909 are items configured to be highlighted by enclosing text information as guidance for voice input for realizing the same operations as the above-mentioned items 605 to 609 (refer to FIG. 6) with a bubble.
  • the text information highlighted in the items 905 to 909 constitutes simple (short) text information that is not related to the corresponding program title and is composed of numbers and text alone or in combination.
  • the embodiment emphasizes and displays simple (short) text information as a guide for voice input as described above, thereby improving convenience.
  • the item 910 is the same item as the above-mentioned item 710 (refer FIG. 7). Therefore, if textual information such as “main menu” highlighted in the item 910 is input by voice, the initial screen of the application program executed in the information terminal 100 is displayed on the information terminal 100.
  • items 911 and 912 are items for receiving an operation to scroll the image 900. For example, if text information such as "previous" highlighted in item 911 is input by voice, image 900 scrolls upward, and text information such as "next" highlighted in item 912 is input by voice , The image 900 scrolls downward.
  • the communication processing unit 311 of the information terminal 100 sends a command to the video device 120
  • the video device 120 executes an operation instruction corresponding to the operation input.
  • switching from the voice operation image to the touch operation image can be performed.
  • the instruction execution unit 325 of the video device 120 controls each component (each function) of the video device 120 to Causes the execution of the action corresponding to the operation instruction.
  • the above-mentioned functional module group shown in FIG. 3 is implemented by the cooperation of hardware and software as a result of the CPU 107 executing a predetermined application program stored in the memory 108 or the like.
  • at least a part of the above-mentioned functional module group shown in FIG. 3 may be realized as dedicated hardware (circuit).
  • the above-mentioned application program may be provided in a state preloaded in a storage device such as the memory 108, or may be provided as a computer program product recorded in a form that can be installed in a storage medium or a form that can be executed.
  • the storage medium is a computer-readable non-volatile storage medium such as various magnetic disks such as a floppy disk or various optical disks such as DVD (Digital Versatile Disk).
  • the above-mentioned application program may also be provided or distributed via a network such as the Internet. That is, the above-mentioned application program may be provided in the form of being stored on a computer connected to a network such as the Internet and downloaded from the computer via the network.
  • the information terminal 100 of the embodiment includes the display processing unit 312 and the voice operation processing unit 314.
  • the display processing unit 312 is configured to display a touch operation image (refer to FIGS. 5 and 6) as a first operation image that displays an operation item for receiving a manual operation input corresponding to an operation on the video device 120 on the display unit On 104 (display 104a), when the predetermined condition is satisfied in the display of the touch operation image, the sound operation image ( 7 and 9) are displayed on the display unit 104.
  • the voice operation processing unit 314 is configured to receive a voice operation input corresponding to the text information highlighted in the voice operation image as an operation on the video device 120 during the display of the voice operation image.
  • the text information highlighted in the voice operation image can be used as a guidance for the voice operation input. Therefore, it is possible to reduce the load (difficulty level) of the operation input to the voice of the information terminal 100, and to improve the convenience.
  • the voice operation processing unit 314 is configured such that (the voice corresponding to the text information highlighted in the voice operation image is not received as an operation input that is different from the voice as the operation on the video device 120 , And) only the operation input of the sound corresponding to the text information highlighted in the sound operation image is received as the operation of the video device 120. According to such a configuration, the operation of the video device 120 can be realized only when the operation input of the sound is accurately performed.
  • the voice operation processing unit 314 associates the voice operation input with the voice mode corresponding to the text information highlighted in the second operation image, and associates the voice mode with the operation on the video device 120. Establish correspondence. According to such a configuration, it is possible to easily associate a voice operation input with an operation to the video device 120. Therefore, for example, in the case where the sound patterns of the text information differ depending on the region due to dialects or the like, the sounds of these different patterns can be associated with the same operation.
  • the first sound mode "higa si da gou ji o ken sa ku (Japanese pronunciation)” and the second sound mode “si ga si da gou ji o ken sa ku (Japanese pronunciation)” and “su ga su da gou ji o ken sa ku (Japanese pronunciation)”
  • This third sound mode is different as the sound mode, but it is considered that the difference is only an error caused by the dialect. Therefore, it is beneficial to establish correspondences between these three sound modes and the operation input of "Search Higashida Koji", which is used to search Higa Sida Gouji (Japanese pronunciation), which is a specific personal voice. .
  • the voice operation processing unit 314 may be configured to associate a plurality of modes of voice with a voice corresponding to the text information emphasized in the voice operation image, and one of the plurality of modes of voice One operation input is received as an operation on the video device 120 corresponding to the operation input of the voice corresponding to the text information highlighted in the voice operation image. According to such a configuration, for example, when there are a plurality of patterns in which the sound of the text information is emitted depending on the region due to dialects or the like, it is possible to handle appropriately.
  • the display processing unit 312 is configured to be able to highlight text information corresponding to the operation item in the touch operation image on the voice operation image at a position overlapping the operation item in the touch operation image. According to such a configuration, it is possible to easily recognize the correspondence between the voice operation input via the voice operation image and the manual operation input via the touch operation image.
  • the display processing unit 312 is configured to also be able to display text information corresponding to the operation item in the touch operation image in a state surrounded by a bubble on the operation item in the touch operation image on the voice operation image. The nearby location is highlighted. According to such a configuration, it is also possible to easily recognize the correspondence between the voice operation input via the voice operation image and the manual operation input via the touch operation image.
  • the display processing unit 312 is configured such that the operation items displayed in the touch operation image correspond to the operation corresponding to the content (the above-mentioned on-air program, recorded program, etc.) output from the video device 120 Next, the text information in the voice operation image is displayed as identification information composed of a single or a combination of at least one of predetermined text and numbers. According to such a configuration, by using the identification information at the time of voice operation input, for example, it is possible to easily perform operations on content such as display playback and stop.
  • the display processing unit 312 superimposes the sound operation image on the touch operation image in a display format in which the operation item in the touch operation image is transmitted through the sound operation image. According to such a configuration, it is possible to easily recognize the correspondence between the operation items displayed in the touch operation image and the character information highlighted in the voice operation image.
  • a configuration in which a transmissive sound operation image is displayed in a form superimposed on the touch operation image is exemplified.
  • a voice operation image that does not have transparency may be displayed in a form that is switched from the touch operation image.
  • the configuration in which the text information in the voice operation image is highlighted at a position overlapping with the operation item in the touch operation image or a position in the vicinity has been exemplified.
  • the text information and the operation items exist in separate locations, as long as the text information serving as the guidance of the voice operation input is highlighted, it is possible to reduce the load (difficulty level) of the voice operation input to the information terminal 100 and This effect improves convenience.
  • the configuration in which the text information corresponding to the operation on the content is displayed as identification information composed of at least one of predetermined characters and numbers alone or in combination has been exemplified.
  • the entire title of the content may be highlighted in the voice operation image as text information serving as a guide for the voice operation input.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Acoustics & Sound (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Selective Calling Equipment (AREA)
  • Telephone Function (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请涉及电子设备及非易失性存储介质。在为了操作视像装置而对信息终端进行声音的操作输入的情况下，降低对电子设备的声音的操作输入的负荷(难易程度)，提高便利性。实施方式的电子设备具备显示处理部以及声音操作处理部。显示处理部将包含操作输入用的操作项目的显示在内的第1操作图像显示在显示部，在第1操作图像的显示中满足了规定的条件的情况下，将第2操作图像显示在显示部，该第2操作图像是强调显示了与第1操作图像中的操作项目对应的文字信息的操作图像。在第2操作图像的显示中，声音操作处理部将与第2操作图像中强调显示的文字信息对应的声音的操作输入作为对视像装置的操作来接收。

Description

电子设备及非易失性存储介质
本申请要求在2019年10月29日提交日本专利局、申请号为2019-196719、发明名称为“电子设备及程序”的日本专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明的实施方式涉及电子设备及非易失性存储介质。
背景技术
以往，已知通过声音输入来进行游戏的操作的技术。
在先技术文献
专利文献
专利文献1：日本特开2009－125077号公报
发明内容
上述这样的声音输入不仅可以利用于游戏的操作，例如还可以利用于通过对智能手机等电子设备进行声音的操作输入，从而远程地操作电视机、录像机等视像装置的技术。在此情况下，希望降低对电子设备的声音的操作输入的负荷(难易程度)并提高便利性。
实施方式的电子设备具备显示处理部、以及声音操作处理部。显示处理部将包含操作输入用的操作项目的显示在内的第1操作图像显示在显示部，并且在第1操作图像的显示中满足了规定的条件的情况下，将第2操作图像显示在显示部，该第2操作图像是强调显示了与第1操作图像中的操作项目对应的文字信息的操作图像。声音操作处理部在第2操作图像的显示中将与第2操作图像中强调显示的文字信息对应的声音的操作输入作为对视像装置的操作来接收。
附图说明
图1A是示出了应用实施方式的技术的状况的一个例子的例示性且示意性的图；
图1B是示出了应用实施方式的技术的状况的另一个例子的例示性且示意性的图；
图2是示出了实施方式的信息终端的硬件构成的例示性且示意性的框图；
图3是示出了实施方式的信息终端及视像装置的功能的例示性且示意性的框图；
图4是将实施方式的信息终端及视像装置执行的处理以流程图方式示出的例示性且示意性的图；
图5是示出了实施方式的触摸操作图像的一个例子的例示性且示意性的图；
图6是示出了实施方式的触摸操作图像的另一个例子的例示性且示意性的图；
图7是示出了实施方式的声音操作图像的一个例子的例示性且示意性的图；
图8是用于说明实施方式的经由声音操作图像进行的检索的例示性且示意性的图；
图9是示出了实施方式的声音操作图像的另一个例子的例示性且示意性的图。
附图标记说明
100···信息终端(电子设备)，104···显示部，120···视像装置，121···录像机(视像装置)，122···电视机(视像装置)，312···显示处理部，314···声音操作处理部。
具体实施方式
以下，基于附图说明本申请的实施方式及变形例。以下记载的实施方式及变形例的构成、以及由该构成产生的作用及效果只不过是一个例子，并不限于以下的记载内容。
＜实施方式＞
图1A是示出了应用实施方式的技术的状况的一个例子的例示性且示意性的图。
如图1A所示，实施方式的技术作为一个例子被应用于利用智能手机那样的信息终端100来远程地操作录像机121及电视机122那样的视像装置120的状况。信息终端100是“电子设备”的一个例子。
在图1A所示的例子中，信息终端100和视像装置120经由路由器那样的中继装置110而能够相互通信地连接，但是，在实施方式中，信息终端100和视像装置120也可以经由互联网那样的网络而连接。
另外，实施方式的技术还能够应用于下面的图1B所示那样的状况。
图1B是示出了应用实施方式的技术的状况的另一个例子的例示性且示意性的图。在图1B所示的例子中，信息终端100和视像装置120不经由中继装置110而直接(端对端(peer-to-peer))连接。实施方式的技术也能够基于这样的连接关系而应用于利用信息终端100远程地操作视像装置120的状况。
图1A及图1B所示的信息终端100具有下面的图2所示那样的硬件构成。
图2是示出了实施方式的信息终端的硬件构成的例示性且示意性的框图。
如图2所示，信息终端100具备通信接口(I/F)101、声音输入接口102、传感器组103、显示部104、图形控制器105、触摸面板控制器106、CPU(Central Processing Unit，中央处理器)107、以及存储器108。信息终端100除了这些硬件之外，还可以具备照相机及扬声器等。
通信接口101是在经由中继装置110等进行的与视像装置120那样的其他设备之间的通信中使用的设备。
声音输入接口102是麦克风等声音输入设备，检测由信息终端100的用 户进行的声音输入。
传感器组103例如包含加速度传感器、方位传感器、及陀螺仪传感器等中的至少任意一种传感设备。加速度传感器是构成为在信息终端100被移动了的情况下对信息终端100的加速度的方向及大小进行检测的传感设备。另外，方位传感器是构成为对信息终端100的方位进行检测的传感设备。另外，陀螺仪传感器是构成为在信息终端100旋转了的情况下对信息终端100的角速度进行检测的传感设备。
显示部104被构成为组合有显示器104a和触摸面板104b的所谓触摸屏设备。显示器104a例如构成为LCD(Liquid Crystal Display，液晶显示器)或有机EL(Electro Luminescence，电致发光)显示器等。触摸面板104b检测由用户的手指或触摸笔等触碰的显示器104a的显示画面上的触碰位置。
图形控制器105控制显示部104的显示器104a。触摸面板控制器106控制显示部104的触摸面板104b，从触摸面板104b获取表示用户触碰的显示画面上的触碰位置的坐标数据等。
CPU107通过执行各种计算机程序，从而控制信息终端100的各组件。存储器108具备ROM(Read Only Memory，只读存储器)、RAM(Random Access Memory，随机存取处理器)等，存储由CPU107执行的各种运算处理所使用的各种计算机程序及各种数据等。
利用以上那样的硬件构成，实施方式实现利用信息终端100远程操作视像装置120。更具体而言，在实施方式中，信息终端100根据经由触摸面板104b检测的用户的手动的操作输入、或经由声音输入接口102检测的用户的声音的操作输入，从通信接口101发送对视像装置120的指示，视像装置120根据来自信息终端100的指示而动作。
在此，在为了操作视像装置120而对信息终端100进行声音的操作输入的情况下，希望降低对信息终端100的声音的操作输入的负荷(难易程度)，并提高便利性。
因此，实施方式通过使信息终端100(及视像装置120)具有下面的图3 所示那样的功能，从而降低用于使用信息终端100远程地操作视像装置120的声音的操作输入的负荷(难易程度)，并提高便利性。
图3是示出了实施方式的信息终端100及视像装置120的功能的例示性且示意性的框图。
信息终端100具备通信处理部311、显示处理部312、手动操作处理部313、以及声音操作处理部314。
通信处理部311使用通信接口101执行与视像装置120的通信的控制。显示处理部312使用图形控制器105执行显示器104a的控制。
手动操作处理部313执行与经由触摸面板104b进行的用户的手动的操作输入相关的控制。声音操作处理部314执行与经由声音输入接口102进行的用户的声音的操作输入相关的控制。
另外，视像装置120具备通信处理部321、内容管理部322、第1操作图像生成部323、第2操作图像生成部324、以及指示执行部325。
通信处理部321执行与信息终端100的通信的控制。另外，内容管理部322执行在视像装置120中记录的内容(例如录像视像等)的管理。
第1操作图像生成部323生成为了接收用户的手动的操作输入而应该使信息终端100显示的作为第1操作图像的触摸操作图像。第2操作图像生成部324生成为了接收用户的声音的操作输入而应该使信息终端100显示的作为第2操作图像的声音操作图像。此外，对于第1操作图像及第2操作图像的具体例在后描述。
指示执行部325控制视像装置120的各组件(各功能)，以使得根据经由触摸操作图像或声音操作图像输入到信息终端100的用户的操作输入来执行与从信息终端100发送的操作指示相应的动作。
基于以上的功能，实施方式的信息终端100及视像装置120按照下面的图4所示的流程来执行处理。
图4是将实施方式的信息终端100及视像装置120执行的处理以流程图方式示出的例示性且示意性的图。
如图4所示，在实施方式中，首先，在S411中，信息终端100的通信处理部311在实现对视像装置120的远程的操作时，对视像装置120请求触摸操作图像(触摸操作图像的显示所需要的信息)的发送。该S411的处理例如在如下情况下被执行：信息终端100的用户在信息终端100侧执行用于实现对视像装置120的远程的操作的应用程序，在由该应用程序向显示器104a提供的例如被称为主菜单等的初始画面(省略图示)中进行了规定的操作的情况。
若视像装置120的通信处理部321接收了来自信息终端100的请求，则在S421中，视像装置120的第1操作图像生成部323生成触摸操作图像(触摸操作图像的显示所需要的信息)，视像装置120的通信处理部321将生成的触摸操作图像发送到信息终端100。然后，若信息终端100的通信处理部311从视像装置120接收了触摸操作图像，则在S412中，信息终端100的显示处理部312将接收到的触摸操作图像显示在显示器104a上。
在此，对于触摸操作图像的具体例子与附图一起进行说明。
图5是示出了实施方式的触摸操作图像的一个例子的例示性且示意性的图。图5所示的图像500在信息终端100的用户借助由上述的应用程序提供的初始画面进行了规定的操作的情况下显示在显示器104a上。
如图5所示，在作为触摸操作图像的一个例子的图像500中显示了包含表示操作项目的按钮、图标或区域等的项目501～512，其中，该操作项目接收与对视像装置120的各种操作对应的手动的操作输入(触碰或点击输入)。
项目501是接收与从触摸操作图像向声音操作图像的切换操作对应的手动的操作输入的项目，项目502是接收与触摸操作图像的更新操作对应的手动的操作输入的项目。由应用程序来提供这些项目501及502，该应用程序是为了实现对视像装置120的远程的操作而在信息终端100中执行的应用程序。
另外，项目503～511是接收与对视像装置120的各种操作对应的手动的操作输入的项目。根据与由信息终端100进行的触摸操作图像的发送的请求，基于从视像装置120发送的信息来提供这些项目503～511。
更具体而言，项目503是接收在视像装置120中记录的内容的检索操作的项目。另外，项目504是接收使由视像装置120当前正在接收的广播中节目(节目的一览)显示在信息终端100或视像装置120上的操作的项目。另外，项目505是接收使在视像装置120中记录的录像节目(节目的一览)显示在信息终端100或视像装置120上的操作的项目。
另外，项目506是接收使在视像装置120中记录的录像节目(节目的一览)以节目表形式显示在信息终端100或视像装置120上的操作的项目。另外，项目507是接收使在视像装置120中记录的录像节目以更高的播放速度显示在信息终端100或视像装置120上的操作的项目。另外，项目508是接收使模仿了用于操作视像装置120的遥控器的UI的图像显示在信息终端100上的操作的项目。
进一步，项目509是接收使节目的排位(ranking)显示在信息终端100或视像装置120上的操作的项目，其中，该节目是由从设置在多个场所的视像装置120收集信息的运营商的服务器等提供的、进行了录像预约或收看预约的节目。另外，项目510是接收从服务器等下载节目表并使其显示在信息终端100上的操作的项目。另外，项目511是接收使成为录像预约的对象的节目的一览显示在信息终端100或视像装置120上的操作的项目。
此外，项目512是接收使应用程序的初始画面显示在信息终端100上的操作的项目，其中，该应用程序是为了实现对视像装置120的远程的操作而在信息终端100中执行的应用程序。该项目512也与上述的项目501及502同样由该应用程序提供。
作为与图5所示的图像500不同的触摸操作图像的另一个例子，举出下面的图6所示的图像600。
图6是示出了实施方式的触摸操作图像的另一个例子的例示性且示意性的图。在图5所示的图像500中进行了对项目504的手动的操作输入的情况下，以从该图像500切换的形式在信息终端100的显示器104a上显示图6所示的图像600。因此，在图6所示的图像600中，显示地面波数字广播、BS (Broadcasting Satellites，广播卫星)广播、及CS(Communication Satellites，通信卫星)广播中的至少任意一种广播中节目的一览。
如图6所示，在作为触摸操作图像的另一个例子的图像600中，也与上述的图5所示的图像500同样，显示项目601～610，该项目601～610包含表示接收与对视像装置120的各种操作对应的手动的操作输入(触碰或点击输入)的操作项目的按钮、图标或区域等。
项目601、602及610分别与上述的项目501、502及512相同，因此，在此省略说明。
项目603是接收调用与上述的图像500同样的操作项目的一览的操作的项目。另外，项目604是接收对在图像600中显示地面波数字广播的广播中节目的一览、还是在图像600中显示BS广播的广播中节目的一览、还是在图像600中显示CS广播的广播中节目进行选择的操作的项目。
另外，项目605～609是接收从与项目604中的选择结果相应的广播中节目的一览中选择使视像装置120显示播放的广播中节目的操作的项目。因此，信息终端100的用户能够通过进行选择项目605～609中的任一个的手动的操作输入(触碰或点击输入)，从而使视像装置120显示播放被选择的广播中节目。
这样，在实施方式的触摸操作图像中显示多个操作项目，该多个操作项目构成为接收与对视像装置120的操作对应的手动的操作输入。
因此，返回到图4，若进行了对触摸操作图像中的操作项目的手动的操作输入，则在S413中，信息终端100的通信处理部311将用于使视像装置120执行与该操作输入对应的操作的操作指示发送到视像装置120。
然后，若视像装置120的通信处理部321接收了来自信息终端100的操作指示，则在S422中，视像装置120的指示执行部325控制视像装置120的各组件(各功能)，以使得其执行与该操作指示相应的动作。
需要说明的是，实施方式的目的之一是，如上所述降低用于使用信息终端100远程地操作视像装置120的声音的操作输入的负荷(难易程度)，提 高便利性。
因此，在作为第1操作图像的触摸操作图像的显示中满足了规定的条件的情况下，实施方式的显示处理部312将能够达成上述的目的的作为第2操作图像的声音操作图像显示在信息终端100的显示器104a上。
更具体而言，返回到图4，在触摸操作图像的显示中满足了规定的条件的情况下，在S414中，信息终端100的通信处理部311对视像装置120请求声音操作图像(图像的显示所需要的信息)的发送。然后，在S423中，视像装置120的第2操作图像生成部324根据来自信息终端100的请求的接收，生成声音操作图像(图像的显示所需要的信息)，视像装置120的通信处理部321将生成的声音操作图像发送到信息终端100。然后，在S415中，信息终端100的显示处理部312根据来自视像装置120的声音操作图像的接收，将声音操作图像显示在显示器104a上。
在此，对于声音操作图像的具体例子与附图一起进行说明。
图7是示出了实施方式的声音操作图像的一个例子的例示性且示意性的图。图7所示的图像700构成为具有一定的透过性，在图5所示的图像500中进行了对项目501的手动的操作输入的情况下，以与该图像500重叠的形式使其显示在信息终端100的显示器104a上。
此外，在此，说明具有一定的透过性的图像700以与图像500重叠的形式显示在显示器104a上的例子，但是在实施方式中，也可以将不具有透过性的图像700以从图像500切换的形式显示在显示器104a上。另外，在实施方式中，图像700的重叠不限于进行了对项目501的手动的操作输入的情况，在将信息终端100向规定的方向或位置移动了的情况、或将规定的声音输入到信息终端100的情况下等，满足预先设定的规定的条件的情况下也可执行。
如图7所示，在作为声音操作图像的一个例子的图像700中显示项目703～712，该项目703～712构成为：将与图5所示的图像500中的作为操作项目的项目503～512对应的文字信息(信息中的至少一部分)强调显示在与项目503～512重叠的位置。
更具体而言，项目703是构成为将项目503中的“跨库检索”(Cross-Database search)中的一部分的“检索”这种文字信息(字符串)强调显示的项目。另外，项目704是构成为将项目504中的“观看广播中节目”中的一部分的“广播中节目”这种文字信息强调显示的项目。另外，项目705是构成为将项目505中的“观看录像节目”中的一部分的“录像节目”这种文字信息强调显示的项目。
另外，项目706是构成为将项目506中的“观看过去节目表”中的一部分的“过去节目表”这种文字信息强调显示的项目。另外，项目707是构成为将项目507中的“短时间观看”中的一部分的“短时间”这种文字信息强调显示的项目。另外，项目708是构成为将项目508中的“遥控”的全部的文字信息强调显示的项目。
另外，项目709是构成为将项目509中的“从排位进行预约”中的一部分的“排位”这种文字信息强调显示的项目。另外，项目710是构成为将项目510中的“从节目表中进行预约”中的一部分的“节目表”这种文字信息强调显示的项目。另外，项目711是构成为将项目511中的“预约一览”的全部的文字信息强调显示的项目。另外，项目712是构成为将项目512中的“去往主菜单”中的一部分的“主菜单”这种文字信息强调显示的项目。
在上述的项目703～712中强调显示的文字信息作为声音的操作输入的引导(关键词)而发挥功能。因此，用户通过一边观看图像700一边将在项目703～712中强调显示的文字信息作为声音输入到信息终端100，从而能够得到与对项目503～512进行了手动的操作输入的情况同等的结果。
例如，若将项目703中的“检索”这种文字信息作为声音而输入到信息终端100，则该项目703的显示按照下面的图8所示的形式变化。
图8是用于说明实施方式的借助声音操作图像进行的检索的例示性且示意性的图。
如图8所示，在图7所示的图像700中将“检索”这种文字信息作为声音输入到信息终端100，检索框800为空，在该检索框800的附近，显示包含 催促检索词的输入的“输入”这种文字信息在内的气泡状的项目801。在这样的构成中，用户通过在说出了项目801中的“输入”这种文字信息之后，说出检索词，从而能够在检索框800中输入检索词。此外，在检索词的输入后，包含用于取消该检索词的输入的“取消”这种文字信息在内的气泡状的项目802代替上述的项目801，显示在检索框800的附近。
返回到图7，在图像700中显示了未在图像500中显示的项目713。项目713是接收使图像700滚动的操作的项目。与上述的项目703～712同样，在项目713中也强调显示了作为声音的操作输入的引导的“下一个”这种文字信息。因此，用户通过将在项目713中强调显示的“下一个”这种文字信息作为声音而输入到信息终端100中，从而能够执行图像700的滚动。
这样，实施方式通过利用项目703～712强调显示成为基于声音的操作输入的关键词的文字信息，从而降低声音的操作输入的负荷(难易程度)，提高便利性。另外，实施方式通过使项目703～712与项目503～512的位置相互重叠，从而进一步提高便利性。
此外，在实施方式中，预先确定了成为声音的操作输入的关键词的文字信息。因此，在实施方式中，仅在将强调显示的文字信息作为声音进行了输入的情况下，执行对应的操作，在将强调显示的文字信息以外的文字信息作为声音进行了输入的情况下，不执行任何的操作。
然而，允许地域各自的方言等少许的误差是有益的。因此，在实施方式中，可以采用如下构成：准备多种模式的、作为声音的操作输入的关键词而预先确定的文字信息，并根据将该多种模式中的某一种文字信息作为声音进行输入的情况，而执行对应的操作。例如，对项目507中的“短时间”，除了和作为正规的读法的“Jidan(日文读音)”这种关键词建立对应之外，还和“Yidan(日文读音)”那样的关键词等也建立对应。
此外，声音操作图像并不仅限于上述的图像700。在实施方式中，作为与图6所示的图像600对应的声音操作图像，也存在下面的图9所示的图像900。
图9是示出了实施方式的声音操作图像的另一个例子的例示性且示意性 的图。在图7所示的图像700中进行了与项目704的“广播中节目”这种文字信息对应的声音的操作输入的情况下、或在图6所示的图像600中进行了对项目601的手动的操作输入的情况下，将图9所示的图像900显示在信息终端100的显示器104a上。
图9所示的图像900与图7所示的图像700同样，具有一定的透过性，以与图6所示的图像600重叠的形式显示在信息终端100的显示器104a上。因此，与图6所示的图像600同样，在图9所示的图像900中也以可视的形式显示地面波数字广播、BS广播、及CS广播中的至少任意一个广播中节目的一览。
如图9所示，在作为声音操作图像的另一个例子的图像900中，显示了与图6所示的图像600中的作为操作项目的项目603～610对应的项目903～910、以及与图6所示的图像600的操作项目不对应的项目911及912。
更具体而言，项目903是构成为将用于实现与上述的项目603(参照图6)同样的操作的作为声音输入的引导的“菜单”这种文字信息在项目603的附近的位置用气泡框包围从而强调显示的项目。因此，用户通过用声音输入“菜单”这种文字信息，从而能够得到与对上述的项目603(参照图6)进行了手动的操作输入的情况同等的结果。
另外，项目904是构成为将用于实现与上述的项目604(参照图6)同样的操作的作为声音输入的引导的“地面数字”、“BS”及“CS”这种文字信息在与原来的项目604重叠的位置强调显示的项目。因此，用户通过将“地面数字”、“BS”、及“CS”中的任一种文字信息用声音进行输入，从而能够选择是在图像900中显示地面波数字广播的广播中节目的一览、还是在图像900中显示BS广播的广播中节目的一览、还是在图像900中显示CS广播的广播中节目。
另外，项目905～909是构成为通过将用于实现与上述的项目605～609(参照图6)同样的操作的作为声音输入的引导的文字信息用气泡框包围从而强调显示的项目。在实施方式中，在项目905～909中强调显示的文字信息构 成为与对应的节目标题不关联的、由数字及文字的单独或组合等构成的简单的(短的)文字信息。
通常，节目标题会各种各样地变化，因此，表示节目标题的文字信息作为声音输入的引导不合适的情况较多。对此，实施方式通过如上所述将简单的(短的)文字信息作为声音输入的引导进行强调显示，从而提高了便利性。
此外，项目910是与上述的项目710(参照图7)同样的项目。因此，若用声音输入了在项目910中强调显示的“主菜单”这种文字信息，则在信息终端100上显示在信息终端100中执行的应用程序的初始画面。
另外，项目911及912是接收使图像900滚动的操作的项目。例如，若用声音输入了在项目911中强调显示的“前一个”这种文字信息，则图像900向上方滚动，若用声音输入了在项目912中强调显示的“下一个”这种文字信息，则图像900向下方滚动。
这样，在实施方式中，通过对声音操作图像的声音的操作输入，也能够执行与对触摸操作图像的手动的操作输入同等的操作。
因此，返回到图4，若进行了与在声音操作图像中强调显示的文字信息对应的声音的操作输入，则在S415中，信息终端100的通信处理部311向视像装置120发送用于使视像装置120执行与该操作输入对应的操作的操作指示。此外，在实施方式中，在声音操作图像的显示中一定时间地未进行声音的操作输入的情况下，可以执行从声音操作图像向触摸操作图像的切换。
而且，若视像装置120的通信处理部321接收了来自信息终端100的操作指示，则在S424中，视像装置120的指示执行部325控制视像装置120的各组件(各功能)，以使得执行与该操作指示相应的动作。
此外，在实施方式中，上述的图3所示的功能模块组作为由CPU107执行了存储器108等中存储的预定的应用程序的结果，通过硬件与软件的协作来实现。但是，在实施方式中，上述的图3所示的功能模块组的至少一部分也可以作为专用的硬件(电路)来实现。
上述的应用程序既可以以预先装入在存储器108等存储装置中的状态来 提供，也可以作为以在存储介质中能够安装的形式或能够执行的形式记录的计算机程序产品来提供，其中，该存储介质是软盘那样的各种磁盘、或DVD(Digital Versatile Disk：数字通用盘)那样的各种光盘等这样的计算机可读取的非易失性存储介质。
另外，上述的应用程序也可以经由互联网等网络来提供或发布。即，上述的应用程序也可以以在存储在连接于互联网等网络的计算机上的状态下，从该计算机经由网络下载这样的形式来提供。
如以上说明的那样，实施方式的信息终端100具备显示处理部312、以及声音操作处理部314。
显示处理部312构成为：将显示了接收与对视像装置120的操作对应的手动的操作输入的操作项目的作为第1操作图像的触摸操作图像(参照图5及图6)显示在显示部104(显示器104a)上，在触摸操作图像的显示中满足了规定的条件的情况下，将强调显示了与触摸操作图像中的操作项目对应的文字信息的作为第2操作图像的声音操作图像(参照图7及图9)显示在显示部104上。而且，声音操作处理部314构成为：在声音操作图像的显示中，将与在声音操作图像中强调显示的文字信息对应的声音的操作输入，作为对视像装置120的操作进行接收。
根据上述这样的构成，在为了操作视像装置120而对信息终端100进行声音的操作输入的情况下，能够将在声音操作图像中强调显示的文字信息作为声音的操作输入的引导来利用。因此，能够降低对信息终端100的声音的操作输入的负荷(难易程度)，提高便利性。
在此，在实施方式中，声音操作处理部314构成为：(不会将与在声音操作图像中强调显示的文字信息对应的声音不同于声音的操作输入作为对视像装置120的操作进行接收，而)仅将与在声音操作图像中强调显示的文字信息对应的声音的操作输入作为对视像装置120的操作进行接收。根据这样的构成，仅在精确地执行了声音的操作输入的情况下，能够实现对视像装置120的操作。
另外，在实施方式中，声音操作处理部314将声音的操作输入和与第2操作图像中强调显示的文字信息对应的声音的模式建立关联，并将声音的模式和对视像装置120的操作建立对应。根据这样的构成，能够容易将声音的操作输入和对视像装置120的操作建立对应。因此，例如在因方言等导致按照地域而文字信息的发出声音的模式不同这样的情况下，能够将这些不同的模式的发出的声音与同一操作建立对应。例如，“hi ga si da gou ji o ken sa ku(日文发音)”这种第1声音模式和“si ga si da gou ji o ken sa ku(日文发音)”这种第2声音模式和“su ga su da gou ji o ken sa ku(日文发音)”这种第3声音模式作为声音的模式各不相同，但是，认为该差异只不过是方言所导致的误差。因此，这3个声音模式均与“检索东田光司”这种用于检索东田光司(hi ga si da gou ji(日文发音))这一特定的个人的声音的操作输入建立对应是有益的。
换言之，在实施方式中，声音操作处理部314可以构成为：预先将多个模式的声音与在声音操作图像中强调显示的文字信息对应的声音建立对应，将该多个模式的声音中的某一个的操作输入，作为与在声音操作图像中强调显示的文字信息对应的声音的操作输入所对应的、对视像装置120的操作来接收。根据这样的构成，例如在因方言等导致按照地域而文字信息的发出声音的模式存在多个模式的情况下，能够适当地处理。
此外，在实施方式中，显示处理部312构成为：能够在声音操作图像上，将与触摸操作图像中的操作项目对应的文字信息在与触摸操作图像中的操作项目重叠的位置强调显示。根据这样的构成，能够容易识别经由声音操作图像进行的声音的操作输入和经由触摸操作图像进行的手动的操作输入的对应关系。
另外，在实施方式中，显示处理部312构成为：还能够在声音操作图像上，将与触摸操作图像中的操作项目对应的文字信息以由气泡框包围的状态在触摸操作图像中的操作项目的附近的位置强调显示。根据这样的构成，也能够容易识别经由声音操作图像进行的声音的操作输入和经由触摸操作图像 进行的手动的操作输入的对应关系。
另外，在实施方式中，显示处理部312构成为：在触摸操作图像中显示的操作项目与对应该使视像装置120输出的内容(上述的广播中节目及录像节目等)的操作对应的情况下，将声音操作图像中的文字信息作为由预先确定的文字及数字中的至少一方的单独或组合构成的识别信息来显示。根据这样的构成，在声音的操作输入时，通过使用识别信息，从而例如能够简单地执行对显示播放及停止等的内容的操作。
另外，在实施方式中，显示处理部312以触摸操作图像中的操作项目隔着声音操作图像而透过这样的显示形态，将声音操作图像重叠于触摸操作图像地显示。根据这样的构成，能够容易地识别在触摸操作图像中显示的操作项目和在声音操作图像中强调显示的文字信息的对应关系。
＜变形例＞
此外，在上述的实施方式中，例示了将具有透过性的声音操作图像以重叠于触摸操作图像的形式来显示的构成。然而，只要是成为声音的操作输入的引导的文字信息被强调显示的构成，则也可以将不具有透过性的声音操作图像以从触摸操作图像切换的形式来显示。
另外，在上述的实施方式中，例示了将声音操作图像中的文字信息在与触摸操作图像中的操作项目重叠的位置或附近的位置强调显示的构成。然而，即使文字信息和操作项目存在于分离的位置，只要成为声音的操作输入的引导的文字信息被强调显示，则能够得到降低对信息终端100的声音的操作输入的负荷(难易程度)并提高便利性这种效果。
另外，在上述的实施方式中，例示了将与对内容的操作对应的文字信息作为由预先确定的文字及数字中的至少一方的单独或组合构成的识别信息来显示的构成。然而，只要能够获取内容的标题等，则也可以将内容的标题整体作为成为声音的操作输入的引导的文字信息而在声音操作图像中强调显示。
以上，说明了本申请的实施方式及变形例，但是，上述的实施方式及变形例只不过是一个例子，并不意图限定申请的范围。上述的新的实施方式及 变形例能够以各种各样的形态来实施，在不脱离申请的主旨的范围内，能够进行各种省略、置换、变更。上述的实施方式及变形例包含在申请的范围、主旨中，并且包含在权利要求书的范围中记载的发明及其等同的范围内。

Claims (8)

  1. 一种电子设备，具备：
    显示处理部，其将包含操作输入用的操作项目的显示在内的第1操作图像显示在显示部，在所述第1操作图像的显示中满足了规定的条件的情况下，将第2操作图像显示在所述显示部，其中，所述第2操作图像是强调显示了与所述第1操作图像中的所述操作项目对应的文字信息的操作图像；以及
    声音操作处理部，其在所述第2操作图像的显示中，将与所述第2操作图像中强调显示的所述文字信息对应的声音的操作输入作为对视像装置的操作来接收。
  2. 根据权利要求1所述的电子设备，其中，
    所述声音操作处理部仅将与所述第2操作图像中强调显示的所述文字信息对应的声音的操作输入作为对所述视像装置的操作来接收。
  3. 根据权利要求2所述的电子设备，其中，
    所述声音操作处理部将所述声音的操作输入和对应于所述第2操作图像中强调显示的所述文字信息的声音的模式建立关联，并将所述声音的模式和对所述视像装置的操作建立对应。
  4. 根据权利要求1～3中任一项所述的电子设备，其中，
    所述显示处理部在所述第2操作图像上，将与所述第1操作图像中的所述操作项目对应的所述文字信息强调显示在与所述第1操作图像中的所述操作项目重叠的位置。
  5. 根据权利要求1～3中任一项所述的电子设备，其中，
    所述显示处理部在所述第2操作图像上，将与所述第1操作图像中的所述操作项目对应的所述文字信息以由气泡框包围的状态强调显示。
  6. 根据权利要求1～5中任一项所述的电子设备，其中，
    在所述第1操作图像中显示的所述操作项目与针对应该向所述视像装置输出的内容的操作相对应的情况下，所述显示处理部将所述第2操作图像中 的所述文字信息作为由预先确定的文字及数字中的至少一方的单独或组合所构成的识别信息来显示。
  7. 根据权利要求1～6中任一项所述的电子设备，其中，
    所述显示处理部以所述第1操作图像中的所述操作项目经由所述第2操作图像而透过的显示形态，将所述第2操作图像重叠于所述第1操作图像来进行显示。
  8. 一种计算机可读的非易失性存储介质，所述存储介质存储有程序，所述程序用于使计算机执行：
    将包含操作输入用的操作项目的显示在内的第1操作图像显示在显示部，在所述第1操作图像的显示中满足了规定的条件的情况下，将第2操作图像显示在所述显示部，其中，所述第2操作图像是强调显示了与所述第1操作图像中显示的所述操作项目对应的文字信息的操作图像；以及
    在所述第2操作图像的显示中，将与所述第2操作图像中强调显示的所述文字信息对应的声音的操作输入作为对视像装置的操作来接收。
PCT/CN2020/121423 2019-10-29 2020-10-16 电子设备及非易失性存储介质 WO2021082947A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080004679.5A CN112602330B (zh) 2019-10-29 2020-10-16 电子设备及非易失性存储介质
US17/644,536 US12039228B2 (en) 2019-10-29 2021-12-15 Electronic device and non-transitory storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019196719A JP2021071807A (ja) 2019-10-29 2019-10-29 電子機器およびプログラム
JP2019-196719 2019-10-29

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/644,536 Continuation US12039228B2 (en) 2019-10-29 2021-12-15 Electronic device and non-transitory storage medium

Publications (1)

Publication Number Publication Date
WO2021082947A1 true WO2021082947A1 (zh) 2021-05-06

Family

ID=75713109

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/121423 WO2021082947A1 (zh) 2019-10-29 2020-10-16 电子设备及非易失性存储介质

Country Status (2)

Country Link
JP (1) JP2021071807A (zh)
WO (1) WO2021082947A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009125077A (ja) * 2007-11-19 2009-06-11 Taito Corp 携帯端末、ゲームプログラム
CN103885662A (zh) * 2012-12-20 2014-06-25 联想(北京)有限公司 辅助语音输入的方法和装置
EP2986014A1 (en) * 2011-08-05 2016-02-17 Samsung Electronics Co., Ltd. Method for controlling electronic apparatus based on voice recognition and motion recognition, and electronic apparatus applying the same
CN106910503A (zh) * 2017-04-26 2017-06-30 海信集团有限公司 用于智能终端显示用户操控指令的方法、装置和智能终端

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005242183A (ja) * 2004-02-27 2005-09-08 Toshiba Corp 音声認識装置、表示制御装置、レコーダ装置、表示方法およびプログラム
JP2007171809A (ja) * 2005-12-26 2007-07-05 Canon Inc 情報処理装置及び情報処理方法
JP5280070B2 (ja) * 2008-03-05 2013-09-04 シャープ株式会社 ユーザインタフェース提供装置、デジタルテレビ、ユーザインタフェースシステム、ユーザインタフェース提供装置の制御方法、ユーザインタフェース提供プログラム、および該プログラムを記録したコンピュータ読み取り可能な記録媒体
JP2010130445A (ja) * 2008-11-28 2010-06-10 Sharp Corp 視聴システム、表示装置、リモートコントローラ装置、及び視聴方法
JP5629890B2 (ja) * 2011-03-30 2014-11-26 日立マクセル株式会社 ネットスーパーシステムにおける画面表示方法
US20140181672A1 (en) * 2012-12-20 2014-06-26 Lenovo (Beijing) Co., Ltd. Information processing method and electronic apparatus
JP2017204745A (ja) * 2016-05-11 2017-11-16 知紘 松野 プログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009125077A (ja) * 2007-11-19 2009-06-11 Taito Corp 携帯端末、ゲームプログラム
EP2986014A1 (en) * 2011-08-05 2016-02-17 Samsung Electronics Co., Ltd. Method for controlling electronic apparatus based on voice recognition and motion recognition, and electronic apparatus applying the same
CN103885662A (zh) * 2012-12-20 2014-06-25 联想(北京)有限公司 辅助语音输入的方法和装置
CN106910503A (zh) * 2017-04-26 2017-06-30 海信集团有限公司 用于智能终端显示用户操控指令的方法、装置和智能终端

Also Published As

Publication number Publication date
JP2021071807A (ja) 2021-05-06

Similar Documents

Publication Publication Date Title
JP6111030B2 (ja) 電子装置及びその制御方法
JP5819269B2 (ja) 電子装置及びその制御方法
KR102575230B1 (ko) 원격제어장치 및 그 동작방법
JP2014532933A (ja) 電子装置及びその制御方法
KR20130016024A (ko) 전자 장치 및 전자 장치의 ｕｉ 제공방법
US10250935B2 (en) Electronic apparatus controlled by a user's voice and control method thereof
EP3024220A2 (en) Display apparatus and display method
KR20170024372A (ko) 디스플레이 디바이스 및 그 제어 방법
KR20210061199A (ko) 디스플레이 기기, 모바일 기기, 디스플레이 기기의 화면 미러링 방법 및 모바일 기기의 화면 미러링 방법
JP6223744B2 (ja) 方法、電子機器およびプログラム
CN110659010A (zh) 一种画中画显示方法以及显示设备
JP2014109897A (ja) 情報処理装置およびコンテンツ検索方法
CN112885354B (zh) 一种显示设备、服务器及基于语音的显示控制方法
US20170019710A1 (en) Image display apparatus and method of operating the same
US20240184519A1 (en) Display control device for selecting item on basis of speech
EP2605527B1 (en) A method and system for mapping visual display screens to touch screens
US12039228B2 (en) Electronic device and non-transitory storage medium
WO2024037480A1 (zh) 交互方法、装置、电子设备和存储介质
US20150382070A1 (en) Method, electronic device, and computer program product
CN113473241A (zh) 一种显示设备、及图文样式菜单的显示控制方法
WO2021082947A1 (zh) 电子设备及非易失性存储介质
KR20090124240A (ko) 자막 편집 장치 및 그 방법
WO2023221791A1 (zh) 聊天频道显示方法、装置、设备、可读存储介质及产品
JP6792214B2 (ja) ディスプレイ装置用の通知プロファイルに基づくライブインタラクティブイベント表示
JP4735731B2 (ja) 画像処理システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20883176

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20883176

Country of ref document: EP

Kind code of ref document: A1