CN113573095B - Display device and method of operating the same - Google Patents

Abstract

A display device and a method of operating the same. A display apparatus capable of providing diversity of searches through a plurality of applications capable of providing search results of a search source is provided. An execution command of a context search application is received. One or more search source icons corresponding to one or more search source applications matching the calling application are displayed on the display according to the received execution command. When one of the one or more search source icons is selected, search results for the search source are obtained by a search source application corresponding to the selected icon. The retrieved search results are displayed on a display.

Description

Display device and method of operating the same

Technical Field

The present disclosure relates to a display device and a method of operating the same.

Background

Digital TV services using wired or wireless communication networks are becoming commonplace. The digital TV service may provide various services that cannot be provided in the conventional analog broadcasting service.

For example, IPTV (internet protocol television) and smart TV services, which are one type of digital TV service, provide bi-directionality that allows a user to actively select the type of viewing program and the viewing time. IPTV, smart TV services can provide various additional services such as internet search, home shopping, and online gaming based on the bi-directionality.

In addition, recent TVs may provide search results of text or images to users.

However, in the related art, there is inconvenience in the search process. For example, the user must directly input text or upload an image.

Disclosure of Invention

The present disclosure provides a display apparatus capable of providing search results without a user directly inputting text or images.

The present disclosure provides a display apparatus capable of providing diversity of searches through a plurality of applications capable of providing search results of a search source.

According to one embodiment of the present disclosure, a display device includes: a display configured to display an execution screen of the calling application; and a controller configured to receive an execution command of a contextual search application providing search results of the search source, display one or more search source icons corresponding to one or more search source applications matching the calling application on the display according to the received execution command, acquire search results of the search source through the search source application corresponding to the selected icon when one of the one or more search source icons is selected, and display the acquired search results on the display.

According to one embodiment of the present disclosure, a method of operating a display device includes: displaying an execution picture of the calling application; receiving an execution command of a contextual search application providing search results of a search source; displaying one or more search source icons corresponding to one or more search source applications matched with the calling application according to the received execution command; when one of the one or more search source icons is selected, obtaining search results of a search source through a search source application corresponding to the selected icon; and displaying the retrieved search results on a display

The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims.

Drawings

Fig. 1 is a block diagram showing a configuration of a display device according to an embodiment of the present disclosure.

Fig. 2 is a block diagram illustrating a remote control device according to an embodiment of the present disclosure.

Fig. 3 is a diagram illustrating an actual configuration of a remote control device according to an embodiment of the present disclosure.

Fig. 4 is a diagram utilizing a remote control device according to an embodiment of the present disclosure.

Fig. 5 is a flowchart for describing an operation method of a display device according to an embodiment of the present disclosure.

Fig. 6 is a diagram illustrating an example of displaying an indicator indicating an executable contextual search application when a pointer is located on a search source according to an embodiment of the present disclosure.

Fig. 7 is a diagram for describing a process of extracting a plurality of search source applications matching a calling application according to an embodiment of the present disclosure.

Fig. 8 is a diagram showing a configuration of a source application table.

Fig. 9 is a diagram illustrating an example of displaying a search source application list according to an embodiment of the present disclosure.

Fig. 10 is a diagram illustrating an example of outputting search results of a search source in a preview form according to an embodiment of the present disclosure.

Fig. 11 is a diagram illustrating an example of outputting search results of a search source on a full screen according to an embodiment of the present disclosure.

Fig. 12 and 13 are diagrams for describing a process of providing search results using an image as a search source according to an embodiment of the present disclosure.

Fig. 14 and 15 are diagrams for describing a process of providing search results using an object included in an image as a search source according to an embodiment of the present disclosure.

Fig. 16 is a diagram illustrating an example of controlling a home appliance according to an embodiment of the present disclosure.

Fig. 17 is a diagram for describing a method of generating successive additional search sources based on any one search source according to another embodiment of the present disclosure.

Fig. 18 and 19 are diagrams illustrating a process of generating a mind map according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, embodiments related to the present disclosure will be described in detail with reference to the accompanying drawings. The suffix "module" and "interface" of the components used in the following description are assigned or mixed in consideration of the ease of writing the specification, and do not have a unique meaning or effect per se.

For example, as an artificial display device adding a computer support function to a broadcast receiving function, when an internet function is added while the broadcast receiving function is implemented, the display device according to an embodiment of the present disclosure may have an easy-to-use interface such as a writing input device, a touch screen, or a spatial remote control device. The internet and computer may then be accessed to perform email, web browsing, banking or gaming functions with the support of wired or wireless internet functions. To perform such various functions, a standardized general-purpose OS may be used.

Accordingly, since various applications are freely added or deleted on the general-purpose OS kernel, the display device described herein can perform various user-friendly functions, for example. In more detail, the display device may be a web TV, a Hybrid Broadcast Broadband TV (HBBTV), a smart TV, a Light Emitting Diode (LED) TV, an Organic Light Emitting Diode (OLED) TV, or the like, and in some cases, may be applied to a smart phone.

Fig. 1 is a block diagram showing a configuration of a display device according to an embodiment of the present disclosure.

Referring to fig. 1, the display device 100 may include a broadcast receiver 130, an external device interface 135, a storage part 140, a user input interface 150, a controller 170, a wireless communication interface 173, a display 180, an audio output interface 185, and a power supply 190.

The broadcast receiver 130 may include a tuner 131, a demodulator 132, and a network interface 133.

The tuner 131 may select a specific broadcast channel according to a channel selection command. The tuner 131 may receive a broadcast signal of a selected specific broadcast channel.

The demodulator 132 may divide the received broadcast signal into a video signal, an audio signal, and a broadcast program related data signal, and restore the divided video signal, audio signal, and data signal to an output usable form.

The external device interface 135 may receive an application or a list of applications in an adjacent external device and transmit it to the controller 170 or the storage part 140.

The external device interface 135 may provide a connection path between the display device 100 and an external device. The external device interface 135 may receive at least one image or audio output from an external device connected to the display device 100 wirelessly or by wire and transmit it to the controller 170. The external device interface 135 may include a plurality of external input terminals. The plurality of external input terminals may include an RGB terminal, at least one High Definition Multimedia Interface (HDMI) terminal, and a component terminal.

The image signal of the external device input through the external device interface 135 may be output through the display 180. The voice signal of the external device input through the external device interface 135 may be output through the audio output interface 185.

The external devices connectable to the external device interface 135 may be a set-top box, a blu-ray player, a DVD player, a game player, a sound bar, a smart phone, a PC, a USB memory, and a home theater system, but this is merely exemplary.

The network interface 133 may provide an interface for connecting the display device 100 to a wired/wireless network including an internet network. The network interface 133 may send data to or receive data from another user or another electronic device through an access network or another network linked to the access network.

In addition, some content data stored in the display device 100 may be transmitted to a user or electronic device selected from other users or other electronic devices pre-registered in the display device 100.

The network interface 133 may access a predetermined web page through an access network or another network linked to the access network. In other words, the network interface 133 may transmit data to or receive data from a corresponding server by accessing a predetermined web page through a network.

The network interface 133 may then receive content or data provided from a content provider or network operator. In other words, the network interface 133 may receive contents such as movies, advertisements, games, VOD and broadcast signals and information related thereto provided from a content provider or a network provider through a network.

In addition, the network interface 133 may receive firmware update information and update files provided from a network operator and transmit data to the internet or a content provider or a network operator.

The network interface 133 may select and receive a desired application among applications open to an air interface (air) through a network.

The storage section 140 may store the image, voice, or data signals processed by the signals stored by the program for respective signal processing and control in the controller 170.

In addition, the storage section 140 may perform a function of temporarily storing an image, voice, or data signal output from the external device interface 135 or the network interface 133, and may store information on a predetermined image through a channel memory function.

The storage section 140 may store an application or an application list input from the external device interface 135 or the network interface 133.

The display device 100 may play and provide content files (e.g., video files, still image files, music files, document files, application files, etc.) stored in the storage section 140 to a user.

The user input interface 150 may transmit signals input by a user to the controller 170 or transmit signals from the controller 170 to the user. For example, the user input interface 150 may receive or process control signals such as power on/off, channel selection, and screen setting from the remote control device 200 or transmit control signals from the controller 170 to the remote control device 200 according to various communication methods such as bluetooth, ultra Wideband (WB), zigBee, video (RF), and IR.

In addition, the user input interface 150 may transmit control signals input from local keys (not shown) such as a power key, a channel key, a volume key, and a set key to the controller 170.

The image signal image-processed in the controller 170 may be input to the display 180 and displayed as an image corresponding to the corresponding image signal. In addition, the image signal image-processed in the controller 170 may be input to an external output device through the external device interface 135.

The voice signal processed in the controller 170 may be output to the audio output interface 185. In addition, the voice signal processed in the controller 170 may be input to an external output device through the external device interface 135.

In addition, the controller 170 may control overall operations in the display device 100. In addition, the controller 170 may download a desired application or application list into the display device 100 by controlling the display device 100 to access the network through a user command or an internal program input through the user input interface 150.

The controller 170 may output channel information selected by the user together with the processed image or voice signal through the display 180 or the audio output interface 185.

In addition, the controller 170 may output an image signal or a voice signal of an external device such as a camera or video camera, which is input through the external device interface 135, through the display 180 or the audio output interface 185 according to an external device image playback command received through the user input interface 150.

Further, the controller 170 may control the display 180 to display an image and control a broadcast image input through the tuner 131, an external input image input through the external device interface 135, an image input through the network interface, or an image stored in the storage part 140 to be displayed on the display 180. In this case, the image displayed on the display 180 may be a still image or a video, and may also be a 2D image or a 3D image.

In addition, the controller 170 may play the content stored in the display apparatus 100, the received broadcast content, and the externally input content input from the outside, and the content may be in various formats such as a broadcast image, an externally input image, an audio file, a still image, an accessed web screen, and a document file.

The wireless communication interface 173 may perform wired or wireless communication with an external electronic device. The wireless communication interface 173 may perform short-range communication with an external device. To this end, the wireless communication interface 173 may support short-range communication using at least one of bluetooth, radio Frequency Identification (RFID), infrared data association (IrDA), ultra Wideband (UWB), zigBee, near Field Communication (NFC), wireless fidelity (Wi-Fi), wi-Fi direct, and wireless Universal Serial Bus (USB) technologies. The wireless communication interface 173 may support wireless communication between the display device 100 and a wireless communication system, between the display device 100 and another display device 100, or between a network including the display device 100 and another display device 100 (or an external server) through a wireless area network. The wireless area network may be a wireless personal area network.

Herein, another display device 100 may be a mobile terminal capable of exchanging data (or interworking) with the display device 100, such as a wearable device (e.g., a smart watch, smart glasses, and a Head Mounted Display (HMD)) or a smart phone. The wireless communication interface 173 may detect (or identify) wearable devices that may communicate around the display device 100. Further, if the detected wearable device is a device authenticated to communicate with the display device 100, the controller 170 may send at least part of the data processed in the display device 100 to the wearable device through the wireless communication interface 173. Thus, a user of the wearable device may use the data processed in the display device 100 through the wearable device.

The display 180 may convert an image signal, a data signal, or an On Screen Display (OSD) signal processed in the controller 170 or an image signal or a data signal received in the external device interface 135 into R, G and B signals to generate a driving signal.

In addition, the display device 100 shown in fig. 1 is only one embodiment of the present disclosure, and thus, some of the components shown may be integrated, added, or omitted according to the specifications of the display device 100 actually implemented.

In other words, if desired, two or more components may be integrated into one component, or one component may be divided into two or more components and configured. In addition, the functions performed by the various blocks are for the purpose of describing embodiments of the present disclosure, and the specific operation or apparatus thereof is not intended to limit the scope of the present disclosure.

According to another embodiment of the present disclosure, unlike fig. 1, the display device 100 may receive images and play them through the network interface 133 or the external device interface 135, without including the tuner 131 and the demodulator 132.

For example, the display apparatus 100 may be divided into an image processing apparatus (e.g., a set top box) for receiving a broadcast signal or content according to various network services and a content playback apparatus for playing the content input from the image processing apparatus.

In this case, the operation method of the display device according to the embodiment of the present disclosure described below may be performed by one of the display device described with reference to fig. 1, an image processing device such as a separate set top box, and a content playback device including the display 180 and the audio output interface 185.

Next, referring to fig. 2 and 3, a remote control device is described according to an embodiment of the present disclosure.

Fig. 2 is a block diagram illustrating a remote control device according to an embodiment of the present disclosure, and fig. 3 is a diagram illustrating an actual configuration of a remote control device 200 according to an embodiment of the present disclosure.

First, referring to fig. 2, the remote control device 200 may include a fingerprint identifier 210, a wireless communication interface 220, a user input interface 230, a sensor 240, an output interface 250, a power supply 260, a storage 270, a controller 280, and a voice acquirer 290.

Referring to fig. 2, the wireless communication interface 220 transmits/receives signals to/from any one of the display devices according to the above-described embodiments of the present disclosure.

The remote control device 200 may include a Radio Frequency (RF) module 221 for transmitting/receiving signals to/from the display device 100 according to an RF communication standard and an IR module 223 for transmitting/receiving signals to/from the display device 100 according to an IR communication standard. In addition, the remote control device 200 may include a bluetooth module 225 for transmitting/receiving signals to/from the display device 100 according to a bluetooth communication standard. In addition, the remote control device 200 may include a Near Field Communication (NFC) module 227 for transmitting/receiving signals to/from the display device 100 according to an NFC communication standard, and a Wireless LAN (WLAN) module 229 for transmitting/receiving signals to/from the display device 100 according to a WLAN communication standard.

In addition, the remote control device 200 may transmit a signal containing information about the movement of the remote control device 200 to the display device 100 through the wireless communication interface 220.

In addition, the remote control device 200 may receive a signal transmitted from the display device 100 through the RF module 221, and if necessary, may transmit commands regarding power on/off, channel change, and volume change to the display device 100 through the IR module 223.

The user input interface 230 may be configured with keypad buttons, a touch pad, or a touch screen. The user may manipulate the user input interface 230 to input commands related to the display device 100 to the remote control device 200. If the user input interface 230 includes a hard key button, the user may input a command related to the display apparatus 100 to the remote control apparatus 200 through a push operation of the hard key button. This will be described with reference to fig. 3. .

Referring to fig. 3, the remote control device 200 may include a plurality of buttons. The plurality of buttons may include a fingerprint recognition button 212, a power button 231, a home button 232, a live button 233, an external input button 234, a voice adjustment button 235, a voice recognition button 236, a channel change button 237, a check button 238, and a return button 239.

The fingerprint recognition button 212 may be a button for recognizing a fingerprint of a user. According to an embodiment of the present disclosure, the fingerprint recognition button 212 may perform a push operation and receive the push operation and the fingerprint recognition operation. The power button 231 may be a button for turning on/off the power of the display apparatus 100. The home button 232 may be a button for moving to a home screen of the display apparatus 100. The live button 233 may be a button for displaying live broadcast programs. The external input button 234 may be a button for receiving an external input connected to the display device 100. The voice adjusting button 235 may be a button for adjusting the volume level output from the display device 100. The voice recognition button 236 may be a button for receiving a user's voice and recognizing the received voice. The channel change button 237 may be a button for receiving a broadcast signal of a specific broadcast channel. The check button 238 may be a button for selecting a specific function, and the return button 239 may be a button for returning to a previous screen.

Fig. 2 will be described again.

If the user input interface 230 includes a touch screen, the user may touch soft keys of the touch screen to input commands related to the display device 100 to the remote control device 200. In addition, the user input interface 230 may include various types of input means manipulated by a user, such as scroll keys and jog keys, which embodiments do not limit the scope of the present disclosure.

The sensor 240 may include a gyro sensor 241 or an acceleration sensor 243, and the gyro sensor 241 may sense information about movement of the remote control device 200.

For example, the gyro sensor 241 may sense information about the operation of the remote control device 200 based on x, y, and z axes, and the acceleration sensor 243 may sense information about the moving speed of the remote control device 200. In addition, the remote control device 200 may further include a distance measuring sensor and sense a distance with respect to the display 180 of the display device 100.

The output interface 250 may output an image or voice signal corresponding to manipulation of the user input interface 235 or corresponding to a signal transmitted from the display device 100. The user may recognize whether the user input interface 235 is manipulated or the display device 100 is controlled through the output interface 250.

For example, if the user input interface 235 is manipulated or signals are transmitted/received to/from the display apparatus 100 through the wireless communication interface 220, the output interface 250 may include an LED module 251 for flashing, a vibration module 253 for generating vibration, a sound output module 255 for outputting sound, or a display module 257 for outputting an image.

In addition, the power supply 260 supplies power to the remote control device 200, and if the remote control device 200 does not move for a predetermined time, the power supply is stopped, so that power waste can be reduced. If a predetermined key provided at the remote control 200 is manipulated, the power supply 260 may resume the power supply.

The storage 270 may store various types of programs and application data required for control or operation of the remote control device 200. If the remote control device 200 transmits/receives signals wirelessly through the display device 100 and the RF module 221, the remote control device 200 and the display device 100 transmit/receive signals through a predetermined frequency band.

The controller 280 of the remote control device 200 may store information about a frequency band for transmitting/receiving signals to/from the display device 100 paired with the remote control device 200 in the storage section 270 and refer to the information.

The controller 280 controls general matters related to the control of the remote control device 200. The controller 280 may transmit a signal corresponding to a predetermined key manipulation of the user input interface 235 or a signal corresponding to a movement of the remote control device 200 sensed by the sensor 240 to the display device 100 through the wireless communication interface 220.

In addition, the voice acquirer 290 of the remote control device 200 can acquire voice.

The voice acquirer 290 may include at least one microphone 291 and acquire voice through the microphone 291.

Next, fig. 4 is shown.

Fig. 4 is a diagram utilizing a remote control device according to an embodiment of the present disclosure.

Fig. 4 (a) shows that a pointer 205 corresponding to the remote control device 200 is displayed on the display 180.

The user may move or rotate the remote control device 200 vertically or horizontally. The pointer 205 displayed on the display 180 of the display device 100 corresponds to the movement of the remote control device 200. Since the corresponding pointer 205 is moved and displayed according to movement on the 3D space as shown in the drawing, the remote control device 200 may be referred to as a space remote control device.

Fig. 4 (b) shows that if the user moves the remote control device 200, the pointer 205 displayed on the display 180 of the display device 100 moves leftward according to the movement of the remote control device 200.

Information about the movement of the remote control device 200 sensed by the sensor of the remote control device 200 is transmitted to the display device 100. The display device 100 may calculate coordinates of the pointer 205 from information about the movement of the remote control device 200. The display device 100 may display the pointer 205 to match the calculated coordinates.

Fig. 4 (c) shows that the user moves the remote control device 200 away from the display 180 while a specific button in the remote control device 200 is pressed. Accordingly, the selection area of the display 180 corresponding to the pointer 205 may be zoomed in and displayed larger.

Conversely, when the user moves the remote control device 200 to access the display 180, the selection area of the display 180 corresponding to the pointer 205 may be zoomed out and zoomed out.

On the other hand, if the remote control 200 is moved away from the display 180, the selection area may be pulled away, and if the remote control 200 is moved closer to the display 180, the selection area may be pulled closer.

In addition, if a specific button in the remote control device 200 is pressed, recognition of vertical or horizontal movement may be excluded. In other words, if the remote control 200 moves away from or closer to the display 180, up, down, left, or right movement may not be recognized, and only forward and backward movement may be recognized. While a specific button in the remote control device 200 is not pressed, the pointer 205 moves only according to the up, down, left, or right movement of the remote control device 200.

Further, the moving speed or moving direction of the pointer 205 may correspond to the moving speed or moving direction of the remote control device 200.

Further, the pointer in this specification means an object displayed on the display 180 in response to an operation of the remote control device 200. Thus, various forms of objects are possible in addition to the arrow form shown as pointer 205 in the figures. For example, the concepts described above include points, cursors, hints, and bold outlines. The pointer 205 may then be displayed corresponding to one point of the horizontal and vertical axes on the display 180, and may also be displayed corresponding to a plurality of points such as lines and planes.

Next, an operation method of the display device 100 according to an embodiment of the present disclosure will be described with reference to fig. 5.

Fig. 5 is a flowchart for describing an operation method of a display device according to an embodiment of the present disclosure.

Hereinafter, "app" may be an abbreviation of "application".

Referring to fig. 5, the controller 170 of the display apparatus 100 displays an execution screen of the calling application on the display 180 (S501).

In an embodiment, the calling application is a basic application installed on the display apparatus 100, and may be an application that is called according to user input.

The calling application may be one of a live TV application providing real-time broadcasting, a channel banner application providing a channel banner, an Electronic Program Guide (EPG) application providing an EPG, a live menu application providing a live menu, a setup application setting the display device 100, a sports setup alarm application providing a sports alarm, and a gallery application providing images or videos.

The calling application may be displayed on an initial screen of the display device 100. The initial screen of the display apparatus 100 may be a screen displayed when the power of the display apparatus 100 is turned on.

The execution screen of the calling application may include one or more texts or one or more images. The search source may be one of one or more texts or one of one or more images.

The controller 170 determines whether the pointer 205 is located on a search source included in an execution screen of the calling application (S503).

In an embodiment, the search source may be text or an image. The search source may be a source selectable by user input. In detail, the search source may be a source that can be clicked on by the pointer 205.

When it is determined that the pointer 205 is located on the search source, the controller 170 displays an indicator indicating that the contextual search application for the search source is executable on the display 180 (S505).

The contextual search application may be an application that may provide search results of a search source.

In detail, the contextual search application provides search results of a search source, but may be an application associated with a calling application. This will be described in detail below.

The controller 170 may compare the coordinate information about the pointer 205 with the coordinate information about the search source and may display an indicator indicating that the contextual search application is executable when it is determined that the position of the pointer 205 is located on the search source.

The controller 170 may display an indicator on one side of the pointer 205 or search source.

The indicator may be an application icon corresponding to the contextual search application, but this is merely an example.

As another example, when it is determined that the pointer 205 is located on the search source, the controller 170 may display an indicator while changing one of the shape and size of the pointer 205.

As another example, when it is determined that the pointer 205 is located on the search source, the controller 170 may change at least one of the shape or size of the pointer 205 without displaying the indicator.

Operation S505 may be omitted.

Fig. 6 is a diagram illustrating an example of displaying an indicator indicating that a contextual search application is executable when a pointer is located on a search source according to an embodiment of the present disclosure.

Referring to fig. 6, an EPG screen 600 according to the execution of the EPG application is shown.

The EPG application may be a calling application described in operation S601.

The display apparatus 100 may display the EPG screen 600 according to a user input for executing the EPG application.

The EPG screen 600 may include a plurality of channel numbers, a plurality of channel names, and program names broadcasted in each channel with time.

The EPG screen 600 may include a plurality of search sources. Each search source is a source that can be selected or clicked on by pointer 205.

The search source may be text or an image.

Within EPG screen 600, a plurality of selectable search sources can be predetermined.

When the pointer 205 is located over the program name 601, the display device 100 may display an indicator 603 indicating that the contextual search application for the program name 601 is executable.

Indicator 603 may be displayed adjacent to program name 601 or may be displayed adjacent to pointer 205.

As another example, when the pointer 205 is located on the program name 601, the display apparatus 100 may change at least one of the shape or size of the pointer 205 instead of displaying the indicator 603.

Further, when the pointer 205 is located in the area including the program name 601, the display apparatus 100 may display the indicator 603.

The user can intuitively recognize the execution of the search source application through the indicator 603.

Fig. 5 is again described.

The controller 170 receives an execution command of the contextual search application (S507) and displays a search source application list including search source applications on the display 180 according to the received execution command (S509).

When a command for selecting a wheel key provided in the remote control device 200 for a specific time or more is received from the remote control device 200, the controller 170 may be determined to receive an execution command of the context search application.

When the pointer 205 is located on the search source and a command for selecting a specific button provided on the remote control device 200 is removed, the controller 170 may determine that an execution command of the contextual search application is received.

The search source application list may include one or more icons for identifying one or more search source applications.

The contextual search application may be an application for providing a plurality of search source applications that were previously matched to the calling application.

In detail, the contextual search application may be an application for providing search results of a search source through a plurality of search source applications previously matched with a calling application.

Each of the plurality of search source applications may be an application for providing search results of a search source.

Each of the plurality of search source applications may be an application that previously matched the executing calling application.

The controller 170 may acquire a plurality of search source applications matching the calling application using the source application table stored in the storage 140.

This will be described below with reference to fig. 7 and 8.

Fig. 7 is a diagram for describing a process of extracting a plurality of search source applications matching a calling application according to an embodiment of the present disclosure, and fig. 8 is a diagram showing a configuration of a source application table.

Referring to fig. 7, the controller 170 extracts one or more search source applications matching the calling application using the source application table stored in the storage part 140 according to an execution command of the context search application (S701).

The source application table may be a table indicating correspondence between a plurality of search source applications matching the calling application. The source application table may be received from an external server through the network interface 133.

FIG. 8 illustrates a source application table 800 for a plurality of search source applications that have previously been matched with a plurality of calling applications.

The source application table 800 may represent correspondence between a calling application and a plurality of search source applications that match the calling application.

Each of the plurality of calling applications 801 to 807 may be applications installed on the display apparatus 100.

The live TV application 801 may be an application for providing real-time broadcasting.

Channel banner application 802 may be an application for providing a channel banner.

The EPG application 803 may be an application for providing an EPG.

The live menu application 804 may be an application for setting a function when viewing live TV.

The setting application 805 may be an application for setting the functions of the display apparatus 100.

Sports alarm setting application 806 may be an application for setting notifications for sports events.

Gallery application 807 may be an application for providing images or video.

Referring to the source application table 800, one or more search source applications among the first through seventh search source applications 811 through 817 are matched in each calling application.

The respective search source applications may be deleted in the display apparatus 100. In addition, new search source applications may be added. In this case, the source application table 800 may reflect newly added or deleted applications.

The search source application is an application that can output search results of text or images, and may be an application that can output search results such as content providers, web browsers, user guides, and Artificial Intelligence (AI) keywords.

For example, the first through fifth search source applications 811 through 815 and the seventh search source application 817 may be matched in each of the live TV application 801 and the channel banner application 802.

The first through fifth search source applications 811 through 815 may be matched in each of the EPG application 803 and the live menu application 804.

The third search source application 813, the fourth search source application 814, and the sixth search source application 816 may be matched in the setup application 805.

The third search source application 813 through the fifth search source application 815 may be matched in the sports alarm setting application 806.

The third search source application 813 and the fourth search source application 814 may be matched in gallery application 807.

The first search source application 811 may be displayed only on programs in which program information exists. The second search source application 812 is only shown for genres such as dramas or movies. That is, the search source application may be matched with a particular calling application and displayed according to the attributes of the calling application.

The controller 170 may determine what the calling application executed by the display apparatus 100 is, and may extract one or more search source applications corresponding to the determined calling application using the source application table 800.

Further, in addition to the search source, the controller 170 may transmit additional information to the search source application to improve search quality.

For example, when the calling application is the setting application 805, the controller 170 may transmit search information together with intention information indicating intention of function settings of the TV to the third search source application 803.

The intent information may be obtained from attributes of the calling application. That is, since the setting application 805 is an application for setting the function of the display apparatus 100, an intention indicating the purpose of the setting application 805 may be transmitted to the search source application together with the search source.

That is, the controller 170 may obtain, through the search source application, search results of contextual search sources including intent obtained from the attributes of the calling application and the search sources.

Accordingly, compared with the case where a search source such as text is transmitted only through the third search source application 8030, the intention indicating the function of setting the TV is reflected, thereby further improving the search quality.

Further, applications other than the predefined search source application in the calling application may be displayed, or the search source application may be called in an application not designated as the calling application, according to the user's preference.

In this case, when an image of an icon, a color of the icon, or the icon corresponding to the search source application is selected, an identifier of the application to be executed and parameter information to be transmitted to the application to be executed may be required. The parameter information may include parameters required to execute the application.

In order to display applications other than the predefined search source application in the calling application or to call the search source application in an application not designated as the calling application, a separate user setup menu may be required, and the search source application matching the specific application may be set as user-defined through the user setup menu.

Fig. 7 is described again.

The controller 170 determines whether the extracted one or more search source applications are all installed on the display apparatus 100 (S703).

The controller 170 may determine whether to install or not install by comparing an identifier of an application installed on the display apparatus 100 with the extracted one or more identifiers of one or more search source applications.

When the extracted one or more search source applications are all installed on the display apparatus 100, the controller 170 displays one or more search source icons corresponding to each of the extracted one or more search source applications on the display 180 (S705).

That is, one or more search source icons corresponding to one or more search source applications may constitute a search source application list.

The controller 170 may arrange one or more search source icons such that the one or more search source icons form a circle centered on the search source.

The controller 170 displays a search source icon corresponding to a search source application installed on the display apparatus 100 among the extracted one or more search source applications on the display 180 (S707).

The controller 170 displays only search source icons corresponding to search source applications installed on the display apparatus 100 among the extracted one or more search source applications at one side of the search source.

Fig. 9 is a diagram illustrating an example of displaying a search source application list according to an embodiment of the present disclosure.

Fig. 9 is a diagram described in connection with the embodiment of fig. 6.

Referring to fig. 9, when an execution command for executing a contextual search application is received, the controller 170 may display a plurality of search source icons 901 to 904 indicating a plurality of search source applications on an execution screen 600 of the EPG application.

The plurality of search source icons 901 through 904 may be arranged in a circular shape around the pointer 205 or the search source 601.

The first search source icon 901 may be an icon indicating the first search source application 801, the second search source icon 902 may be an icon indicating the second search source application 802, the third search source icon 903 may be an icon indicating the third search source application 803, and the fourth search source icon 904 may be an icon indicating the fourth search source application 804.

The plurality of search source icons 901 to 904 may be differently arranged according to the frequency of use of the search source application.

That is, as the frequency of use of search source applications increases, corresponding search source icons may be placed in front.

When one of the plurality of search source icons 901 to 904 is selected, the controller 170 may display search results for the search source 601 through a search source application corresponding to the selected icon. This will be described below.

Fig. 5 is again described.

The controller 170 receives a command for selecting any one of the search source applications included in the search source application list (S511) and displays search results for the search source through the selected search source application (S513).

As shown in fig. 9, the controller 170 may receive a command for selecting any one of a plurality of search source icons 901 to 904 included in a search source application list through the pointer 205.

The controller 170 may acquire search results of the search source 601 through a search source application corresponding to the selected search source icon according to the received command.

In an embodiment, the controller 170 may display search results for a search source through the selected search source application in a preview form.

In another embodiment, the controller 170 may display the search results applied by the selected search source full screen.

Fig. 10 is a diagram illustrating an example of outputting search results of a search source in a preview form according to an embodiment of the present disclosure.

Fig. 10 may be a screen displayed after the embodiment of fig. 9.

Referring to fig. 10, the controller 170 may determine whether the pointer 205 is located on the second search source icon 902.

When it is determined that the pointer 205 is located on the second search source icon 902, the controller 170 may display the preview search result 1000 of the search source 601 in a preview form on one side of the second search source icon 902.

The search results 1000 may be search information performed by the second search source application 812 that matches the second search source icon 902. When pointer 205 is positioned over second search source icon 902, controller 170 may execute second search source application 812 in the background to obtain search results for search source 601.

The controller 170 may capture the obtained search results and acquire the captured images as preview search results 1000.

In another implementation, the preview search results 1000 may indicate search results previously found by the second search source application 812.

In another embodiment, the preview search result 1000 may be a sample result previously stored in the storage part 140.

According to the embodiment of the present disclosure, the result of the search source application switched to the search source 601 is displayed in advance, similar to the preview search result 1000, thereby preventing the user from switching unnecessary applications.

Fig. 11 is a diagram illustrating an example of search results of a full screen output search source according to an embodiment of the present disclosure.

In fig. 10, when a command for selecting the second search source icon 902 is received, the controller 170 may switch from the EPG screen 600 to the search result screen 1100.

The search results screen 1100 may include search information regarding the search source 601 acquired by the second search source application 812.

When the search source 601 is a program name, the search information may include a broadcasting time of the program, a rating of the program, and a thumbnail of the program.

In this way, the user can easily examine the search results of the search source without having to enter separate text for searching. In addition, since a plurality of search source applications are provided for one search source, various search results can be checked.

Fig. 12 and 13 are diagrams for describing a process of providing search results using an image as a search source according to an embodiment of the present disclosure.

Referring to fig. 12, the display apparatus 100 displays an execution screen 1200 of a live TV application as a calling application. The execution screen 1200 of the live TV application may be a real-time broadcast screen.

When receiving an execution command of the contextual search application, the display apparatus 100 may display a plurality of search source icons 901 to 905 and 907 corresponding to the plurality of search source applications around the pointer 205.

The plurality of search source icons 901 to 905 and 907 may be icons corresponding to a plurality of search source applications matching a live TV application, respectively.

As shown in fig. 13, when a command for selecting any one of the plurality of search source icons 901 to 905 and 907 is selected through the pointer 205, the display apparatus 100 may display a search result 1300 for an image of the execution screen 1200 by a search source application corresponding to the search source icon selected according to the command.

That is, the display apparatus 100 may acquire an image of the execution screen 1200 of the live TV application as a search source.

When the image is a landscape photograph of rufihn, switzerland, the search result may include search information about rufihn.

As described above, according to the embodiments of the present disclosure, a user can easily check search results of images while watching TV.

Fig. 14 and 15 are diagrams for describing a process of providing search results using an object included in an image as a search source according to an embodiment of the present disclosure.

Referring to fig. 14, the display apparatus 100 displays an execution screen 1400 of a live TV application.

The display device 100 may periodically receive data of image frames from an Automatic Content Recognition (ACR) server.

The data of the image frame may include identification data of an object included in the image frame and coordinate data of the object. The identification data of the object is an identifier for identifying the object, and may include a name of the object, a brand of the object, and a model of the object.

The object may be selected or clicked on by pointer 205.

When the pointer 205 is located on the object 1410, the display device 100 may extract identification data of the object 1410 from data regarding the image frame.

When receiving an execution command of the contextual search application, the display apparatus 100 may display a plurality of search source icons 901 to 905 and 907 corresponding to the plurality of search source applications around the pointer 205.

When one of the plurality of search source icons 901 to 905 and 907 is selected, the display apparatus 100 may transmit identification data of the object 1410 to a search source application corresponding to the selected icon.

When the search source application corresponding to the selected icon is a shopping application for providing shopping information, the shopping application may acquire a search result based on the identification data of the object 1410.

As shown in fig. 15, the display apparatus 100 may display a search screen 1500 including search information of an object 1410 acquired through a shopping application.

Further, according to another embodiment, the display apparatus 100 may transmit the image frame of the execution screen 1400 and the coordinate information of the pointer 205 located in the image frame to the ACR server.

The ACR server may acquire identification data of an object based on the received image frame and coordinate information of the pointer 205 and transmit the acquired object identification data to the display apparatus 100.

The display device 100 may acquire object identification data received from the ACR server as a search source. The display apparatus 100 may provide search information on the object identification data through the selected search source application.

As described above, according to the embodiments of the present disclosure, a user can easily check search results of a desired object while viewing an image.

Fig. 16 is a diagram illustrating an example of controlling a home appliance according to an embodiment of the present disclosure.

Referring to fig. 16, a main screen 1600 of the display device 100 is shown.

The home screen 1600 may include IoT device items that are capable of controlling multiple home appliances connected to the display device 100 through the IoT.

The IoT device item may include device buttons corresponding to a plurality of home appliances.

When the pointer 205 is located on the air conditioner button 1610, the display device 100 may arrange a plurality of control icons 1611 to 1614, which may control the operation of the air conditioner 1610, around the pointer 205.

The first control icon 1611 may be an icon for increasing the wind speed of the air conditioner, the second control icon 1612 may be an icon for decreasing the wind speed of the air conditioner, the third control icon 1613 may be an icon for setting the operation mode of the air conditioner, and the fourth control icon 1614 may be an icon for adjusting the wind volume of the air conditioner.

When one of the plurality of control icons 1611 to 1614 is selected, the display device 100 may transmit a command for controlling an operation of the air conditioner corresponding to the selected icon to the air conditioner.

Further, according to another embodiment of the present disclosure, the search source may be voice data corresponding to a voice spoken by the user.

In this case, operations S503 and S505 of fig. 5 may be omitted.

The display device 100 may convert voice data into text data and acquire the converted text data as a search source. The display apparatus 100 may display a plurality of search source applications corresponding to the calling application according to an execution command of the contextual search application.

The display apparatus 100 may display search results of text data on the display 180 through a search source application selected among a plurality of search source applications.

As described above, according to embodiments of the present disclosure, search results may be obtained through a desired search source application using a voice command spoken by a user as a search source.

Fig. 17 is a diagram for describing a method of generating successive additional search sources based on any one search source according to another embodiment of the present disclosure.

The display apparatus 100 may acquire the second search source and the third search source based on the metadata of the first search source.

Fig. 17 shows a mind map 1700 based on a first search source, including a second search source generated and a third search source generated by the second search source.

The search source may be text such as keywords. Mind map 1700 may be a map that includes additional keywords dynamically generated based on any one keyword.

In fig. 9, one of a plurality of search source icons 901 to 904 may be used to output a mind map 1700.

For example, when the fourth search source icon 904 is an icon corresponding to an application for providing a mind map, a mind map 1700 using the program name 601 as the first search source may be displayed according to selection of the fourth search source icon 904.

The initial mind map 1700 may include only the first search source 1710 corresponding to the program name 601. The first search source 1710 may be implemented as an icon including a program name.

Thereafter, when the first search source 1710 is focused by the pointer 205, the display apparatus 100 may acquire a plurality of found second search sources 1731 through 1735 based on the first search source 1710 and display the plurality of acquired second search sources 1173 through 1735.

The display device 100 may obtain the search results of the first search source 1710 through a website.

The plurality of second search sources 1173 through 1735 may be distinguished from the third search source area 1750 by a second search source area 1730.

Each of the plurality of second search sources 1731 through 1735 is a name of a respective actor (actress 1, actress 2, actress 3), a live view response, a detailed genre of the program appearing in the program indicated by the first search source 1710.

When pointer 205 is located on detailed genre 1735, display device 100 may search for and generate popular latest works 1751 and 1753 as third search sources based on detailed genre 1735.

The display device 100 may obtain a third search source for the detailed genre 1735 through a website.

In this way, a third search source area 1750 including a plurality of third search sources may be generated by the plurality of second search sources 1173 through 1735.

According to an embodiment of the present disclosure, the display apparatus 100 may automatically generate the third search source when one of the plurality of second search sources 1173 to 1735 is not focused for a specific time.

As described above, a user can easily grasp search information at a glance through the mind map 1700 according to embodiments of the present disclosure without having to input a separate search term.

Fig. 18 and 19 are diagrams illustrating a process of generating a mind map according to an embodiment of the present disclosure.

Specifically, fig. 18 shows a search source table 1800 showing a process of generating a mind map when the genre of a program is drama, movie, and entertainment, and fig. 19 shows a search source table 1900 showing a process of showing a mind map when the genre of a program is sports.

The search source table 1800 of fig. 18 may include five second search sources (actor 1, actor 2, actor 3, detailed genre, and related recommendation keywords) generated based on the first search sources.

The display apparatus 100 may generate a third search source based on the second search source, a fourth search source based on the third search source, and a fifth search source based on the fourth search source.

For example, when the second search source is actor 1 of a program, the latest exposure 1 of actor 1 may be generated as the third search source.

The display device 100 may acquire the actor of the latest exposure field 1 as the third search source as the fourth search source.

The display device 100 may acquire the exposed surface of the performer as the fourth search source as the fifth search source.

In an embodiment, the top search source and the bottom search source may be used for search results simultaneously. For example, actor 1 as the second search source and the fashion as the third search source may be used to search for the search result "fashion of actor 1" at the same time.

Further, the related recommendation keywords may indicate recommendation keywords associated with program names.

Actor 1, actor 2, and actor 3 may be the primary roles among actors appearing in a program.

The display apparatus 100 may search for a character and a program name included in metadata of a program through the web and determine the first character as a main character. The display apparatus 100 may acquire the name of the actor determined to be the main character as the second search source.

As another example, when the primary performer of the program is not known, the display device 100 may randomly or sequentially display some performers.

Next, fig. 19 is described.

The search source table 1900 of fig. 19 may include seven second search sources (team 1, team 2, sports alarm, four related recommendation keywords) generated based on a first search source (e.g., baseball).

The display device 100 may generate a third search source based on the first search source and the second search source.

For example, when the second search source is team 1 (the name of a baseball team), the third search source may be the uniform of team 1.

The display apparatus 100 may acquire and display search results using a first search source (baseball), a second search source (team 1), and a third search source (uniform).

The sports alarm clock may be used to set an alarm clock for a game based on a play schedule of a baseball game.

The related recommended keywords may include keywords related to baseball as a search source.

According to various embodiments of the present disclosure, search results of a search source may be quickly and easily checked.

According to various embodiments of the present disclosure, search results for one search source may be provided by multiple applications, thereby maximizing the usability of the search.

In addition, according to various embodiments of the present disclosure, attributes of an application are reflected to a search source, thereby greatly improving search efficiency.

According to an embodiment of the present disclosure, the above-described method may be embodied as processor readable codes on a medium having a program recorded thereon. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and may be implemented in the form of carrier waves (e.g., transmission via the internet).

The above-described display device may not be limited to the configuration and method applied to the above-described embodiments, but may be configured by selectively combining all or some of the embodiments so that various modifications may be made.

Cross Reference to Related Applications

The present application claims priority from korean patent application No.10-2020-0052366 (filed on 29 th month 4 in 2020), which is incorporated herein by reference in its entirety.

Claims (13)

1. A display device, the display device comprising:

a display configured to display an execution screen of a calling application including search sources, wherein the execution screen includes a plurality of search sources, and the plurality of search sources includes one or more text or images; and

A controller configured to:

Receiving a command for selecting a search source among the plurality of search sources;

displaying, on the display according to the received command, a plurality of search source icons identifying a plurality of search source applications matching the calling application, wherein each of the plurality of search source applications is an application that outputs a different search result for the search source;

When one of the plurality of search source icons is selected, obtaining search results for the search source by identifying a search source application of the selected icon; and

The acquired search results are displayed on the display.

2. The display device of claim 1, wherein when a pointer controlled by a remote control device is located on one of the one or more texts, the controller is configured to acquire the text on which the pointer is located as the search source.

3. The display device of claim 2, wherein when a pointer controlled by a remote control is located on the search source, the controller is configured to display an indicator on the display indicating that a contextual search application is executable.

4. The display device of claim 1, further comprising a storage configured to store a source table in which the plurality of search source applications match a plurality of calling applications.

5. The display apparatus according to claim 4, wherein the controller is configured to extract the plurality of search source applications matching the plurality of calling applications by using the source table stored in the storage section.

6. The display device of claim 1, wherein the controller is configured to obtain, by the search source application, search results including contextual search sources according to attributes of the search source and intent obtained by the calling application.

7. The display device of claim 1, wherein the controller is configured to display the search results in a preview on one side of the selected icon.

8. The display apparatus according to claim 1, wherein the controller is configured to switch the execution screen of the calling application to the search result.

9. The display device of claim 1, wherein the controller is configured to:

determining whether the plurality of search source applications matching the calling application are installed on the display device; and

Only a search source icon corresponding to the search source application installed on the display device is displayed on the display according to the determination result.

10. The display device of claim 1, wherein the controller is configured to:

identifying an object included in an image of the execution screen; and

Identification data of the identified object is acquired as the search source.

11. The display device of claim 10, wherein the controller is configured to receive data regarding the image from an automatic content recognition server, and

Wherein the received data about the image comprises identification data of the object.

12. The display device according to claim 11, wherein when a pointer controlled by a remote controller is located on the object, the controller is configured to acquire identification data of the object corresponding to coordinates where the pointer is located from the data on the image.

13. A method of operating a display device, the method comprising the steps of:

displaying an execution screen of a calling application including search sources, wherein the execution screen includes a plurality of search sources, and the plurality of search sources includes one or more text or images;

Receiving a command for selecting a search source among the plurality of search sources;

Displaying a plurality of search source icons identifying a plurality of search source applications matching the calling application in accordance with the received command, wherein each of the plurality of search source applications is an application that outputs a different search result for the search source;

When one of the plurality of search source icons is selected, obtaining search results for the search source by identifying a search source application of the selected icon; and

And displaying the acquired search results on the display device.