WO2009130664A1 - System and method for controlling video output - Google Patents

System and method for controlling video output Download PDF

Info

Publication number
WO2009130664A1
WO2009130664A1 PCT/IB2009/051629 IB2009051629W WO2009130664A1 WO 2009130664 A1 WO2009130664 A1 WO 2009130664A1 IB 2009051629 W IB2009051629 W IB 2009051629W WO 2009130664 A1 WO2009130664 A1 WO 2009130664A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
output
touch screen
user input
selection
Prior art date
Application number
PCT/IB2009/051629
Other languages
French (fr)
Inventor
Sriram Subramanian
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Publication of WO2009130664A1 publication Critical patent/WO2009130664A1/en

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/001—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
    • G09G3/002—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1626—Constructional details or arrangements for portable computers with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1637—Details related to the display arrangement, including those related to the mounting of the display in the housing
    • G06F1/1639—Details related to the display arrangement, including those related to the mounting of the display in the housing the display being based on projection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3141—Constructional details thereof
    • H04N9/3173—Constructional details thereof wherein the projection device is specially adapted for enhanced portability
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03H—HOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00—Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/22—Processes or apparatus for obtaining an optical image from holograms
    • G03H1/2294—Addressing the hologram to an active spatial light modulator
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/038—Indexing scheme relating to G06F3/038
    • G06F2203/0381—Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0492—Change of orientation of the displayed image, e.g. upside-down, mirrored
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/14—Solving problems related to the presentation of information to be displayed
    • G09G2340/145—Solving problems related to the presentation of information to be displayed related to small screens

Definitions

  • the invention relates to a method of controlling video output.
  • the invention also relates to a system for controlling video output.
  • the invention also relates to a data processing device comprising a screen and a device for displaying a second output of information on a surface other than the screen, and comprising a system for controlling video output.
  • the invention also relates to a computer program.
  • the UltraMon multi-monitor guide discloses a utility for window management on a multi-monitor system.
  • a multi-monitor system is a computer with more than one monitor connected to it, either by adding additional video cards or by using video cards with multiple outputs.
  • There are several ways to get multiple monitors with a laptop Lots of laptops have a VGA port for an external monitor, and by default the external monitor will show the same image as the built-in monitor. If the laptop has a video chipset with multi- monitor support (and the necessary drivers) one can choose to use the external monitor as a second monitor instead of a mirror of the built in monitor.
  • Window management allows a user quickly to move windows between monitors, or to maximize a window to the desktop using additional window buttons added by UltraMon, the utility. With custom window buttons, one can move a window to a different monitor with the click of a button.
  • a problem of the known method and system is that it is not suitable for analogous application in a small portable device with a small primary monitor and a projector or external monitor as the secondary monitor, such as a mobile phone provided with a pico- projector.
  • the screens of such devices are too small to allow one to select a window quickly using a mouse, much less a particular one of several custom window buttons, and any other selection will require a key combination that a user must remember and enter without fault.
  • This object is achieved by providing a method of controlling video output, including: causing a first output of information to be displayed on a touch screen; causing a second output of information to be displayed on a surface other than the touch screen; inferring a selection of part of the information represented in one of the first and second outputs of information from user input to the touch screen; and making an alteration to the second output of information corresponding to the selected part responsive to user input. Because the first output of information is caused to be displayed on a touch screen and the selection of part of the information represented in one of the first and second outputs of information is inferred from user input to the touch screen, the user is able to select visible parts of the first or second output relatively easily.
  • the method is especially suitable for use in conjunction with a projector for projecting images onto the further surface, because a user can ensure that only selected parts of an output of information as visible to him or her on the touch screen are visible to all and sundry on the further surface.
  • the selected parts need not correspond to particular windows of a GUI provided by means of the touch screen. Separate icons or other GUI elements to make a selection from an output of information are also not required. The user can easily select anything that is visible, and what is visible can be limited to useful information.
  • the alteration is made responsive to completion of receipt of the user input from which the selection is inferred and user input obtained through activation of a single control element by a user.
  • An effect is that there is a single instruction to make a transfer, or to copy. The transfer or copy is made immediately on receipt of the single instruction or the single instruction is used as a cue to make the transfer or copy as soon as a valid selection of part of the information represented in one of the first and second outputs of information has been made.
  • An embodiment of the method includes transferring a selected part of the information represented in the first output of information to the second output of information.
  • the selected part of the information is removed from the first output of information.
  • An effect is to free up space on the touch screen. This is useful where the size of the touch screen is limited and the further surface is visible to the user anyway.
  • An embodiment of the method includes: identifying at least part of output of an application running on a data processing device based on the selection inferred from user input to the touch screen, and providing the identified output to a driver for a device caused to display the second output of information on the surface other than the touch screen.
  • An effect is to allow for adaptation to the resolution of the device caused to display the second output of information.
  • the selected part can be moved from the touch screen to the projector output and displayed at a higher resolution. This would not be possible if part of merely a video signal were to be transferred.
  • An embodiment of the method includes: identifying visible elements of a graphical user interface based on the selection inferred from user input to the touch screen, wherein the alteration includes at least one of removal and addition of representations of the respective visible elements from and to the second output of information.
  • An effect is that aspects of the representation of the visible elements can be varied according to whether they are displayed on the touch screen or the further surface.
  • the aspect ratio of a window or output box can be changed, for instance, or the font size relative to a text box.
  • a system for controlling video output including: an interface to a display device for displaying a first output of information on a touch screen, an interface to a device for displaying a second output of information on a surface other than the touch screen, and an interface to user controls, which system is configured to: infer a selection of part of information represented in one of the first and second outputs of information from user input to the touch screen, and to make an alteration to the second output of information corresponding to the selected part responsive to user input.
  • the system may be implemented by suitably programming a data processing device comprising or interfacing with a touch screen and a further display device, e.g. an external monitor or a projector.
  • the system is configured to perform a method according to the invention.
  • a data processing device comprising a touch screen and a device for displaying a second output of information on a surface other than the touch screen and comprising a system for controlling video output according to the invention.
  • the touch screen and device for displaying the second output of information are, in a useful implementation, configured to display information at different formats, in particular sizes.
  • the touch screen has a small area of display whereas the device for displaying the second output of information provides a large area of display.
  • the touch screen can be kept small to display information only the user needs to see.
  • the device for displaying the second output of information can be used to display elements of information the user wishes to share.
  • the device for displaying the second output of information comprises a projector.
  • a user of such a device can easily select elements of the information displayed to him on the touch screen for display by the projector on a surface, so that such elements of information can be shared with other users. Information the user would rather keep private can be kept out of the selection.
  • the projector is an integral component of the device.
  • Typical display resolutions achievable with a pico-projector for integration into e.g. a mobile telephone are 848 x480, 1024 x 678 or even 1600 x 1024 pixels.
  • Suitable projectors are based on Liquid-Crystal-on-Silicon (LCOS) technology or Digital Light Processing (DLP) technology, optionally using holographic diffraction or multiple laser light sources to achieve such a relatively high resolution. Integrating the projector into the device improves the portability of the device.
  • the system of the invention is particularly adapted to portable devices, because the touch screen will generally be of limited size in such a device.
  • a computer program including a set of instructions capable, when incorporated in a machine-readable medium, of causing a system having information processing capabilities to perform a method according to the invention.
  • Fig. 1 is a schematic depiction of a mobile phone equipped with a touch screen and pico-projector
  • Fig. 2 is a schematic diagram illustrating selected components of the mobile phone; and Fig. 3 is a flow chart giving a simplified outline of a method of controlling video output.
  • a mobile phone 1 is shown as an example of a data processing device comprising a system for controlling video output, a touch screen 2 and a device for displaying information on a surface other than that of the touch screen 2.
  • a data processing device comprising portable media players, e.g. MP3 players or DVD players, Personal Digital Assistants (PDAs), alarm clocks and other household appliances with a Graphical User Interface, in-car displays (e.g. those providing a head-up display in addition to the vehicle instrument display), and devices and apparatus combining several of these functions.
  • user input is provided to the touch screen 2 using a stylus 3.
  • the user may use one or more fingers or some other implement.
  • Touch screens are known per se, and the methods outlined herein are suitable for any type of touch screen technology, e.g. capacitive sensing, surface-acoustic waves, etc.
  • Typical display resolutions for the touch screen 2 are 800 x 480 pixels or lower, e.g. 320 x 240 pixels.
  • the touch screen 2 will have a diagonal of 11 inches or less (27.94 cm).
  • the actual display device comprised in the touch screen 2 can be an LCD display device, an OLED display device or a passive display device using electronic ink, for example.
  • the mobile phone 1 of the example is provided with an integrated projector 4 for projecting images through a lens 5 onto a screen (not shown).
  • the size and focus can be adjusted, e.g. automatically or responsive to user input.
  • the mobile phone 1 is provided with numerical keys 6, function keys 7 and a navigation key 8 for navigating through menus provided by a GUI.
  • the GUI may further provide "soft keys" in the form of elements shown on the touch screen 2 that can be selected by using the stylus 3. Activation of such a single control element results in an instruction for processing by the mobile phone 1.
  • the mobile phone 1 is provided with a central processing unit (CPU) 9 (Fig. 2) and main memory 10 for executing software stored, for example, in nonvolatile memory 11.
  • the mobile phone 1 is provided with a microphone module 12 and speaker module 13 for transducing a speech signal into a digital electronic signal and a digital output signal into an acoustic signal, respectively.
  • the mobile phone 1 also comprises a hardware decoder 14 for decoding coded and/or compressed audio signals, e.g. MP3 signals, so that the mobile phone can be used as a portable audio player.
  • a similar alternative embodiment comprises a video decompression chip, e.g. an MPEG-2 or MPEG-4 decoder, for providing video output to the touch screen 2 or projector 4.
  • the CPU 9 receives input through a keyboard interface 15 and a device 16 for locating the position of at least one object, such as the stylus 3, placed against the touch screen 2. Output is provided by means of a touch screen display driver 17 to a display device 18 comprised in the touch screen 2. A second output of information is provided to a driver 19 for the projector 4.
  • an operating system of the mobile phone 1 will generally comprise device drivers implemented in software for providing appropriate output to the drivers 17,19.
  • the device drivers implemented in software constitute interfaces for communicating with the display device 18 and driver 17 and the projector 4 and driver 19. They are hardware-dependent and operating system-dependent software components that enable applications such as an audio player application, diary application, GPS navigation application, address book application, etc. to cause an output of information to be represented on the display device 18 or a surface at which the projector 4 is aimed.
  • the mobile phone 1 comprises a wireless communications unit 20 and antenna amplifier 21.
  • the mobile phone 1 allows the user, when both the display device 18 and projector 4 are operational, to transfer and/or copy certain parts of outputs of information represented on the screen of the display device and on the surface at which the projector 4 is aimed from the one to the other. Examples of use cases include:
  • the projector 4 could project the title of the song that is currently playing (but not the rest of the user's playlist displayed on the touch screen 2);
  • streaming video may be displayed by the projector 4, but not information relating to the use of the mobile phone 1 for personal communications;
  • slides When provided with presentation software, slides may be projected by the projector 4 but not the output of any other applications running on the mobile phone 1.
  • the operating system of the mobile phone 1 detects (step 22) the actuation of a single key, e.g. the above-mentioned soft key or function key 7, or any other type of user control providing a binary output.
  • a single key e.g. the above-mentioned soft key or function key 7, or any other type of user control providing a binary output.
  • This provides a cue that input received (step 23) from the device 16 for locating the position of the stylus 3 should be used to infer (step 24) a selection of part of the information represented on either the touch screen 2 or the surface at which the projector 4 is aimed.
  • the mobile phone 1 is provided with a function for speech recognition, enabling it to interpret spoken commands. This makes it possible to use a voice command as a cue instead of the single key.
  • the key may appear after navigation of a menu, but is generally a key that is permanently accessible in at least the default operating mode in which the projector 4 is operational.
  • the inference it is possible to infer from the selection of particular region within the touch screen output, that part of the information represented in a corresponding region of the projected output should be selected.
  • a particular GUI element may have been transferred (i.e. removed) from the output displayed on the touch screen 2, and selection of the region where it was represented is an indication that it is to be transferred back again.
  • the selection will be of part of the information represented in the output of information caused to be displayed on the touch screen 2.
  • the inference will be of the selection of an element of a GUI, e.g. a complete window, a text box within a window, a soft button or dial, etc.
  • Selection by the user may be by encircling ("lassoing") part of the information represented on the touch screen 2.
  • lassoing encircling
  • a simple tap on the element will be sufficient.
  • a step 25 corresponding to the first step 22 is carried out generally simultaneously or subsequent to receipt of touch screen input. Receipt of the user input signaling the actuation of a single particular key, e.g. the above-mentioned soft key or function key 7 is a cue that the selection is that of part of an output of information to be copied or transferred between the output caused to be displayed by the projector 4 and the output of information caused to be displayed by the touch screen 2.
  • the first step 22 is omitted in this alternative embodiment.
  • the CPU 9 responsive to completion of receipt of the user input from which the selection is inferred and the user input obtained through activation of the single control element, makes (step 26) an alteration to the output of information caused to be displayed by the projector 4 corresponding to the selection of information made by the user.
  • the CPU 9 may resize, crop, rotate or change the font of text comprised in the GUI element(s) concerned, and compose the output using the thus processed element(s), which output is provided to the software driver for the projector 4.
  • the adaptation of the GUI elements can be in accordance with certain pre-programmed rules or user settings, for example.
  • the selected output information is still that of information in a format for provision to a software driver for the projector, so that at least a resolution and possibly color and brightness settings appropriate to the projector 4.
  • a software driver for the projector so that at least a resolution and possibly color and brightness settings appropriate to the projector 4.
  • persons in the vicinity can enjoy the movie, but are not shown menus, soft keys and other output provided for display on the touch screen 2.
  • the movie may continue to be displayed on the touch screen 2, or be removed to free space, depending on the embodiment or, for example, user-adjustable settings.
  • CPU 9 and the drivers 17,19 selects part of the graphics signal provided to the one and transfers or copies that part to the other responsive to an inference of a selection made by the CPU 9 on the basis of input to the touch screen 2.
  • the second output of information can be caused to be displayed on a surface other than the touch screen 2 by transmitting it via a local wireless link to an external projector. It may even be transmitted, e.g. by using a short messaging service, to a server arranged to communicate it to a projector.
  • Suitable wireless links to a local external projector include those based on Bluetooth, IEEE 802.1 Ix or ZigBee (IEEE 802.15.4) technology, amongst others.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

A method of controlling video output includes causing a first output of information to be displayed on a touch screen (2). A second output of information is caused to be displayed on a surface other than the touch screen (2). A selection of part of the information represented in one of the first and second outputs of information is inferred from user input to the touch screen (2). An alteration is made to the second output of information corresponding to the selected part responsive to user input.

Description

System and method for controlling video output
FIELD OF THE INVENTION
The invention relates to a method of controlling video output. The invention also relates to a system for controlling video output. The invention also relates to a data processing device comprising a screen and a device for displaying a second output of information on a surface other than the screen, and comprising a system for controlling video output.
The invention also relates to a computer program.
BACKGROUND OF THE INVENTION The UltraMon multi-monitor guide, downloaded from http://www.realtimesoft.com/multimon/guide, on 3 April 2008, discloses a utility for window management on a multi-monitor system. A multi-monitor system is a computer with more than one monitor connected to it, either by adding additional video cards or by using video cards with multiple outputs. There are several ways to get multiple monitors with a laptop. Lots of laptops have a VGA port for an external monitor, and by default the external monitor will show the same image as the built-in monitor. If the laptop has a video chipset with multi- monitor support (and the necessary drivers) one can choose to use the external monitor as a second monitor instead of a mirror of the built in monitor. Window management allows a user quickly to move windows between monitors, or to maximize a window to the desktop using additional window buttons added by UltraMon, the utility. With custom window buttons, one can move a window to a different monitor with the click of a button.
A problem of the known method and system is that it is not suitable for analogous application in a small portable device with a small primary monitor and a projector or external monitor as the secondary monitor, such as a mobile phone provided with a pico- projector. The screens of such devices are too small to allow one to select a window quickly using a mouse, much less a particular one of several custom window buttons, and any other selection will require a key combination that a user must remember and enter without fault. SUMMARY OF THE INVENTION
It is an object of the invention to provide a method, system, computer program and data processing device that are suitable for use in small form- factor data processing devices for showing only information intended for wider dissemination on the surface other than the screen of the data processing device.
This object is achieved by providing a method of controlling video output, including: causing a first output of information to be displayed on a touch screen; causing a second output of information to be displayed on a surface other than the touch screen; inferring a selection of part of the information represented in one of the first and second outputs of information from user input to the touch screen; and making an alteration to the second output of information corresponding to the selected part responsive to user input. Because the first output of information is caused to be displayed on a touch screen and the selection of part of the information represented in one of the first and second outputs of information is inferred from user input to the touch screen, the user is able to select visible parts of the first or second output relatively easily. These can then be shown at a larger size by alteration of the second output of information. The method is especially suitable for use in conjunction with a projector for projecting images onto the further surface, because a user can ensure that only selected parts of an output of information as visible to him or her on the touch screen are visible to all and sundry on the further surface. The selected parts need not correspond to particular windows of a GUI provided by means of the touch screen. Separate icons or other GUI elements to make a selection from an output of information are also not required. The user can easily select anything that is visible, and what is visible can be limited to useful information.
In an embodiment, the alteration is made responsive to completion of receipt of the user input from which the selection is inferred and user input obtained through activation of a single control element by a user. An effect is that there is a single instruction to make a transfer, or to copy. The transfer or copy is made immediately on receipt of the single instruction or the single instruction is used as a cue to make the transfer or copy as soon as a valid selection of part of the information represented in one of the first and second outputs of information has been made. An embodiment of the method includes transferring a selected part of the information represented in the first output of information to the second output of information.
Thus, the selected part of the information is removed from the first output of information. An effect is to free up space on the touch screen. This is useful where the size of the touch screen is limited and the further surface is visible to the user anyway.
An embodiment of the method includes: identifying at least part of output of an application running on a data processing device based on the selection inferred from user input to the touch screen, and providing the identified output to a driver for a device caused to display the second output of information on the surface other than the touch screen.
An effect is to allow for adaptation to the resolution of the device caused to display the second output of information. For example, where the touch screen is relatively small, and the device caused to display the second output of information the surface other than the touch screen is capable of higher resolutions, the selected part can be moved from the touch screen to the projector output and displayed at a higher resolution. This would not be possible if part of merely a video signal were to be transferred.
An embodiment of the method includes: identifying visible elements of a graphical user interface based on the selection inferred from user input to the touch screen, wherein the alteration includes at least one of removal and addition of representations of the respective visible elements from and to the second output of information.
An effect is that aspects of the representation of the visible elements can be varied according to whether they are displayed on the touch screen or the further surface. The aspect ratio of a window or output box can be changed, for instance, or the font size relative to a text box.
According to another aspect of the invention, there is provided a system for controlling video output, including: an interface to a display device for displaying a first output of information on a touch screen, an interface to a device for displaying a second output of information on a surface other than the touch screen, and an interface to user controls, which system is configured to: infer a selection of part of information represented in one of the first and second outputs of information from user input to the touch screen, and to make an alteration to the second output of information corresponding to the selected part responsive to user input. The system may be implemented by suitably programming a data processing device comprising or interfacing with a touch screen and a further display device, e.g. an external monitor or a projector.
In an embodiment, the system is configured to perform a method according to the invention. According to another aspect of the invention, there is provided a data processing device comprising a touch screen and a device for displaying a second output of information on a surface other than the touch screen and comprising a system for controlling video output according to the invention.
The touch screen and device for displaying the second output of information are, in a useful implementation, configured to display information at different formats, in particular sizes. Usefully, the touch screen has a small area of display whereas the device for displaying the second output of information provides a large area of display. The touch screen can be kept small to display information only the user needs to see. The device for displaying the second output of information can be used to display elements of information the user wishes to share.
In an embodiment, the device for displaying the second output of information comprises a projector.
This has advantages in conjunction with a small touch screen, such as those typically provided on portable media players or Personal Digital Assistants (PDAs). A user of such a device can easily select elements of the information displayed to him on the touch screen for display by the projector on a surface, so that such elements of information can be shared with other users. Information the user would rather keep private can be kept out of the selection.
In an embodiment, the projector is an integral component of the device. Typical display resolutions achievable with a pico-projector for integration into e.g. a mobile telephone are 848 x480, 1024 x 678 or even 1600 x 1024 pixels. Suitable projectors are based on Liquid-Crystal-on-Silicon (LCOS) technology or Digital Light Processing (DLP) technology, optionally using holographic diffraction or multiple laser light sources to achieve such a relatively high resolution. Integrating the projector into the device improves the portability of the device. The system of the invention is particularly adapted to portable devices, because the touch screen will generally be of limited size in such a device.
According to another aspect of the invention, there is provided a computer program including a set of instructions capable, when incorporated in a machine-readable medium, of causing a system having information processing capabilities to perform a method according to the invention.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention will be explained in further detail with reference to the accompanying drawings, in which:
Fig. 1 is a schematic depiction of a mobile phone equipped with a touch screen and pico-projector;
Fig. 2 is a schematic diagram illustrating selected components of the mobile phone; and Fig. 3 is a flow chart giving a simplified outline of a method of controlling video output.
DETAILED DESCRIPTION
A mobile phone 1 is shown as an example of a data processing device comprising a system for controlling video output, a touch screen 2 and a device for displaying information on a surface other than that of the touch screen 2. Examples of other suitable data processing devices comprise portable media players, e.g. MP3 players or DVD players, Personal Digital Assistants (PDAs), alarm clocks and other household appliances with a Graphical User Interface, in-car displays (e.g. those providing a head-up display in addition to the vehicle instrument display), and devices and apparatus combining several of these functions.
In the illustrated embodiment, user input is provided to the touch screen 2 using a stylus 3. In an alternative embodiment, the user may use one or more fingers or some other implement. Touch screens are known per se, and the methods outlined herein are suitable for any type of touch screen technology, e.g. capacitive sensing, surface-acoustic waves, etc. Typical display resolutions for the touch screen 2 are 800 x 480 pixels or lower, e.g. 320 x 240 pixels. In general, the touch screen 2 will have a diagonal of 11 inches or less (27.94 cm). The actual display device comprised in the touch screen 2 can be an LCD display device, an OLED display device or a passive display device using electronic ink, for example.
The mobile phone 1 of the example is provided with an integrated projector 4 for projecting images through a lens 5 onto a screen (not shown). In an embodiment, the size and focus can be adjusted, e.g. automatically or responsive to user input.
The mobile phone 1 is provided with numerical keys 6, function keys 7 and a navigation key 8 for navigating through menus provided by a GUI. The GUI may further provide "soft keys" in the form of elements shown on the touch screen 2 that can be selected by using the stylus 3. Activation of such a single control element results in an instruction for processing by the mobile phone 1.
To this end, the mobile phone 1 is provided with a central processing unit (CPU) 9 (Fig. 2) and main memory 10 for executing software stored, for example, in nonvolatile memory 11. In the illustrated embodiment, the mobile phone 1 is provided with a microphone module 12 and speaker module 13 for transducing a speech signal into a digital electronic signal and a digital output signal into an acoustic signal, respectively. In the illustrated embodiment, the mobile phone 1 also comprises a hardware decoder 14 for decoding coded and/or compressed audio signals, e.g. MP3 signals, so that the mobile phone can be used as a portable audio player. A similar alternative embodiment comprises a video decompression chip, e.g. an MPEG-2 or MPEG-4 decoder, for providing video output to the touch screen 2 or projector 4.
The CPU 9 receives input through a keyboard interface 15 and a device 16 for locating the position of at least one object, such as the stylus 3, placed against the touch screen 2. Output is provided by means of a touch screen display driver 17 to a display device 18 comprised in the touch screen 2. A second output of information is provided to a driver 19 for the projector 4. It is noted that an operating system of the mobile phone 1 will generally comprise device drivers implemented in software for providing appropriate output to the drivers 17,19. The device drivers implemented in software constitute interfaces for communicating with the display device 18 and driver 17 and the projector 4 and driver 19. They are hardware-dependent and operating system-dependent software components that enable applications such as an audio player application, diary application, GPS navigation application, address book application, etc. to cause an output of information to be represented on the display device 18 or a surface at which the projector 4 is aimed.
Of course, to make cellular communications possible, the mobile phone 1 comprises a wireless communications unit 20 and antenna amplifier 21. The mobile phone 1 allows the user, when both the display device 18 and projector 4 are operational, to transfer and/or copy certain parts of outputs of information represented on the screen of the display device and on the surface at which the projector 4 is aimed from the one to the other. Examples of use cases include:
When using the audio player functionality, the projector 4 could project the title of the song that is currently playing (but not the rest of the user's playlist displayed on the touch screen 2);
When using the telephone functionality, streaming video may be displayed by the projector 4, but not information relating to the use of the mobile phone 1 for personal communications;
When provided with presentation software, slides may be projected by the projector 4 but not the output of any other applications running on the mobile phone 1.
Referring to Fig. 3, in one embodiment, the operating system of the mobile phone 1 detects (step 22) the actuation of a single key, e.g. the above-mentioned soft key or function key 7, or any other type of user control providing a binary output. This provides a cue that input received (step 23) from the device 16 for locating the position of the stylus 3 should be used to infer (step 24) a selection of part of the information represented on either the touch screen 2 or the surface at which the projector 4 is aimed. In an embodiment, the mobile phone 1 is provided with a function for speech recognition, enabling it to interpret spoken commands. This makes it possible to use a voice command as a cue instead of the single key. However, in the environment in which the mobile phone 1 is likely to be used, ambient noise will be at quite a high level. Therefore, a single key is preferred. The key may appear after navigation of a menu, but is generally a key that is permanently accessible in at least the default operating mode in which the projector 4 is operational.
As far as the inference is concerned, it is possible to infer from the selection of particular region within the touch screen output, that part of the information represented in a corresponding region of the projected output should be selected. For example, a particular GUI element may have been transferred (i.e. removed) from the output displayed on the touch screen 2, and selection of the region where it was represented is an indication that it is to be transferred back again. Generally, however, the selection will be of part of the information represented in the output of information caused to be displayed on the touch screen 2. In particular, the inference will be of the selection of an element of a GUI, e.g. a complete window, a text box within a window, a soft button or dial, etc.
Selection by the user may be by encircling ("lassoing") part of the information represented on the touch screen 2. Alternatively, and in particular in the case of selection of discrete GUI elements, a simple tap on the element will be sufficient.
In an alternative embodiment, a step 25 corresponding to the first step 22 is carried out generally simultaneously or subsequent to receipt of touch screen input. Receipt of the user input signaling the actuation of a single particular key, e.g. the above-mentioned soft key or function key 7 is a cue that the selection is that of part of an output of information to be copied or transferred between the output caused to be displayed by the projector 4 and the output of information caused to be displayed by the touch screen 2. The first step 22 is omitted in this alternative embodiment.
In both embodiments, the CPU 9, responsive to completion of receipt of the user input from which the selection is inferred and the user input obtained through activation of the single control element, makes (step 26) an alteration to the output of information caused to be displayed by the projector 4 corresponding to the selection of information made by the user.
In an embodiment, where the selection inferred is that of one or more GUI elements, the CPU 9 may resize, crop, rotate or change the font of text comprised in the GUI element(s) concerned, and compose the output using the thus processed element(s), which output is provided to the software driver for the projector 4. The adaptation of the GUI elements can be in accordance with certain pre-programmed rules or user settings, for example.
Even in the case of a straightforward transfer, e.g. of video output corresponding to partial or complete scenes of a movie, the selected output information is still that of information in a format for provision to a software driver for the projector, so that at least a resolution and possibly color and brightness settings appropriate to the projector 4. Thus, persons in the vicinity can enjoy the movie, but are not shown menus, soft keys and other output provided for display on the touch screen 2. The movie may continue to be displayed on the touch screen 2, or be removed to free space, depending on the embodiment or, for example, user-adjustable settings.
It should be noted that the embodiments described above illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Other applications of the inventions include DVD players, allowing a user to project scenes from a movie to the environment, but not the remainder of the movie, or the output of the user interface. In yet another use case, a user may select to project only the time or only the radio station information provided by an alarm clock having a touch screen and a projector. In an alternative embodiment to the one described, a switch between the
CPU 9 and the drivers 17,19 selects part of the graphics signal provided to the one and transfers or copies that part to the other responsive to an inference of a selection made by the CPU 9 on the basis of input to the touch screen 2.
As an alternative to implementing the method outlined above that uses a device with the integrated projector 4, the second output of information can be caused to be displayed on a surface other than the touch screen 2 by transmitting it via a local wireless link to an external projector. It may even be transmitted, e.g. by using a short messaging service, to a server arranged to communicate it to a projector. Suitable wireless links to a local external projector include those based on Bluetooth, IEEE 802.1 Ix or ZigBee (IEEE 802.15.4) technology, amongst others.

Claims

CLAIMS:
1. Method of controlling video output, including: causing a first output of information to be displayed on a touch screen (2); causing a second output of information to be displayed on a surface other than the touch screen (2); inferring a selection of part of the information represented in one of the first and second outputs of information from user input to the touch screen (2); and making an alteration to the second output of information corresponding to the selected part responsive to user input.
2. Method according to claim 1, wherein the alteration is made responsive to completion of receipt of the user input from which the selection is inferred and user input obtained through activation of a single control element (7) by a user.
3. Method according to claim 1, including transferring a selected part of the information represented in the first output of information to the second output of information.
4. Method according to claim 1, including: identifying at least part of output of an application running on a data processing device (1) based on the selection inferred from user input to the touch screen (2), and providing the identified output to a driver for a device (4,19) caused to display the second output of information on the surface other than the touch screen (2).
5. Method according to claim 1, including: identifying visible elements of a graphical user interface based on the selection inferred from user input to the touch screen (2), wherein the alteration includes at least one of removal and addition of representations of the respective visible elements from and to the second output of information.
6. System for controlling video output, including: an interface to a display device (18) for displaying a first output of information on a touch screen (2), an interface to a device (4,19) for displaying a second output of information on a surface other than the touch screen (2), and an interface (15) to user controls (6-8,16), which system is configured to: infer a selection of part of information represented in one of the first and second outputs of information from user input to the touch screen (2), and to make an alteration to the second output of information corresponding to the selected part responsive to user input.
7. System according to claim 6, configured to perform a method according to any one of claims 1-5.
8. Data processing device (1) comprising a touch screen (2) and a device (4) for displaying a second output of information on a surface other than the touch screen (2), and comprising a system (9-11) for controlling video output according to claim 6 or 7.
9. Data processing device according to claim 8, wherein the device (4) for displaying the second output of information comprises a projector.
10. Data processing device according to claim 9, wherein the projector is an integral component of the device.
11. Computer program including a set of instructions capable, when incorporated in a machine-readable medium, of causing a system having information processing capabilities to perform a method according to any one of claims 1-5.
PCT/IB2009/051629 2008-04-25 2009-04-21 System and method for controlling video output WO2009130664A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP08155208.5 2008-04-25
EP08155208 2008-04-25

Publications (1)

Publication Number Publication Date
WO2009130664A1 true WO2009130664A1 (en) 2009-10-29

Family

ID=40756556

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2009/051629 WO2009130664A1 (en) 2008-04-25 2009-04-21 System and method for controlling video output

Country Status (1)

Country Link
WO (1) WO2009130664A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011138551A1 (en) * 2010-05-06 2011-11-10 France Telecom Terminal including a video projector and a screen, having one area that enables control of a remote pointer projected by said video projector

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NL1015114C1 (en) * 2000-05-04 2000-09-18 Arndt Hermann Albert Hans Rudol Mobile phone or palm-top computer, has built-in image projector, to display message on nearby flat surface
EP1215568A2 (en) * 2000-12-15 2002-06-19 Nokia Corporation Method for transmitting information
US20020075250A1 (en) * 2000-10-10 2002-06-20 Kazuyuki Shigeta Image display apparatus and method, information processing apparatus using the image display apparatus, and storage medium
EP1217499A2 (en) * 2000-11-29 2002-06-26 Xybernaut Corporation Digital projection system for communication device and portable computing device
US20060238833A1 (en) * 2005-04-22 2006-10-26 Lu-Cheng Chen Portable information product having a built-in projection device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NL1015114C1 (en) * 2000-05-04 2000-09-18 Arndt Hermann Albert Hans Rudol Mobile phone or palm-top computer, has built-in image projector, to display message on nearby flat surface
US20020075250A1 (en) * 2000-10-10 2002-06-20 Kazuyuki Shigeta Image display apparatus and method, information processing apparatus using the image display apparatus, and storage medium
EP1217499A2 (en) * 2000-11-29 2002-06-26 Xybernaut Corporation Digital projection system for communication device and portable computing device
EP1215568A2 (en) * 2000-12-15 2002-06-19 Nokia Corporation Method for transmitting information
US20060238833A1 (en) * 2005-04-22 2006-10-26 Lu-Cheng Chen Portable information product having a built-in projection device

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011138551A1 (en) * 2010-05-06 2011-11-10 France Telecom Terminal including a video projector and a screen, having one area that enables control of a remote pointer projected by said video projector
FR2959839A1 (en) * 2010-05-06 2011-11-11 France Telecom TERMINAL INCORPORATING A VIDEOPROJECTOR AND A SCREEN WITH A ZONE ALLOWING THE CONTROL OF A REMOTE POINTER PROJECTED BY THIS VIDEOPROJECTOR

Similar Documents

Publication Publication Date Title
JP6301530B2 (en) Function operation method and apparatus of touch device
EP2177976B1 (en) Mobile terminal with image projection
US20180129250A1 (en) Foldable device and method of controlling the same
US10956008B2 (en) Automatic home screen determination based on display device
WO2021244443A1 (en) Split-screen display method, electronic device, and computer readable storage medium
KR101588733B1 (en) Mobile terminal
EP2180676B1 (en) Mobile communication terminal and screen scrolling method thereof
EP3770745B1 (en) Method and device for executing an application
US20160231885A1 (en) Image display apparatus and method
US20110154249A1 (en) Mobile device and related control method for external output depending on user interaction based on image sensing module
US20140204001A1 (en) Method and system for controlling dual-processing of screen data in mobile terminal
US20100138797A1 (en) Portable electronic device with split vision content sharing control and method
US20110151936A1 (en) Input key output method and apparatus of projector-enabled mobile terminal
KR20150110559A (en) Electronic system with display mode mechanism and method of operation thereof
KR20110069526A (en) Method and apparatus for controlling external output of a portable terminal
KR20110070338A (en) Method and apparatus for controlling external output of a portable terminal
KR20080076390A (en) Apparatus and method for displaying of information according to touch event in a portable terminal
EP2214174A2 (en) Apparatus and method for playing of multimedia item
KR101964914B1 (en) Method for performing auto naming for content and apparatus having auto naming function for content, and computer readable recording medium thereof
US8737819B2 (en) Electronic device and media contents reproducing method thereof
US20130222228A1 (en) Automatic projector behaviour changes based on projection distance
WO2009130664A1 (en) System and method for controlling video output
EP2634629B1 (en) Automatic projector behaviour changes based on projection distance
US20160239256A1 (en) Electronic Apparatus and Operation Mode Enabling Method Thereof
CN112463085A (en) Function integrating system and function integrating device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09734513

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09734513

Country of ref document: EP

Kind code of ref document: A1